# pmlogrewrite

## NAME



           **pmlogrewrite** - rewrite Performance Co-Pilot archives
    


## SYNOPSIS



           **pmlogrewrite**[**-Cdiqsvw?**]  [**-c** _config_] [**-D** _debug_] [**-V** _version_]
           _inlog_ [_outlog_]
    


## DESCRIPTION



           **pmlogrewrite** reads a set of Performance Co-Pilot (PCP) archives
           identified by _inlog_ and creates a PCP archive in _outlog_.  Under
           normal usage, the **-c** option will be used to nominate a
           configuration file or files that contains specifications (see the
           **REWRITING RULES SYNTAX** section below) that describe how the data
           and metadata from _inlog_ should be transformed to produce _outlog_.
    
           The typical uses for **pmlogrewrite** would be to accommodate the
           evolution of Performance Metric Domain Agents (PMDAs) where the
           names, metadata and semantics of metrics and their associated
           instance domains may change over time, e.g. promoting the type of
           a metric from a 32-bit to a 64-bit integer, or renaming a group of
           metrics.  Refer to the **EXAMPLES** section for some additional use
           cases.
    
           **pmlogrewrite** may also be used to redact sensitive information from
           PCP archives in situations where the archives need to be shipped
           to another organization or to meet privacy policies or legislative
           requirements.  See pmlogredact(1) for an example use of
           **pmlogrewrite** in this context.
    
           **pmlogrewrite** is most useful where PMDA changes, or errors in the
           production environment, result in archives that cannot be combined
           with pmlogextract(1).  By pre-processing the archives with
           **pmlogrewrite** the resulting archives may be able to be merged with
           pmlogextract(1).
    
           The input _inlog_ must be a set of PCP archives created by
           pmlogger(1), or possibly one of the tools that read and create PCP
           archives, e.g.  pmlogextract(1) and pmlogreduce(1).  _inlog_ is a
           comma-separated list of names, each of which may be the base name
           of an archive or the name of a directory containing one or more
           archives.
    
           If no **-c** option is specified, then the default behavior simply
           creates _outlog_ as a copy of _inlog_.  This is a little more
           complicated than cat(1), as each PCP archive is made up of several
           physical files.
    
           While **pmlogrewrite** may be used to repair some data consistency
           issues in PCP archives, there is also a class of repair tasks that
           cannot be handled by **pmlogrewrite** and pmloglabel(1) may be a
           useful tool in these cases.
    


## OPTIONS



           The available command line options are:
    
           **-c** _config_ , **--config** =_config_
                If _config_ is a file or symbolic link, read and parse
                rewriting rules from there.  If _config_ is a directory, then
                all of the files or symbolic links in that directory
                (excluding those beginning with a period ``.'') will be used
                to provide the rewriting rules.  Multiple **-c** options are
                allowed.
    
           **-C** , **--check**
                Parse the rewriting rules and quit.  _outlog_ is not created,
                so this command line argument is optional with **-C**.  When **-C**
                is specified, this also sets **-v** and **-w** so that all warnings
                and verbose messages are displayed as _config_ is parsed.
    
           **-d** , **--desperate**
                Desperate mode.  Normally if a fatal error occurs, all trace
                of the partially written PCP archive _outlog_ is removed.  With
                the **-d** option, the partially created _outlog_ archive is not
                removed.
    
           **-i** Rather than creating _outlog_ , _inlog_ is rewritten in place when
                the **-i** option is used.  A new archive is created using
                temporary file names and then renamed to _inlog_ in such a way
                that if any errors (not warnings) are encountered, _inlog_
                remains unaltered.
    
           **-q** , **--quick**
                Quick mode, where if there are no rewriting actions to be
                performed (none of the global data, instance domains or
                metrics from _inlog_ will be changed), then **pmlogrewrite** will
                exit (with status 0, so success) immediately after parsing
                the configuration file(s) and _outlog_ is not created.
    
           **-s** , **--scale**
                When the ``units'' of a metric are changed, if the dimension
                in terms of space, time and count is unaltered, then the
                scaling factor is being changed, e.g. BYTE to KBYTE, or
                MSEC-1 to USEC-1, or the composite MBYTE.SEC-1 to
                KBYTE.USEC-1.  The motivation may be (a) that the original
                metadata was wrong but the values in _inlog_ are correct, or
                (b) the metadata is changing so the values need to change as
                well.  The default **pmlogrewrite** behaviour matches case (a).
                If case (b) applies, then use the **-s** option and the values of
                all the metrics with a scale factor change in each result
                will be rescaled.  For finer control over value rescaling
                refer to the _**RESCALE** _option for the _**UNITS** _clause of the
                metric rewriting rule described below.
    
           **-v** , **--verbose**
                Enable verbose mode.
    
           **-V** _version_ , **--version** =_version_
                Specifies the version of the output PCP archive being
                produced.  Currently versions 2 and 3 of the archive format
                is supported.  The version of _inlog_ must be at least _version_
                (so version upgrade is allowed, but version downgrade is
                not).  By default, in the absence of the _-V_ option, the
                version of _outlog_ is the same as the version of _inlog_.
    
           **-w** , **--warnings**
                Emit warnings.  Normally **pmlogrewrite** remains silent for any
                warning that is not fatal and it is expected that for a
                particular archive, some (or indeed, all) of the rewriting
                specifications may not apply.  For example, changes to a PMDA
                may be captured in a set of rewriting rules, but a single
                archive may not contain all of the modified metrics nor all
                of the modified instance domains and/or instances.  Because
                these cases are expected, they do not prevent **pmlogrewrite**
                executing, and rules that do not apply to _inlog_ are silently
                ignored by default.  Similarly, some rewriting rules may
                involve no change because the metadata in _inlog_ already
                matches the intent of the rewriting rule to correct data from
                a previous version of a PMDA.  The **-w** flag forces warnings to
                be emitted for all of these cases.
    
           **-?** , **--help**
                Display usage message and exit.
    
           The argument _outlog_ is required in all cases, except when **-i** is
           specified.
    


## REWRITING RULES SYNTAX



           A configuration file contains zero or more rewriting rules as
           defined below.
    
           Keywords and special punctuation characters are shown below in
           _**bolditalic** _font and are case-insensitive, so _**METRIC**_ , _**metric** _and
           _**Metric** _are all equivalent in rewriting rules.
    
           The character ``#'' introduces a comment and the remainder of the
           line is ignored.  Otherwise the input is relatively free format
           with optional white space (spaces, tabs or newlines) between
           lexical items in the rules.
    
           A **global** rewriting rule has the form:
    
           _**GLOBAL {** globalspec_ ...  **}**
    
           where _globalspec_ is zero or more of the following clauses:
    
               _**HOSTNAME - > **hostname_
    
                   Modifies the label records in the _outlog_ PCP archive, so
                   that the metrics will appear to have been collected from
                   the host _hostname_.
    
               _**TIME - > **delta_
    
                   Both metric values and the instance domain metadata in a
                   PCP archive carry timestamps.  This clause forces all the
                   timestamps to be adjusted by _delta_ , where _delta_ is an
                   optional sign ``+'' (the default) or ``-'', an optional
                   number of hours followed by a colon ``:'', an optional
                   number of minutes followed by a colon ``:'', a number of
                   seconds, an optional fraction of seconds following a
                   period ``.''.  The simplest example would be ``30'' to
                   increase the timestamps by 30 seconds.  A more complex
                   example would be ``-23:59:59.999'' to move the timestamps
                   backwards by one millisecond less than one day.
    
               **TIMEZONE - > "**timezone**"**
               **TZ - > "**timezone**"**
    
                   Modifies the label records in the _outlog_ PCP archive, so
                   that the metrics will appear to have been collected from a
                   host with a local timezone of _timezone_.  _timezone_ must be
                   enclosed in double quotes, and should conform to the valid
                   timezone syntax rules for the local platform, usually a
                   Posix TZ format, e.g. **AEST-10**.  See tzset(3) for more
                   information.
    
                   _**TZ** _is an alias for _**TIMEZONE**_.
    
               **ZONEINFO - > "**zoneinfo**"**
    
                   Modifies the label records in the _outlog_ PCP archive, so
                   that the metrics will appear to have been collected from a
                   host with a local timezone of _zoneinfo_.  _zoneinfo_ must be
                   enclosed in double quotes, and should conform to the valid
                   zoneinfo timezone syntax rules for the local platform,
                   usually a colon followed by a pathname below
                   _/usr/share/zoneinfo_ , e.g. **:Africa/Timbuktu**.  See tzset(3)
                   for more information.
    
                   The _**zoneinfo** _clause is only allowed if the output archive
                   version is at least 3.
    
               _**FEATURES - > **feature-bits_
    
                   Modifies the label records in the _outlog_ PCP archive, so
                   that the metrics will appear to have been collected from
                   system with a pmlogger(1) that supports the ``features''
                   defined by the integer value _feature-bits_ , which is formed
                   by ``or''ing the desired feature flags as defined in
                   LOGARCHIVE(5).  Alternatively, _feature-bits_ can be
                   specified using the ``macro'' _**BITS()** _that takes a comma
                   separated argument list of integers (in the inclusive
                   range 0 to 31) and sets the corresponding bits.  For
                   example
                       features -> bits(31,7,1)
    
                   The _**features** _clause is only allowed if the output archive
                   version is at least 3.
    
           An **indom** rewriting rule modifies an instance domain and has the
           form:
    
           _**INDOM** domain**.** serial **{** indomspec_ ...  **}**
    
           where _domain_ and _serial_ identify one or more existing instance
           domains from _inlog_ - typically _domain_ would be an integer in the
           range 1 to 510 and _serial_ would be an integer in the range 0 to
           4194304.
    
           As a special case _serial_ could be an asterisk ``*'' which means
           the rule applies to every instance domain with a domain number of
           _domain_.
    
           If a designated instance domain is not in _inlog_ the rule has no
           effect.
    
           The _indomspec_ is zero or more of the following clauses:
    
               _**INAME** _"_oldname_ " _**- >** _"_newname_ "
    
                   The instance identified by the external instance name
                   _oldname_ is renamed to _newname_.  Both _oldname_ and _newname_
                   must be enclosed in double quotes.
    
                   As a special case, the new name may be the keyword **DELETE**
                   (with no double quotes), and then the instance _oldname_
                   will be expunged from _outlog_ which removes it from the
                   instance domain metadata and removes all values of this
                   instance for all the associated metrics.
    
                   If the instance names contain any embedded spaces then
                   special care needs to be taken in respect of the PCP
                   instance naming rule that treats the leading non-space
                   part of the instance name as the unique portion of the
                   name for the purposes of matching and ensuring uniqueness
                   within an instance domain, refer to pmdaInstance(3) for a
                   discussion of this issue.
    
                   As an illustration, consider the hypothetical instance
                   domain for a metric which contains 2 instances with the
                   following names:
                       red
                       eek urk
    
                   Then some possible _**INAME** _clauses might be:
    
                   "eek" -> "yellow like a flower"
                          Acceptable, _oldname_ "eek" matches the "eek urk"
                          instance.
    
                   "red" -> "eek"
                          Error, _newname_ "eek" matches the existing "eek urk"
                          instance.
    
                   "eek urk" -> "red of another hue"
                          Error, _newname_ "red of another hue" matches the
                          existing "red" instance.
    
               _**INAME REPLACE** _/_pattern_ / _**- >** _"_replacement_ "
    
                   Every external instance name in the instance domain is
                   matched against the regular expression _pattern_ and when a
                   match is found, the name is changed based on the parts of
                   the name matched by _pattern_ and the _replacement_ recipe.
                   _pattern_ follows the syntax of a Posix Extended Regular
                   Expression (see regex(7)) and _replacement_ follows the
                   syntax of the **s** command of sed(1), so **& **and **\1** through **\9**
                   may be used to select all or substrings of the instance
                   name that matches _pattern._
    
                   Note that the match-and-replace is done at most once per
                   external instance name, so if there are repeated sequences
                   of the name that match _pattern_ only the **first** one will be
                   matched and _replacement_ applied.
    
                   _pattern_ is normally enclosed in slashes (/) or double
                   quotes (").  An escape (\) may be used before any of the
                   metacharacters described in regex(3) to specify a literal
                   character, e.g. \( or \[ or \+ ...  The enclosing
                   delimiters cannot be escaped, so to embed a literal slash,
                   use double quotes as the delimiter, or vice versa.
    
                   _replacement_ is normally enclosed by double quotes (") or
                   slashes (/).  An escape (\) may be used before **& **or **\** to
                   specify these characters literally.  The enclosing
                   delimiters cannot be escaped, so to embed a double quote,
                   use slashes as the delimiter, or vice versa.
    
                   If the instance names after replacement contain any
                   embedded spaces then special care needs to be taken in
                   respect of the PCP instance naming rule that treats the
                   leading non-space part of the instance name as the unique
                   portion of the name for the purposes of matching and
                   ensuring uniqueness within an instance domain.  Refer to
                   pmdaInstance(3) for a discussion of this issue.
    
                   Here are some examples:
    
                   { iname replace /[a-z]*foo[a-z]*/ -> "FOO" }
                          replace any word containing "foo" with "FOO"
    
                   { iname replace "([0-9]+) /.*/(.*)" -> "\1 \2" }
                          removes a directory path, so the instance name (for
                          one of the proc PMDA's metrics) "2981799
                          /home/kenj/bin/foobar" would become "2981799
                          foobar", hiding the user's home directory and
                          implicitly the user's login name
    
               **INAME REDACT**
    
                   Replace every external instance name in the instance
                   domain by the string "_inst_ **[redacted]** " where _inst_ is the
                   internal instance identifier in ASCII format.
    
               _**INDOM - > **newdomain**.** newserial_
    
                   Modifies the metadata for the instance domain and every
                   metric associated with the instance domain.  As a special
                   case, _newserial_ could be an asterisk ``*'' which means use
                   _serial_ from the **indom** rewriting rule, although this is
                   most useful when _serial_ is also an asterisk.  So for
                   example:
                       indom 29.* { indom -> 109.* }
                   will move all instance domains from domain 29 to domain
                   109.
    
               _**INDOM - > DUPLICATE **newdomain**.** newserial_
    
                   A special case of the previous _**INDOM** _clause where the
                   instance domain is a duplicate copy of the _domain**.** serial_
                   instance domain from the _indom_ rewriting rule, and then
                   any mapping rules are applied to the copied
                   _newdomain**.** newserial_ instance domain.  This is useful when
                   a PMDA is split and the same instance domain needs to be
                   replicated for domain _domain_ and domain _newdomain_.  So for
                   example if the metrics _foo.one_ and _foo.two_ are both
                   defined over instance domain 12.34, and _foo.two_ is moved
                   to another PMDA using domain 27, then the following
                   rewriting rules could be used:
                       indom 12.34 { indom -> duplicate 27.34 }
                       metric foo.two { indom -> 27.34 pmid -> 27.*.*  }
    
               _**INST** oldid **- > **newid_
    
                   The instance identified by the internal instance
                   identifier _oldid_ is renumbered to _newid_.  Both _oldid_ and
                   _newid_ are integers in the range 0 to 231-1.
    
                   As a special case, _newid_ may be the keyword _**DELETE** _and
                   then the instance _oldid_ will be expunged from _outlog_ which
                   removes it from the instance domain metadata and removes
                   all values of this instance for all the associated
                   metrics.
    
           A **metric** rewriting rule has the form:
    
           _**METRIC** metricid **{** metricspec_ ...  **}**
    
           where _metricid_ identifies one or more existing metrics from _inlog_
           using either a metric name, or the internal encoding for a
           metric's PMID as _domain**.** cluster**.** item_.  In the latter case,
           typically _domain_ would be an integer in the range 1 to 510,
           _cluster_ would be an integer in the range 0 to 4095, and _item_ would
           be an integer in the range 0 to 1023.
    
           As special cases _item_ could be an asterisk ``*'' which means the
           rule applies to every metric with a domain number of _domain_ and a
           cluster number of _cluster_ , or _cluster_ could be an asterisk which
           means the rule applies to every metric with a domain number of
           _domain_ and an item number of _item_ , or both _cluster_ and _item_ could
           be asterisks, and rule applies to every metric with a domain
           number of _domain_.
    
           If a designated metric is not in _inlog_ the rule has no effect.
    
           The _metricspec_ is zero or more of the following clauses:
    
               **DELETE**
    
                   The metric is completely removed from _outlog_ , both the
                   metadata and all values in results are expunged.
    
               _**INDOM - > **newdomain**.** newserial_ [ _pick_ ]
    
                   Modifies the metadata to change the instance domain for
                   this metric.  The new instance domain must exist in
                   _outlog_.
    
                   The optional _pick_ clause may be used to select one input
                   value, or compute an aggregate value from the instances in
                   an input result, or assign an internal instance identifier
                   to a single output value.  If no _pick_ clause is specified,
                   the default behaviour is to copy all input values from
                   each input result to an output result, however if the
                   input instance domain is singular (indom **PM_INDOM_NULL**)
                   then the one output value must be assigned an internal
                   instance identifier, which is 0 by default, unless over-
                   ridden by a _**INST** _or _**INAME** _clause as defined below.
    
                   The choices for _pick_ are as follows:
    
                   **OUTPUT FIRST**
                          choose the value of the first instance from each
                          input result
    
                   **OUTPUT LAST**
                          choose the value of the last instance from each
                          input result
    
                   _**OUTPUT INST** instid_
                          choose the value of the instance with internal
                          instance identifier _instid_ from each result; the
                          sequence of rewriting rules ensures the **OUTPUT**
                          processing happens before instance identifier
                          renumbering from any associated **indom** rule, so
                          _instid_ should be one of the internal instance
                          identifiers that appears in _inlog_
    
                   _**OUTPUT INAME** _"_name_ "
                          choose the value of the instance with _name_ for its
                          external instance name from each result; the
                          sequence of rewriting rules ensures the **OUTPUT**
                          processing happens before instance renaming from
                          any associated **indom** rule, so _name_ should be one of
                          the external instance names that appears in _inlog_
    
                   **OUTPUT MIN**
                          choose the smallest value in each result (metric
                          type must be numeric and output instance will be 0
                          for a non-singular instance domain)
    
                   **OUTPUT MAX**
                          choose the largest value in each result (metric
                          type must be numeric and output instance will be 0
                          for a non-singular instance domain)
    
                   **OUTPUT SUM**
                          choose the sum of all values in each result (metric
                          type must be numeric and output instance will be 0
                          for a non-singular instance domain)
    
                   **OUTPUT AVG**
                          choose the average of all values in each result
                          (metric type must be numeric and output instance
                          will be 0 for a non-singular instance domain)
    
                   If the input instance domain is singular (indom
                   **PM_INDOM_NULL**) then independent of any _pick_
                   specifications, there is at most one value in each input
                   result and so _**FIRST**_ , _**LAST**_ , _**MIN**_ , _**MAX**_ , _**SUM** _and _**AVG** _are all
                   equivalent and the output instance identifier will be 0.
    
                   In general it is an error to specify a rewriting action
                   for the same metadata or result values more than once,
                   e.g. more than one _**INDOM** _clause for the same instance
                   domain.  The one exception is the possible interaction
                   between the _**INDOM** _clauses in the **indom** and **metric** rules.
                   For example the metric _sample.bin_ is defined over the
                   instance domain 29.2 in _inlog_ and the following is
                   acceptable (albeit redundant):
                       indom 29.* { indom -> 109.* }
                       metric sample.bin { indom -> 109.2 }
                   However the following is an error, because the instance
                   domain for _sample.bin_ has two conflicting definitions:
                       indom 29.* { indom -> 109.* }
                       metric sample.bin { indom -> 123.2 }
    
               _**INDOM - > NULL**_[ _pick_ ]
    
                   The metric (which must have been previously defined over
                   an instance domain) is being modified to be a singular
                   metric.  This involves a metadata change and collapsing
                   all results for this metric so that multiple values become
                   one value.
    
                   The optional _pick_ part of the clause defines how the one
                   value for each result should be calculated and follows the
                   same rules as described for the non-NULL _**INDOM** _case above.
    
                   In the absence of _pick_ , the default is _**OUTPUT FIRST**_.
    
               _**NAME - > **newname_
    
                   Renames the metric in the PCP archive's metadata that
                   supports the Performance Metrics Name Space (PMNS).
                   _newname_ should not match any existing name in the
                   archive's PMNS and must follow the syntactic rules for
                   valid metric names as outlined in PMNS(5).
    
               _**PMID - > **newdomain**.** newcluster**.** newitem_
    
                   Modifies the metadata and results to renumber the metric's
                   PMID.  As special cases, _newcluster_ could be an asterisk
                   ``*'' which means use _cluster_ from the **metric** rewriting
                   rule and/or _item_ could be an asterisk which means use _item_
                   from the **metric** rewriting rule.  This is most useful when
                   _cluster_ and/or _item_ is also an asterisk.  So for example:
                       metric 30.*.* { pmid -> 123.*.* }
                   will move all metrics from domain 30 to domain 123.
    
               _**SEM - > **newsem_
    
                   Change the semantics of the metric.  _newsem_ should be the
                   XXX part of the name of one of the **PM_SEM_XXX** macros
                   defined in <pcp/pmapi.h> or pmLookupDesc(3), e.g.  **COUNTER**
                   for **PM_TYPE_COUNTER**.
    
                   No data value rewriting is performed as a result of the
                   _**SEM** _clause, so the usefulness is limited to cases where a
                   version of the associated PMDA was exporting incorrect
                   semantics for the metric.  pmlogreduce(1) may provide an
                   alternative in cases where re-computation of result values
                   is desired.
    
               _**TYPE - > **newtype_
    
                   Change the type of the metric which alters the metadata
                   and may change the encoding of values in results.  _newtype_
                   should be the XXX part of the name of one of the
                   **PM_TYPE_XXX** macros defined in <pcp/pmapi.h> or
                   pmLookupDesc(3), e.g.  _**FLOAT** _for **PM_TYPE_FLOAT**.
    
                   Type conversion is only supported for cases where the old
                   metric type is numeric, so **PM_TYPE_STRING** ,
                   **PM_TYPE_AGGREGATE** , **PM_TYPE_EVENT** and **PM_TYPE_HIGHRES_EVENT**
                   are not allowed.  Similarly, the new metric type must be
                   numeric or **PM_TYPE_STRING**.  Even for the numeric cases,
                   some conversions may produce run-time errors, e.g. integer
                   overflow, or attempting to rewrite a negative value into
                   an unsigned type, or imprecision, e.g. when converting a
                   float or a double to a string.
    
               _**TYPE IF** oldtype **- > **newtype_
    
                   The same as the preceding _**TYPE** _clause, except the type of
                   the metric is only changed to _newtype_ if the type of the
                   metric in _inlog_ is _oldtype._
    
                   This useful in cases where the type of _metricid_ in _inlog_
                   may be platform dependent and so more than one type
                   rewriting rule is required.
    
               _**UNITS - > **newunits_ [ _**RESCALE** _]
    
                   _newunits_ is six values separated by commas.  The first 3
                   values describe the dimension of the metric along the
                   dimensions of space, time and count; these are integer
                   values, usually 0, 1 or -1.  The remaining 3 values
                   describe the scale of the metric's values in the
                   dimensions of space, time and count.  Space scale values
                   should be 0 (if the space dimension is 0), else the XXX
                   part of the name of one of the **PM_SPACE_XXX** macros, e.g.
                   _**KBYTE** _for **PM_TYPE_KBYTE**.  Time scale values should be 0
                   (if the time dimension is 0), else the XXX part of the
                   name of one of the **PM_TIME_XXX** macros, e.g.  _**SEC** _for
                   **PM_TIME_SEC**.  Count scale values should be 0 (if the time
                   dimension is 0), else _**ONE** _for **PM_COUNT_ONE**.
    
                   The **PM_SPACE_XXX** , **PM_TIME_XXX** and **PM_COUNT_XXX** macros are
                   defined in <pcp/pmapi.h> or pmLookupDesc(3).
    
                   When the scale is changed (but the dimension is unaltered)
                   the optional keyword _**RESCALE** _may be used to chose value
                   rescaling as per the **-s** command line option, but applied
                   to just this metric.
    
               _**VALUE REPLACE** _/_pattern_ / _**- >** _"_replacement_ "
    
                   The value for every instance of the metric is matched
                   against the regular expression _pattern_ and when a match is
                   found, the value is changed based on the parts of the
                   value matched by _pattern_ and the _replacement_ recipe.
                   _pattern_ follows the syntax of a Posix Extended Regular
                   Expression (see regex(7)) and _replacement_ follows the
                   syntax of the **s** command of sed(1), so **& **and **\1** through **\9**
                   may be used to select all or substrings of the value that
                   matches _pattern._
    
                   Note that the match-and-replace is done at most once per
                   metric value, so if there are repeated sequences of the
                   metric value that match _pattern_ only the **first** one will be
                   matched and _replacement_ applied.
    
                   _pattern_ is normally enclosed in slashes (/) or double
                   quotes (").  An escape (\) may be used before any of the
                   metacharacters described in regex(3) to specify a literal
                   character, e.g. \( or \[ or \+ ...  The enclosing
                   delimiters cannot be escaped, so to embed a literal slash,
                   use double quotes as the delimiter, or vice versa.
    
                   _replacement_ is normally enclosed by double quotes (") or
                   slashes (/).  An escape (\) may be used before **& **or **\** to
                   specify these characters literally.  The enclosing
                   delimiters cannot be escaped, so to embed a double quote,
                   use slashes as the delimiter, or vice versa.
    
                   The _**REPLACE** _keyword is optional.
    
                   This clause can only be applied to metrics with values of
                   type **PM_TYPE_STRING**.
    
                   Here are some examples:
    
                   { value /.*/ -> "" }
                          remove the value everywhere
    
                   value "/.*/(.*)" -> "\1" }
                          removes a directory path, so the metric value
                          "mumble /home/kenj/bin/foobar fumble" would become
                          "mumble foobar fumble", hiding the user's home
                          directory and implicitly the user's login name
    
               When changing the domain number for a metric or instance
               domain, the new domain number will usually match an existing
               PMDA's domain number.  If this is not the case, then the new
               domain number should not be randomly chosen; consult
               **$PCP_VAR_DIR/pmns/stdpmid** for domain numbers that are already
               assigned to PMDAs.
    
           A **text** rewriting rule modifies a help text record and has the
           form:
    
           _**TEXT** textid_ [ _texttype_ ] [ "_textcontent_ " ] _**{** textspec_ ...  **}**
    
           where _textid_ identifies the metric or instance domain with which
           the text is currently associated, and is either _**METRIC** metricid_ or
           _**INDOM** domain**.** serial_.
    
           _metricid_ has the same form and meaning as for a _**METRIC** _rewriting
           rule (see above) and _domain**.** serial_ has the same form and meaning
           as for an _**INDOM** _rewriting rule (see above).
    
           The optional _texttype_ identifies the type of text and may be one
           of _**ONELINE** _to select the one line help text, _**HELP** _to select the
           full help text, or _**ALL** _or an asterisk ``*'' to select both types
           of help text.  If _texttype_ is not specified, then the default is
           _**ONELINE**_.
    
           The optional _textcontent_ further restricts the selected text
           records to those containing the specified content.  Characters
           such as double quotes may be escaped by preceding them with a
           backslash ``\''.
    
           If a designated help text record is not in _inlog_ the rule has no
           effect.
    
           The _textspec_ is zero or more of the following clauses:
    
               **DELETE**
    
                   The selected text is completely removed from _outlog_.
    
               _**INDOM - > **newdomain**.** newserial_
    
                   Re-associates the text with the specified instance domain.
                   As a special case, _newserial_ could be an asterisk ``*''
                   which means use _serial_ from the **text** rewriting rule,
                   although this is most useful when _serial_ is also an
                   asterisk.  So for example:
                       text indom 29.* all { indom -> 109.* }
                   will re-associate all text associated with instance
                   domains from domain 29 to domain 109.
    
               _**METRIC - > **newdomain**.** newcluster**.** newitem_
    
                   Re-associates the text with the specified metric.  As
                   special cases, _newcluster_ could be an asterisk ``*'' which
                   means use _cluster_ from the **text** rewriting rule and/or _item_
                   could be an asterisk which means use _item_ from the **text**
                   rewriting rule.  This is most useful when _cluster_ and/or
                   _item_ is also an asterisk.  So for example:
                       text metric 30.*.* all { metric -> 123.*.* }
                   will re-associate all text associated with metrics from
                   domain 30 to domain 123.
    
               **TEXT - > "**new-text**"**
    
                   Replaces the content of the selected text with _new-text_.
    
           A **label** rewriting rule modifies a label record and has the form:
    
           _**LABEL** labelid_ [ _instance_ ] [ **"** label-name[1m"[24m [22m] [ **"** label-value[1m"[24m [22m] **{**
           _labelspec_ ...  **}**
    
           where _labelid_ refers to the global context or identifies the
           metric domain, metric cluster, metric item, instance domain, or
           instance domain instances with which the label is currently
           associated, and is either _**CONTEXT** _or _**DOMAIN** domainid_ or **CLUSTER**
           _domainid**.** clusterid_ or _**ITEM** metricid_ or _**INDOM** domain**.** serial_ or
           _**INSTANCES** domain**.** serial_.
    
           _metricid_ has the same form and meaning as for a _**METRIC** _rewriting
           rule (see above).  _clusterid_ may be an asterisk ``*'' which means
           the rule applies to every metric with a domain number of _domainid_
           in the same way as an asterisk may be used for the cluster within
           _metricid_.
    
           _domain**.** serial_ has the same form and meaning as for an **INDOM**
           rewriting rule (see above).
    
           In the case of an _**INSTANCES** labelid_, the name or number of a
           specific instance may be optionally specified as _instance_.  This
           name or number number may be omitted or specified as an asterisk
           ``*'' to indicate that labels for all instances of the specified
           instance domain are selected.  If an instance name is specified,
           it must be within double quotes.  If the instance name contains
           any embedded spaces then special care needs to be taken in respect
           of the PCP instance naming rule that treats the leading non-space
           part of the instance name as the unique portion of the name for
           the purposes of matching and ensuring uniqueness within an
           instance domain, refer to pmdaInstance(3) for a discussion of this
           issue.
    
           In all cases, a **"** label-name[1m"[24m [22mand/or a **"** label-value[1m"[24m [22mmay be
           optionally specified in double quotes in order to select labels
           with the given name and/or given value.  These may individually be
           omitted or specified as asterisks ``*'' to indicate that labels
           with all names and/or values are selected.
    
           If a designated label record is not in _inlog_ the rule has no
           effect.
    
           The _labelspec_ is zero or more of the following clauses:
    
               **DELETE**
    
                   The selected labels are completely removed from _outlog_.
    
               **NEW "** new-label-name**" "** new-label-value**"**
    
                   A new label with the name **"** new-label-name[1m"[24m [22mand the value
                   **"** new-label-value[1m"[24m [22mis created and associated with the
                   specified _labelid_ and optional _instance_ (in the case of a
                   _**INSTANCES** labelid_).  If **"** label-name[1m"[24m [22mor **"** label-value[1m"[24m [22mwere
                   specified, then they are ignored with a warning.  If
                   _instance_ is not specified for an _**INSTANCES** labelid_, then a
                   new label will be created for each instance in the
                   specified instance domain.
    
               **LABEL - > "**new-label-name**"**
    
                   The name of the selected label(s) is changed to _**"** new-_
                   label-name[1m"[24m[22m.
    
               **VALUE - > "**new-label-value**"**
    
                   The value of the selected label(s) is changed to _**"** new-_
                   label-value[1m"[24m[22m.
    
               _**DOMAIN - > **newdomain_
    
                   Re-associates the selected label(s) with the specified
                   metric domain.  For example:
                       label domain 30 { domain -> 123 }
                   will re-associate all labels associated with domains from
                   domain 30 to domain 123.
    
               _**CLUSTER - > **newdomain**.** newcluster_
    
                   Re-associates the selected label(s) with the specified
                   metric cluster.  As a special case, _newcluster_ could be an
                   asterisk ``*'' which means use _cluster_ from the **label**
                   rewriting rule.  This is most useful when _cluster_ is also
                   an asterisk.  So for example:
                       label cluster 30.* { cluster -> 123.* }
                   will re-associate all labels associated with clusters from
                   domain 30 to domain 123.
    
               _**ITEM - > **newdomain**.** newcluster**.** newitem_
    
                   Re-associates the selected label(s) with the specified
                   metric item.  As special cases, _newcluster_ could be an
                   asterisk ``*'' which means use _cluster_ from the **label**
                   rewriting rule and/or _item_ could be an asterisk which
                   means use _item_ from the **label** rewriting rule.  This is
                   most useful when _cluster_ and/or _item_ is also an asterisk.
                   So for example:
                       label item 30.*.* { item -> 123.*.* }
                   will re-associate all labels associated with metrics from
                   domain 30 to domain 123.
    
               _**INDOM - > **newdomain**.** newserial_
    
                   Re-associates the selected label(s) with the specified
                   instance domain.  As a special case, _newserial_ could be an
                   asterisk ``*'' which means use _serial_ from the **label**
                   rewriting rule, although this is most useful when _serial_
                   is also an asterisk.  So for example:
                       label indom 29.* { indom -> 109.* }
                   will re-associate all labels associated with instance
                   domains from domain 29 to domain 109.
    
               _**INSTANCES - > **newdomain**.** newserial_
    
                   This is the same as _**INDOM** _except that it re-associates the
                   selected label(s) with the instances of the specified
                   instance domain.
    


## EXAMPLES



           To promote the values of the per-disk IOPS metrics to 64-bit to
           allow aggregation over a long time period for capacity planning,
           or because the PMDA has changed to export 64-bit counters and we
           want to convert old archives so they can be processed alongside
           new archives.
               metric disk.dev.read { type -> U64 }
               metric disk.dev.write { type -> U64 }
               metric disk.dev.total { type -> U64 }
    
           The instances associated with the load average metric
           **kernel.all.load** could be renamed and renumbered by the rules
           below.
               # for the Linux PMDA, the kernel.all.load metric is defined
               # over instance domain 60.2
               indom 60.2 {
                   inst 1 -> 60 iname "1 minute" -> "60 second"
                   inst 5 -> 300 iname "5 minute" -> "300 second"
                   inst 15 -> 900 iname "15 minute" -> "900 second"
               }
    
           If we decide to split the ``proc'' metrics out of the Linux PMDA,
           this will involve changing the domain number for the PMID of these
           metrics and the associated instance domains.  The rules below
           would rewrite an old archive to match the changes after the PMDA
           split.
               # all Linux proc metrics are in 7 clusters
               metric 60.8.* { pmid -> 123.*.* }
               metric 60.9.* { pmid -> 123.*.* }
               metric 60.13.* { pmid -> 123.*.* }
               metric 60.24.* { pmid -> 123.*.* }
               metric 60.31.* { pmid -> 123.*.* }
               metric 60.32.* { pmid -> 123.*.* }
               metric 60.51.* { pmid -> 123.*.* }
               # only one instance domain for Linux proc metrics
               indom 60.9 { indom -> 123.0 }
    
           If the metric foo.count_em was exported as a native ``long'' then
           it could be a 32-bit integer on some platforms and a 64-bit
           integer on other platforms.  Subsequent investigations show the
           value is in fact unsigned, so the following rules could be used.
               metric foo.count_em {
                    type if 32 -> U32
                    type if 64 -> U64
               }
    


## DIAGNOSTICS



           All error conditions detected by **pmlogrewrite** are reported on
           _stderr_ with textual (if sometimes terse) explanation.
    
           Should the input archive be corrupted (this can happen if the
           **pmlogger** instance writing the archive suddenly dies), then
           **pmlogrewrite** will detect and report the position of the corruption
           in the file, and any subsequent information from that archive will
           not be processed.
    
           If the input archive contains no archive records then an ``empty
           archive'' warning is issued and no processing is performed.
    
           If any error is detected, **pmlogrewrite** will exit with a non-zero
           status.
    


## FILES



           For each of the _inlog_ and _outlog_ archives, several physical files
           are used.
    
           _archive_**.meta**
                metadata (metric descriptions, instance domains, etc.) for
                the archive
    
           _archive_**.0**
                initial volume of metrics values (subsequent volumes have
                suffixes **1** , **2** , ...).
    
           _archive_**.index**
                temporal index to support rapid random access to the other
                files in the archive.
    


## PCP ENVIRONMENT



           Environment variables with the prefix **PCP_** are used to
           parameterize the file and directory names used by PCP.  On each
           installation, the file _/etc/pcp.conf_ contains the local values for
           these variables.  The **$PCP_CONF** variable may be used to specify an
           alternative configuration file, as described in pcp.conf(5).
    
           For environment variables affecting PCP tools, see
           pmGetOptions(3).
    


## DEBUGGING OPTIONS



           The **-D** or **--debug** option enables the output of additional
           diagnostics on _stderr_ to help triage problems, although the
           information is sometimes cryptic and primarily intended to provide
           guidance for developers rather end-users.  _debug_ is a comma
           separated list of debugging options; use pmdbg(1) with the **-l**
           option to obtain a list of the available debugging options and
           their meaning.
    
           Debugging options specific to **pmlogrewrite** are as follows:
           ┌────────┬───────────────────────────────────────────────────────┐
           │ **Option** │ **Description** │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl0** │ archive reads and writes for data and metadata        │
           │        │ volumes                                               │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl1** │ metadata changes (metric descriptors, instance        │
           │        │ domains, help text, metric labels)                    │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl2** │ metric value (**pmResult**) changes                       │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl3** │ **-q** handling and explanation of required changes that  │
           │        │ are the reason for not taking a quick exit            │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl4** │ _config_ file parser diagnostics                        │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl5** │ regex(7) matching for metric value changes and        │
           │        │ instance name changes                                 │
           ├────────┼───────────────────────────────────────────────────────┤
           │ **appl6** │ lexical scanner called from _config_ file parser        │
           └────────┴───────────────────────────────────────────────────────┘
    


* * *