# pcp2arrow

## NAME



           **pcp2arrow** - pcp-to-arrow metrics exporter
    


## SYNOPSIS



           **pcp2arrow**[**-jLnrRVz?**]  [**-8** |**-9** _limit_] [**-a** _archive_] [**-A** _align_]
           [**--archive-folio** _folio_] [**-c** _config_] [**--container** _container_] [**-D**
           _debug_] [**-h** _host_] [**-i** _instances_] [**-J** _rank_] [**-K** _spec_] [**-o** _outfile_]
           [**-O** _origin_] [**-s** _samples_] [**-S** _starttime_] [**-t** _interval_] [**-T** _endtime_]
           [**-Z** _timezone_] [_metricspec_...]
    


## DESCRIPTION



           **pcp2arrow** is a customizable performance metrics exporter tool from
           PCP to Apache Arrow.  It is particularly useful as a mechanism for
           producing the Parquet columnar data format, for use with Pandas or
           similar data analysis modules.  Each PCP metric, and each instance
           of each metric, will form a unique column named according to the
           PCP metric specification - that is, metric name followed by square
           bracket enclosed instance name (for metrics with an instance
           domain).
    
           Any available performance metric, live or archived, system and/or
           application, can be selected for exporting using either command
           line arguments or a configuration file.
    
           With no _metricspec_ options, all available metrics are considered
           for exporting.
    
           **pcp2arrow** is a close relative of pmrep(1).  Refer to pmrep(1) for
           the _metricspec_ description accepted on **pcp2arrow** command line.
           See pmrep.conf(5) for description of the **pcp2arrow.conf**
           configuration file syntax.  This page describes **pcp2arrow** specific
           options and configuration file differences with pmrep.conf(5).
           pmrep(1) also lists some usage examples of which most are
           applicable with **pcp2arrow** as well.
    
           Only the command line options listed on this page are supported,
           other options available for pmrep(1) are not supported.
    
           Options via environment values (see pmGetOptions(3)) override the
           corresponding built-in default values (if any).  Configuration
           file options override the corresponding environment variables (if
           any).  Command line options override the corresponding
           configuration file options (if any).
    


## CONFIGURATION FILE



           **pcp2arrow** uses a configuration file with syntax described in
           pmrep.conf(5).  The following options are common with **pmrep.conf** :
           **version** , **source** , **speclocal** , **derived** , **header** , **globals** , **samples** ,
           **interval** , **type** , **type_prefer** , **ignore_incompat** , **names_change** ,
           **instances** , **live_filter** , **rank** , **limit_filter** , **limit_filter_force** ,
           **invert_filter** , **predicate** , **omit_flat** , **include_labels** , **precision** ,
           **precision_force** , **count_scale** , **count_scale_force** , **space_scale** ,
           **space_scale_force** , **time_scale** , **time_scale_force**.  The rest of the
           **pmrep.conf** options are recognized but ignored for compatibility.
    


## OPTIONS



           The available command line options are:
    
           **-8** _limit_ , **--limit-filter** =_limit_
                Limit results to instances with values above/below _limit_.  A
                positive integer will include instances with values at or
                above the limit in reporting.  A negative integer will
                include instances with values at or below the limit in
                reporting.  A value of zero performs no limit filtering.
                This option will _not_ override possible per-metric
                specifications.  See also **-J** and **-N**.
    
           **-9** _limit_ , **--limit-filter-force** =_limit_
                Like **-8** but this option _will_ override per-metric
                specifications.
    
           **-a** _archive_ , **--archive** =_archive_
                Performance metric values are retrieved from the set of
                Performance Co-Pilot (PCP) archive files identified by the
                _archive_ argument, which is a comma-separated list of names,
                each of which may be the base name of an archive or the name
                of a directory containing one or more archives.
    
           **-A** _align_ , **--align** =_align_
                Force the initial sample to be aligned on the boundary of a
                natural time unit _align_.  Refer to PCPIntro(1) for a complete
                description of the syntax for _align_.
    
           **--archive-folio** =_folio_
                Read metric source archives from the PCP archive _folio_
                created by tools like pmchart(1) or, less often, manually
                with mkaf(1).
    
           **-c** _config_ , **--config** =_config_
                Specify the _config_ file or directory to use.  In case _config_
                is a directory all files in it ending **.conf** will be included.
                The default is the first found of: _./pcp2arrow.conf_ ,
                $HOME _/.pcp2arrow.conf_ , $HOME _/pcp/pcp2arrow.conf_ , and
                $PCP_SYSCONF_DIR _/pcp2arrow.conf_.  For details, see the above
                section and pmrep.conf(5).
    
           **--container** =_container_
                Fetch performance metrics from the specified _container_ ,
                either local or remote (see **-h**).
    
           **-C** , **--check**
                Exit before reporting any values, but after parsing the
                configuration and metrics and printing possible headers.
    
           **-h** _host_ , **--host** =_host_
                Fetch performance metrics from pmcd(1) on _host_ , rather than
                from the default localhost.
    
           **-H** , **--no-header**
                Do not print any headers.
    
           **-i** _instances_ , **--instances** =_instances_
                Retrieve and report only the specified metric _instances_.  By
                default all instances, present and future, are reported.
    
                Refer to pmrep(1) for complete description of this option.
    
           **-j** , **--live-filter**
                Perform instance live filtering.  This allows capturing all
                named instances even if processes are restarted at some point
                (unlike without live filtering).  Performing live filtering
                over a huge number of instances will add some internal
                overhead so a bit of user caution is advised.  See also **-n**.
    
           **-J** _rank_ , **--rank** =_rank_
                Limit results to highest/lowest _rank_ ed instances of set-
                valued metrics.  A positive integer will include highest
                valued instances in reporting.  A negative integer will
                include lowest valued instances in reporting.  A value of
                zero performs no ranking.  Ranking does not imply sorting,
                see **-6**.  See also **-8**.
    
           **-K** _spec_ , **--spec-local** =_spec_
                When fetching metrics from a local context (see **-L**), the **-K**
                option may be used to control the DSO PMDAs that should be
                made accessible.  The _spec_ argument conforms to the syntax
                described in pmSpecLocalPMDA(3).  More than one **-K** option may
                be used.
    
           **-L** , **--local-PMDA**
                Use a local context to collect metrics from DSO PMDAs on the
                local host without PMCD.  See also **-K**.
    
           **-n** , **--invert-filter**
                Perform ranking before live filtering.  By default instance
                live filtering (when requested, see **-j**) happens before
                instance ranking (when requested, see **-J**).  With this option
                the logic is inverted and ranking happens before live
                filtering.
    
           **-o** _outfile_ , **--output-file** =_outfile_
                Specify the output file _outfile_.  **-O** _origin_ , **--origin** =_origin_
                When reporting archived metrics, start reporting at _origin_
                within the time window (see **-S** and **-T**).  Refer to PCPIntro(1)
                for a complete description of the syntax for _origin_.
    
           **-r** , **--raw**
                Output raw metric values, do not convert cumulative counters
                to rates.  This option _will_ override possible per-metric
                specifications.
    
           **-R** , **--raw-prefer**
                Like **-r** but this option will _not_ override per-metric
                specifications.
    
           **-s** _samples_ , **--samples** =_samples_
                The _samples_ argument defines the number of samples to be
                retrieved and reported.  If _samples_ is 0 or **-s** is not
                specified, **pcp2arrow** will sample and report continuously (in
                real time mode) or until the end of the set of PCP archives
                (in archive mode).  See also **-T**.
    
           **-S** _starttime_ , **--start** =_starttime_
                When reporting archived metrics, the report will be
                restricted to those records logged at or after _starttime_.
                Refer to PCPIntro(1) for a complete description of the syntax
                for _starttime_.
    
           **-t** _interval_ , **--interval** =_interval_
                Set the reporting _interval_ to something other than the
                default 1 second.  The _interval_ argument follows the syntax
                described in PCPIntro(1), and in the simplest form may be an
                unsigned integer (the implied units in this case are
                seconds).  See also the **-T** option.
    
           **-T** _endtime_ , **--finish** =_endtime_
                When reporting archived metrics, the report will be
                restricted to those records logged before or at _endtime_.
                Refer to PCPIntro(1) for a complete description of the syntax
                for _endtime_.
    
                When used to define the runtime before **pcp2arrow** will exit,
                if no _samples_ is given (see **-s**) then the number of reported
                samples depends on _interval_ (see **-t**).  If _samples_ is given
                then _interval_ will be adjusted to allow reporting of _samples_
                during runtime.  In case all of **-T** , **-s** , and **-t** are given,
                _endtime_ determines the actual time **pcp2arrow** will run.
    
           **-v** , **--omit-flat**
                Report only set-valued metrics with instances (e.g.
                disk.dev.read) and omit single-valued ``flat'' metrics
                without instances (e.g.  kernel.all.sysfork).  See **-i** and **-I**.
    
           **-V** , **--version**
                Display version number and exit.
    
           **-z** , **--hostzone**
                Use the local timezone of the host that is the source of the
                performance metrics, as identified by either the **-h** or the **-a**
                options.  The default is to use the timezone of the local
                host.
    
           **-Z** _timezone_ , **--timezone** =_timezone_
                Use _timezone_ for the date and time.  _Timezone_ is in the
                format of the environment variable **TZ** as described in
                environ(7).  Note that when including a timezone string in
                output, ISO 8601 -style UTC offsets are used (so something
                like -Z EST+5 will become UTC-5).
    
           **-?** , **--help**
                Display usage message and exit.
    


## FILES



           _pcp2arrow.conf_
                **pcp2arrow** configuration file (see **-c**)
    
           $PCP_SYSCONF_DIR _/pmrep/*.conf_
                system provided default **pmrep** configuration files
    


## PCP ENVIRONMENT



           Environment variables with the prefix **PCP_** are used to
           parameterize the file and directory names used by PCP.  On each
           installation, the file _/etc/pcp.conf_ contains the local values for
           these variables.  The **$PCP_CONF** variable may be used to specify an
           alternative configuration file, as described in pcp.conf(5).
    
           For environment variables affecting PCP tools, see
           pmGetOptions(3).
    
           Of particular note, **PCP_DISCRETE_ONCE** can be set to ensure that
           discrete metric values are reported only once, unless they change
           at some point.
    


## DEBUGGING OPTIONS



           The **-D** or **--debug** option enables the output of additional
           diagnostics on _stderr_ to help triage problems, although the
           information is sometimes cryptic and primarily intended to provide
           guidance for developers rather end-users.  _debug_ is a comma
           separated list of debugging options; use pmdbg(1) with the **-l**
           option to obtain a list of the available debugging options and
           their meaning.
    


* * *