# pmfetchgroup

## NAME



           **pmCreateFetchGroup** , **pmExtendFetchGroup_item** ,
           **pmExtendFetchGroup_indom** , **pmExtendFetchGroup_event** ,
           **pmExtendFetchGroup_timespec** , **pmExtendFetchGroup_timeval** ,
           **pmFetchGroup** , **pmGetFetchGroupContext** , **pmClearFetchGroup** ,
           **pmDestroyFetchGroup** - simplified performance metrics value fetch
           and conversion
    


## C SYNOPSIS



           **#include <pcp/pmapi.h>**
    
           **int pmCreateFetchGroup(pmFG ***_ptr_**, int** _type_**, const char ***_name_**);**
           **int pmExtendFetchGroup_item(pmFG** _pmfg_**, const char ***_metric_**,**
                                       **const char ***_instance_**,**
                                       **const char ***_scale_**,**
                                       **pmAtomValue ***_out_value_**, int** _out_type_**,**
                                       **int** _out_sts_**);**
           **int pmExtendFetchGroup_indom(pmFG** _pmfg_**, const char ***_metric_**,**
                                        **const char ***_scale_**,**
                                        **int** _out_inst_codes_**[],**
                                        **char ***_out_inst_names_**[],**
                                        **pmAtomValue** _out_values_**[],**
                                        **int** _out_type_**, int** _out_stss_**[],**
                                        **unsigned int** _out_maxnum_**,**
                                        **unsigned int ***_out_num_**, int ***_out_sts_**);**
           **int pmExtendFetchGroup_event(pmFG** _pmfg_**, const char ***_metric_**,**
                                        **const char ***_instance_**,**
                                        **const char ***_field_**, const char ***_scale_**,**
                                        **struct timespec** _out_times_**[],**
                                        **pmAtomValue** _out_values_**[],**
                                        **int** _out_type_**, int** _out_stss_**[],**
                                        **unsigned int** _out_maxnum_**,**
                                        **unsigned int ***_out_num_**, int ***_out_sts_**);**
           **int pmExtendFetchGroup_timespec(pmFG** _pmfg_**,**
                                           **struct timespec ***_out_value_**);**
           **int pmExtendFetchGroup_timeval(pmFG** _pmfg_**,**
                                          **struct timeval ***_out_value_**);**
           **int pmGetFetchGroupContext(pmFG** _pmfg_**);**
           **int pmFetchGroup(pmFG** _pmfg_**);**
           **int pmClearFetchGroup(pmFG** _pmfg_**);**
           **int pmDestroyFetchGroup(pmFG** _pmfg_**);**
    
           **cc ... -lpcp**
    


## DESCRIPTION



           The  fetchgroup functions implement a registration-based mechanism
           to fetch groups of performance metrics, including  automation  for
           general  unit, rate and type conversions as well as convenient in‐
           stance and value encodings.  They constitute a powerful  and  com‐
           pact  alternative  to  the classic Performance Metrics Application
           Programming Interface  (PMAPI(3))  sequence  of  separate  lookup,
           check, fetch, iterate, extract and convert functions.
    
           The  general idea consists of two stages.  In the setup stage, the
           application identifies metrics of interest by name  and  with  de‐
           sired  conversions, and register a unique **pmAtomValue** output loca‐
           tion where the fetchgroup system is to later deposit  the  result.
           It  is also possible to identify a metric with an instance domain,
           and register a unique _vector_ of pmAtomValue objects for them.   In
           the operation stage, one simple **pmFetchGroup** function fetches, de‐
           codes,  converts,  and  stores  all metrics to their destinations,
           where the application can _read_ them directly.  This  function  may
           be called repeatedly, and each time new **pmAtomValue** values will be
           stored in the same destinations.  Rate conversions between consec‐
           utive samples may be requested.
    
           Each  fetchgroup is associated with a private PMAPI context, so it
           can manipulate instance profiles and other such state without dis‐
           rupting other contexts.  The instance profile  is  manipulated  to
           optimize  fetches  of  individual  items, even if some are derived
           metrics.  This private PMAPI context belongs to the fetchgroup, is
           used for all of its internal operations, and will be destroyed.
    
           Multiple fetchgroups may be used concurrently, independently.   An
           opaque type **pmFG** is used to identify a fetchgroup, which is passed
           to all related function calls.
    
       **Creating a fetchgroup**
    
           **int pmCreateFetchGroup(pmFG ***_ptr_**, int** _type_**, const char ***_name_**);**
    
           This  function  creates  a  new  fetchgroup, associated with a new
           PMAPI context.  The  _type_  and  _name_  parameters  are  relayed  to
           pmNewContext(3) for creation of the context.  The fetchgroup iden‐
           tifier is returned upon success through the _ptr_ pointer.  This ob‐
           ject  is  later  used as a parameter to all other fetchgroup func‐
           tions.  The private PMAPI context may be accessed with **pmGetFetch‐**
           **GroupContext** , if required.
    
           The normal function return code is zero, and  _ptr_  is  set.   This
           function may fail in case of **pmNewContext** or memory allocation er‐
           rors.   Those  are  indicated  with  a  negative return code and a
           cleared _ptr_ value.
    
       **Getting the private PMAPI context**
    
           **int pmGetFetchGroupContext(pmFG** _pmfg_**);**
    
           This function returns the private PMAPI context used by the  given
           fetchgroup.   It  may  be safely used to adjust some configuration
           parameters of the context, such as via pmSetMode(3), before fetch‐
           group extension and fetching begins.
    
           However, _mutation_ of this context by PMAPI  functions  after  this
           time  may disrupt fetchgroup functionality.  For example, a **pmSet‐**
           **Mode** call could invalidate one rate-conversion time-step.
    
           The normal function return code is the context number.
    
       **Extending a fetchgroup with a metric instance of interest**
    
           **int pmExtendFetchGroup_item(pmFG** _pmfg_**, const char ***_metric_**,**
                                       **const char ***_instance_**,**
                                       **const char ***_scale_**,**
                                       **pmAtomValue ***_out_value_**, int** _out_type_**,**
                                       **int ***_out_sts_**);**
    
           This function registers interest in a single metric  and  optional
           instance.   The metric name is given in the mandatory _metric_ para‐
           meter, which is checked immediately via pmLookupName(3) and  other
           calls.  If and only if the metric has an instance domain, the spe‐
           cific instance of interest may be named by the _instance_ parameter,
           which  is  checked  immediately via pmNameInDom(3); otherwise pass
           NULL.  If the fetchgroup context is a set of archives, it is  pos‐
           sible  that  the  metric / instance pair is not yet defined at the
           current time origin.  Therefore, this function may attempt to seek
           to the _end_ of the current set of archives _temporarily_ to retry the
           metric / instance lookup.
    
           The optional _scale_  parameter  specifies  desired  unit/scale/rate
           conversions  for the metric value.  It can take the following val‐
           ues:
    
           NULL
               No unit/scale conversion.  If metric has **PM_SEM_COUNTER** seman‐
               tics, perform rate conversion.
    
           rate
               Perform rate conversion regardless of semantics, and no
               unit/scale conversion.
    
           instant
               Perform no rate conversion regardless of semantics, and no
               unit/scale conversion.
    
           EXPRESSION
               Perform unit/scale/rate conversion as specified by the EXPRES‐
               SION, which is parsed by pmParseUnitsStr(3).  This may be use‐
               ful to assert a canonical scaling for the resulting metric
               value, independent of PCP version or configuration.  Dimen‐
               sionality must match the metric, except if rate conversion is
               requested, in which case the time dimension must be one small‐
               er than the metric's time dimension.  Note that the type of
               rate conversion performed here matches the **rate(x)** function in
               derived metric expressions, in that it is calculated as the
               naive difference between previous and current values of a met‐
               ric, divided by elapsed time.  For example, if a counter wraps
               around, or a non-counter value decreases, a _negative_ output
               rate may be computed.
    
           The optional but usual _out_value_ parameter specifies the **pmAtom‐**
           **Value** where the converted result should later be stored.  If the
           value is NULL, fetching and conversions will be attempted, and
           possible errors reported, but the result tossed away.  The manda‐
           tory _out_type_ parameter specifes the **PM_TYPE_*** requested for the
           output value.  It need not match the metric's native type, as the
           fetchgroup facility is capable of casting between all supported
           types (including to and from strings).
    
           Any errors subsequently encountered during fetching,
           unit/scale/rate conversion, or casting, will result in the assign‐
           ment of a sentinel value to the output **pmAtomValue**(see the ``UN‐
           USUAL SITUATIONS'' section below).  In addition, if the optional
           _out_sts_ parameter is specified, an appropriate PMAPI error code
           will be stored there.
    
           As a review, only the _pmfg_ , _metric_ , and _out_type_ parameters are
           mandatory.  Others may be NULL to indicate application disinter‐
           est.
    
           The normal function return code is zero.  This function may fail
           in case of various lookup, type- and conversion- checking errors.
           Those are indicated with a negative return code.
    
       **Extending a fetchgroup with a metric instance domain of interest**
    
           **int pmExtendFetchGroup_indom(pmFG** _pmfg_**, const char***_metric_**,**
                                        **const char ***_scale_**,**
                                        **int** _out_inst_codes_**[],**
                                        **char ***_out_inst_names_**[],**
                                        **pmAtomValue** _out_values_**[],**
                                        **int** _out_type_**, int** _out_stss_**[],**
                                        **unsigned int** _out_maxnum_**,**
                                        **unsigned int ***_out_num_**, int ***_out_sts_**);**
    
           This function generalizes the **pmExtendFetchGroup_item** function  by
           registering  interest  in a whole instance domain.  Therefore, the
           function registers preallocated _vectors_ for output variables  (in‐
           stead  of  a singleton).  Instances will be stored in sorted order
           in elements of those vectors.   The  concepts  are  otherwise  the
           same.
    
           The  metric  name  is specified by the mandatory _metric_ parameter.
           Note that it _may_ refer to a metric without an instance domain,  in
           which  case the single output value will appear as one unnamed in‐
           stance.
    
           The optional _scale_  parameter  specifies  desired  unit/scale/rate
           conversions for the metric value, same as above.
    
           The  optional _out_inst_codes_ parameter specifies a vector of inte‐
           gers, where the raw instance number of the fetched metrics  should
           later be stored.
    
           The  optional  _out_inst_names_  parameter  specifies  a  vector  of
           strings, where the instance names of the  fetched  metrics  should
           later  be  stored.   If  an instance does not have a corresponding
           name, a NULL pointer is stored instead.  The application must  not
           modify or free(3) strings in that vector.
    
           The optional _out_values_ parameter specifies a vector of **pmAtomVal‐**
           **ue** objects where the converted result should later be stored.  The
           mandatory _out_type_ parameter specifies the **PM_TYPE_*** requested for
           the all output values, same as above.
    
           The  optional  _out_stss_  parameter  specifies a vector of integers
           where per-instance error codes should be stored.
    
           The mandatory _out_maxnum_ parameter specifies the  number  of  ele‐
           ments  of  the vectors above.  In other words, it tells the fetch‐
           group the maximum number of instances which are expected.  The op‐
           tional _out_num_ parameter specifies an  integer  where  the  actual
           number of instances should later be stored.  It will range between
           0 and _out_maxnum_.  It is initialized to 0 by this function.
    
           Finally,  the  optional _out_sts_ parameter specifies a single loca‐
           tion where an integer status code for the overall fetch  for  this
           metric should be stored.  Normally, this will be zero.  Other than
           a severe fetch error, one may see a **PM_ERR_TOOBIG** here if the num‐
           ber of instances actually encountered was larger than _out_maxnum_.
    
           Any    errors    subsequently    encountered    during   fetching,
           unit/scale/rate conversion, or casting, will result in the assign‐
           ment of a sentinel value to  the  appropriate  output  **pmAtomValue**
           (see  the  ``UNUSUAL SITUATIONS'' section below).  In addition, if
           the optional _out_stss_ parameter was specified, a PMAPI error  code
           will be stored in the appropriate position.
    
           As a review, only the _pmfg_ , _metric_ , _out_type_ , and _out_maxnum_ para‐
           meters  are mandatory.  Others may be NULL to indicate application
           disinterest.
    
           The normal function return code is zero.  This function  may  fail
           in  case of various lookup, type- and conversion- checking errors.
           Those are indicated with a negative return code.
    
       **Extending a fetchgroup with an event field**
    
           **int pmExtendFetchGroup_event(pmFG** _pmfg_**, const char ***_metric_**,**
                                        **const char ***_instance_**,**
                                        **const char ***_field_**, const char ***_scale_**,**
                                        **struct timespec** _out_times_**[],**
                                        **pmAtomValue** _out_values_**[],**
                                        **int** _out_type_**, int** _out_stss_**[],**
                                        **unsigned int** _out_maxnum_**,**
                                        **unsigned int ***_out_num_**, int ***_out_sts_**);**
    
           This function registers interest in all instances of one field  of
           all  records  of  an event metric.  Since event metrics may return
           multiple records per fetch, and  each  record  may  have  multiple
           fields  of a given field metric type, this function registers pre‐
           allocated _vectors_ for output  variables,  similarly  to  **pmExtend‐**
           **FetchGroup_indom**.  They are filled in temporal/sequential order.
    
           The  metric  name  is specified by the mandatory _metric_ parameter.
           It must be of **PM_TYPE_EVENT**.  If the metric has  an  instance  do‐
           main, the _instance_ parameter is mandatory to identify the instance
           of interest.
    
           The field to extract from event records is specified by the manda‐
           tory  _field_  parameter,  which  is  a metric name of normal scalar
           type.  As is typical for event field metrics, it should  not  have
           an  instance  domain.   The optional _scale_ parameter specifies de‐
           sired unit/scale conversions on this metric value.   Rate  conver‐
           sions are **not available** , because of ambiguity about which previous
           value to compute rates from.
    
           The  optional  _out_times_  parameter specifies a vector of **timespec**
           structs, which will receive a copy of the timestamp of  the  event
           record where each particular field was found.
    
           The optional _out_values_ parameter specifies a vector of **pmAtomVal‐**
           **ue** objects where the converted result should later be stored.  The
           mandatory _out_type_ parameter specifies the **PM_TYPE_*** requested for
           the output values.
    
           The  optional  _out_stss_  parameter  specifies a vector of integers
           where per-field error codes should be stored.
    
           The mandatory _out_maxnum_ parameter specifies the  number  of  ele‐
           ments  of  the vectors above.  In other words, it tells the fetch‐
           group the maximum number of instances which are expected.  The op‐
           tional _out_num_ parameter specifies an integer where the the actual
           number of instances should later be stored.  It will range between
           zero and _out_maxnum_.  It is initialized to zero by this function.
    
           Finally, the optional _out_sts_ parameter specifies a  single  loca‐
           tion  where  an integer status code for the overall fetch for this
           metric should be stored.  Normally, this will be zero, even if  no
           event field values were found (_out_num_ would then be zero).  Other
           than a severe fetch error, one may see a **PM_ERR_TOOBIG** here if the
           number of fields actually encountered was larger than _out_maxnum_.
    
           Any  errors  subsequently  encountered during fetching, unit/scale
           conversion, or casting, will result in the assignment  of  a  sen‐
           tinel  value  to the appropriate output **pmAtomValue**(see the ``UN‐
           USUAL SITUATIONS'' section below).  In addition, if  the  optional
           _out_stss_  parameter  was  specified,  a  PMAPI  error code will be
           stored in the appropriate position.
    
           As  a  review,  only  the  _pmfg_ ,  _metric_ ,  _field_ ,  _out_type_ ,   and
           _out_maxnum_  parameters are mandatory.  Others may be NULL to indi‐
           cate application disinterest.
    
           The normal function return code is zero.  This function  may  fail
           in  case of various lookup, type- and conversion- checking errors.
           Those are indicated with a negative return code.
    
       **Extending a fetchgroup with the fetch timestamp**
    
           **int    pmExtendFetchGroup_timespec(pmFG** _pmfg_**,     struct time‐**
                                           **spec ***_out_value_**);**
           **int pmExtendFetchGroup_timeval(pmFG** _pmfg_**, struct timeval ***_out_val‐_
                                          _ue_**);**
    
           These  functions  register  interest  in the **pmHighResResult** time‐
           stamp.  If the _out_value_ pointer is non-NULL, at every future  **pm‐**
           **FetchGroup** call, the corresponding result timestamp will be copied
           there.
    
       **Fetching all metrics in a fetchgroup**
    
           **int pmFetchGroup(pmFG** _pmfg_**);**
    
           This  function  performs one **pmFetch** on its private PMAPI context,
           including all the metrics that were registered via prior **pmExtend‐**
           **FetchGroup_*** calls.  It runs all the data extraction  and  conver‐
           sion  operations  necessary  to  populate all the requested output
           variables.
    
           The normal function return code is zero or positive,  as  per  the
           underlying  **pmFetch** function.   This function may fail in case of
           severe fetch errors, which are indicated with  a  negative  return
           code.
    
           In  the  case  of per-metric availability or conversion errors, or
           severe fetch errors, output variables are reset to sentinel values
           and individual error codes are set.   _PM_ERR_AGAIN_  signals  rate-
           conversion failure due to lack of a previous value.
    
           However,  temporarily  absent  metrics with discrete semantics are
           exempt from some  sentinel/error  processing:  if  a  **pmFetchGroup**
           fails  to  collect a result for a discrete metric (pmHighResResult
           pmValueSet.numval==0), then the last seen valid value (if any)  is
           retained.   This  is  intended  to  ease the processing of sets of
           archives with a mixture of once- and  repeatedly-sampled  metrics.
           The  environment  variable **PCP_DISCRETE_ONCE** may be set to disable
           this feature.
    
       **Clearing a fetchgroup**
    
           **int pmClearFetchGroup(pmFG** _pmfg_**);**
    
           When the current fetch state of a fetchgroup is no longer  needed,
           it  may  be  explicitly reset with this function.  It releases any
           dynamically stored state but keeps the private PMAPI  context  in‐
           tact  for subsequent use (i.e. no change to the context is made at
           all and the context remains at  the  current  fetch  offset).   It
           frees  any  pointers  such as indom instance names or strings that
           may have been stored in output variables.
    
       **Destroying a fetchgroup**
    
           **int pmDestroyFetchGroup(pmFG** _pmfg_**);**
    
           When the fetchgroup is no longer  needed,  it  may  be  explicitly
           freed  with  this  function.   It  releases any dynamically stored
           state, as well as the private PMAPI context.  It clears frees  any
           pointers  such  as  indom  instance names or strings that may have
           been stored in output variables.
    


## EXAMPLE



           The following program demonstrates fetchgroup usage.  Run it  with
           different  $PCP_DISK_UNITS  environment variables to see different
           unit/rate conversion in effect.
    
           #include <pcp/pmapi.h>
           #include <stdio.h>
    
           #define pcpassert(sts) \
               while (sts<0) { fprintf(stderr, "%s\n", pmErrStr(sts)); exit(42); }
    
           int main()
           {
               pmFG fg;
               pmAtomValue v, v2;
               enum { v3_maxnum = 100 };
               pmAtomValue v3_values[v3_maxnum];
               char *v3_names[v3_maxnum];
               int v3_stss[v3_maxnum];
               unsigned int v3_num;
               int sts, i;
               char *diskunits = getenv("PCP_DISK_UNITS");
               struct timeval t;
    
               sts = pmCreateFetchGroup(&fg, PM_CONTEXT_HOST, "local:");
               pcpassert(sts);
               sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "1 minute",
                                             NULL, &v, PM_TYPE_FLOAT, NULL);
               pcpassert(sts);
               sts = pmExtendFetchGroup_item(fg, "kernel.all.idletime", NULL,
                                             "hour", &v2, PM_TYPE_DOUBLE, NULL);
               pcpassert(sts);
               sts = pmExtendFetchGroup_indom(fg, "disk.dev.total", diskunits,
                                              NULL, v3_names,
                                              v3_values, PM_TYPE_STRING,
                                              v3_stss, v3_maxnum, &v3_num, NULL);
               pcpassert(sts);
               sts = pmExtendFetchGroup_timeval(fg, &t);
               pcpassert(sts);
    
               for (i=0; i < 10; i++) {
                   unsigned int j;
                   char stamp[28];
    
                   sts = pmFetchGroup(fg);
                   pcpassert(sts);
                   printf("%s", pmCtime(&t.tv_sec, stamp));
                   printf("1-minute load: %f; idletime: %f h\n", v.f, v2.d);
                   for (j=0; j < v3_num; j++) {
                       if (v3_stss[j] == 0)
                            printf("disk %s i/o operations (%s): %s\n",
                                   v3_names[j] ? v3_names[j] : "?",
                                   diskunits ? diskunits : "-",
                                   v3_values[j].cp);
                   }
                   sleep(1);
               }
    
               sts = pmDestroyFetchGroup(fg);
               pcpassert(sts);
               return 0;
           }
    


## UNUSUAL SITUATIONS



           The fetchgroup API supports only the  numeric,  string  and  event
           metric  types.   Aggregates  are  rejected  during  **pmExtendFetch‐**
           **Group_***.
    
           Any strings supplied by the fetchgroup API to the application  are
           "owned" by the API.  The application should consider them read-on‐
           ly, so it should not modify them nor **free** them.
    
           Error  codes  are  always negative integers, whether returned from
           fetchgroup functions as return value, or stored  in  _out_sts_  type
           variables.  Normal result codes are always zero.
    
           Because  of  the unique ways in which extracted data is shared be‐
           tween the application and a fetchgroup, the functions in this  API
           are  _not_  _protected_ by the multi-threading mutexes conventional in
           other parts of PMAPI.  Specifically, for any given **pmFG** , it is _not_
           _safe_ to concurrently call two or more  fetchgroup  API  functions,
           nor  to traverse the registered output variables while calling one
           of the functions.  Instead, the calling  application  must  ensure
           that only one thread at a time uses these calls _and_ the registered
           output  variables.  On the other hand, concurrency between differ‐
           ent **pmFG** instances is unrestricted, because they share  no  global
           data.
    
           Any pointers passed to a successful **pmFetchGroupExtent_*** call must
           stay valid throughout the lifetime of the fetchgroup, since future
           **pmFetchGroup** calls may write into them.
    


## DIAGNOSTICS



           The  fetchgroup API offers several options for collecting diagnos‐
           tics.  Negative integer error codes  may  be  returned  from  each
           function for serious conditions.
    
           In  addition, each output pmAtomValue may have a corresponding in‐
           teger variable, where **pmFetchGroup** can  store  per-metric  per-in‐
           stance error codes.
    
           As  an  alternative,  per-metric per-instance error conditions are
           also signalled by setting the corresponding pmAtomValue to a  sen‐
           tinel  value.   If  unambiguous and precise error detection is not
           required, this may be sufficient.  The sentinel value is  negative
           one  for all integers (including unsigned integers - i.e. all bits
           are set), **NaN** for  floating  point  types,  a  NULL  pointer  for
           strings,  and  0.0s for the timestamp.  The fetchgroup API guaran‐
           tees that once an output pmAtomValue is registered (during a  suc‐
           cessful **pmExtendFetchGroup_*** call), it will be cleared to the sen‐
           tinel value or to a valid converted metric value, from the time of
           registration until the **pmDestroyFetchGroup** call.
    


## COMPATIBILITY



           Prior to PCP 7.0 the **pmExtendFetchGroup_timestamp** was an alias for
           **pmExtendFetchGroup_timeval**.   To support PMAPI transition, the old
           interface and semantics can be used if applications are recompiled
           with **-DPMAPI_VERSION=2**.
    
           **pmExtendFetchGroup_timestamp** although this is now  deprecated  and
           compile-time support will be removed in a future release.
    


* * *