# pmdacache

## NAME



           **pmdaCacheStore** , **pmdaCacheStoreKey** , **pmdaCacheLookup** ,
           **pmdaCacheLookupName** , **pmdaCacheLookupKey** , **pmdaCacheOp** ,
           **pmdaCachePurge** , **pmdaCachePurgeCallback** , **pmdaCacheResize** - manage a
           cache of instance domain information for a PMDA
    


## C SYNOPSIS



           **#include <pcp/pmapi.h>**
           **#include <pcp/pmda.h>**
    
           **int pmdaCacheStore(pmInDom** _indom_**, int** _flags_**, const char ***_name_**,**
                              **void ***_private_**);**
           **int pmdaCacheStoreKey(pmInDom** _indom_**, int** _flags_**, const char ***_name_**,**
                                 **int** _keylen_**, const void ***_key_**, void ***_private_**);**
           **int pmdaCacheLookup(pmInDom** _indom_**, int** _inst_**, char ****_name_**,**
                               **void ****_private_**);**
           **int pmdaCacheLookupName(pmInDom** _indom_**, const char ***_name_**,**
                                   **int ***_inst_**, void ****_private_**);**
           **int pmdaCacheLookupKey(pmInDom** _indom_**, const char ***_name_**,**
                                  **int** _keylen_**, const void ***_key_**, char ****_oname_**,**
                                  **int ***_inst_**, void ****_private_**);**
           **int pmdaCacheOp(pmInDom** _indom_**, int** _op_**);**
           **int pmdaCachePurge(pmInDom** _indom_**, time_t** _recent_**);**
           **int pmdaCachePurgeCallback(pmInDom** _indom_**, time_t** _recent_**,**
                                      **void (***_callback_**)(void *));**
           **int pmdaCacheResize(pmInDom** _indom_**, int** _maximum_**);**
    
           **cc ... -lpcp_pmda -lpcp**
    


## DESCRIPTION



           The  **pmdaCache** family of routines provide services to support the
           maintenance of complex instance domains for  Performance  Co-Pilot
           PMDAs.  There is potentially one cache of information for each in‐
           stance domain, and for each instance the cache maintains:
           - external instance name (supplied by the PMDA)
           - internal instance identifier (assigned by **pmdaCacheStore** or cal‐
             culated from a ``hint'' by **pmdaCacheStoreKey**)
           - state, where **active** instances are visible and part of the cur‐
             rent instance domain, and **inactive** instances are hidden, but not
             forgotten; **pmdaCacheStore** or **pmdaCacheStoreKey** may be used to
             change the state of an instance
           - an optional opaque pointer to data that is associated with the
             instance, but maintained by the PMDA
           - an optional opaque key that is used as a ``hint'' to **pmdaCache‐**
             **StoreKey** when guessing the initial internal instance identifier
           - the last time the cache was saved and the instance had been
             marked as **active** at some point since the previous cache load or
             save operation
    
           The semantics of a PCP instance domain require a number of rules
           to be followed, namely:
           1. Each internal instance identifier must be unique and in the
              range 0 to 2^31 - 1.  This rule is enforced by the **pmdaCache**
              family of routines.
           2. The external instance name must be unique.  When the instance
              name contains a space, it is further constrained such that the
              name to the left of the first space (the short name) must also
              be unique.  Refer to the INSTANCE NAME MATCHING section below.
              The PMDA must honor this rule, the **pmdaCache** family of routines
              will detect attempts to violate this rule.
           3. Where an external instance name corresponds to some object or
              entity, there is an expectation that the association between
              the name and the object is fixed, e.g. ``/dev/hda'' is always
              the name of the same disk on a particular system.  This rule is
              perhaps the responsibility of the PMDA, but is often a charac‐
              teristic of the environment in which the PMDA runs.
           4. It is preferable, although not mandatory, for the association
              between an external instance name and an internal instance
              identifier to be persistent.  This rule is supported by the **pm‐**
              **daCache** family of routines.
           5. When opaque keys are used, the values of the keys must be
              unique across all instances within an instance domain.  This
              rule is enforced by the **pmdaCache** family of routines.
    
           The visible interface to the cache is oriented towards the PMDA
           developer who is most concerned about the names of instances,
           while the details of how the rest of the PCP infrastructure ex‐
           pects the internal instance identifiers to be managed is not rele‐
           vant.
    
           Instances are updated in the cache for instance domain _indom_ by
           calling **pmdaCacheStore** or **pmdaCacheStoreKey** with the external name
           of the instance passed via _name._  The opaque pointer _private_ may
           be used to associate additional data with the entry in the cache;
           if no such data is required, _private_ should be NULL.  Any manipu‐
           lation of the additional data (including allocation or freeing) is
           the responsibility of the PMDA caller, as the cache simply main‐
           tains the pointer to the data (passed via _private_).
    
           The upper bound for identifiers allocated for any given _indom_
           cache can be optionally reduced from the default (2^31 - 1) to
           some lesser _maximum_ , using **pmdaCacheResize**.  This _maximum_ will
           then be persisted and restored in the usual manner, and can thus
           be associated permanently with a cache once set.  This has appli‐
           cations when using these interfaces as general purpose identifier
           caches, and is less applicable when using them for instance domain
           caching.
    
           For cases where the PMDA developer wishes to influence the alloca‐
           tion of internal instance identifiers, e.g. for instance domains
           with more than one natural dimension, or where there is a desire
           to allocate the same instance identifier each time the PMDA is
           started, even on different hosts, **pmdaCacheStoreKey** may be used.
           In this case, an initial ``hint'' for the instance identifier is
           provided as an opaque key via the first _keylen_ bytes in _key_ (which
           could be any sort of data, including binary values) else if _keylen_
           is less than 1 or _key_ is **NULL** then _name_ is used as the ``hint''.
           The ``hint'' is hashed to produce an initial instance identifier
           in the range 0 to 2^31 - 1 (or lesser _maximum_ , if set).  If this
           instance identifier is already allocated, then the value is re‐
           hashed.  This procedure is repeated until an unallocated instance
           identifier is found, or **pmdaCacheStoreKey** gives up and returns
           **PM_ERR_GENERIC**.  For each instance domain, the ``hint'' must be
           unique across all instances, else **pmdaCacheStoreKey** returns
           **PM_ERR_INST**.
    
           The _flags_ argument controls how the instance should be processed
           in the cache as follows:
    
           PMDA_CACHE_ADD
                  Insert the entry into the cache if it is not already there
                  and mark it **active**.  If the entry is already in the cache
                  mark it **active**.
    
           PMDA_CACHE_HIDE
                  Mark the entry in the cache as **inactive** , but remember the
                  details of the association between the external instance
                  name and the internal instance identifier.  Entries that
                  are **inactive** will be hidden from cache traversal via PM‐
                  DA_CACHE_WALK_NEXT operations, but remain visible to **pmda‐**
                  **CacheLookup** , **pmdaCacheLookupName** and **pmdaCacheLookupKey** re‐
                  quests.
    
           PMDA_CACHE_CULL
                  Remove the entry from the cache.
    
           On success **pmdaCacheStore** or **pmdaCacheStoreKey** will return the in‐
           ternal instance identifier of the associated cache entry.  Valid
           instance identifiers are guaranteed to be unique and non-negative.
           Failure will be indicated by a negative value (suitable for decod‐
           ing with pmErrStr(3)) and most likely PM_ERR_INST to indicate the
           requested instance is not in the cache, or -EINVAL to indicate a
           potential violation of the short name uniqueness property (see the
           INSTANCE NAME MATCHING section below).
    
           **pmdaCacheLookup** is used to search the entries in the cache based
           on the internal instance identifier _inst_.
    
           On success the return value will be PMDA_CACHE_ACTIVE or PM‐
           DA_CACHE_INACTIVE (depending on the **active** or **inactive** state of
           the cache entry), _name_ (if not NULL) and _private_ (if not NULL)
           will be set to the external instance name and the associate addi‐
           tional data area as provided when the instance was last activated
           via **pmdaCacheStore** or **pmdaCacheStoreKey**.
    
           **pmdaCacheLookup** failure is indicated by a negative return value
           suitable for decoding with pmErrStr(3).
    
           The **pmdaCacheLookup** interface is required by the PMDA's fetch
           callback that is registered via pmdaSetFetchCallBack(3).  Here the
           internal instance identifier is passed to the fetch callback to
           identifier for which instance a value is required.  Typical usage
           is shown in the code fragment below.
    
             **static int**
             **foo_callback(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)**
             **{**
                 **mydata   *mdp;**
                 **char     *name;**
                 **int      sts;**
    
                 **sts = pmdaCacheLookup(mdesc- >m_desc.indom, inst, &name, (void **)&mdp);**
                 **/***
                  *** expect sts == PMDA_CACHE_ACTIVE except for cataclysmic events**
                  *** use mdp as required, name may be useful for diagnostics**
                  ***/**
                 **...**
    
           **pmdaCacheLookupName** is used to search the entries in the cache
           based on the external instance name _name_.
    
           On success the return value will be PMDA_CACHE_ACTIVE or PM‐
           DA_CACHE_INACTIVE (depending on the **active** or **inactive** state of
           the cache entry), _inst_ (if not NULL) and _private_ (if not NULL)
           will be set to the internal instance identifier and the associate
           additional data area as provided when the instance was last acti‐
           vated via **pmdaCacheStore** or **pmdaCacheStoreKey**.
    
           **pmdaCacheLookupName** failure is indicated by a negative return val‐
           ue suitable for decoding with pmErrStr(3).
    
           The **pmdaCacheLookupName** interface is useful for PMDAs wishing to
           update an instance domain based on the external instance names.
    
           **pmdaCacheLookupKey** is used to search the entries in the cache
           based on an opaque key (or ``hint'') previously used in a call to
           **pmdaCacheStoreKey**.  The ``hint'' is provided via the first _keylen_
           bytes in _key_.  For symmetry with **pmdaCacheStoreKey** , if _keylen_ is
           less than 1 or _key_ is **NULL** then _name_ is used as the ``hint'' (al‐
           though the results will be the same as calling **pmdaCacheLookupName**
           in this case).
    
           On success the return value will be PMDA_CACHE_ACTIVE or PM‐
           DA_CACHE_INACTIVE (depending on the **active** or **inactive** state of
           the cache entry), _oname_ (if not NULL), _inst_ (if not NULL) and _pri‐_
           _vate_ (if not NULL) will be set to the external instance name, the
           internal instance identifier and the associate additional data
           area as provided when the instance was last activated via **pmda‐**
           **CacheStore** or **pmdaCacheStoreKey**.
    
           **pmdaCacheLookupKey** failure is indicated by a negative return value
           suitable for decoding with pmErrStr(3).
    
           To avoid a persistent cache growing without bound, **pmdaCachePurge**
           can be used to cull all entries that have _not_ been **active** in the
           last _recent_ seconds.  For performance reasons, the time accounting
           is imprecise and the entries are timestamped at the time of the
           next cache save operation _after_ the entry has been added or marked
           **active**(refer to PMDA_CACHE_SAVE, PMDA_CACHE_WRITE and PM‐
           DA_CACHE_SYNC below).  On success **pmdaCachePurge** returns the num‐
           ber of culled entries, else in the case of an error the return
           value is negative (and suitable for decoding with pmErrStr(3)).
    
           The **pmdaCachePurgeCallback** function is similar to **pmdaCachePurge**
           except that a _callback_ function will also be called with the _pri‐_
           _vate_ data pointer associated with the cache entry to be culled.
           The callback is not made if _private is_ **NULL**.  This would typically
           be used to free the private data when the associated entry is
           purged in PMDAs that do not separately maintain any references to
           the private data.
    
           **pmdaCacheOp** may be used to perform additional operations on the
           cache as follows:
    
           PMDA_CACHE_LOAD
                  The cache can optionally be maintained as a persistent ex‐
                  ternal file, so that the mapping of instance names to in‐
                  stance identifiers is persistent across executions of a PM‐
                  DA.  This operation loads the cache from the external file,
                  and then all new cache entries are marked **inactive** , and the
                  additional data pointer is set to NULL.  Entries loaded
                  from the external file are checked against the current
                  cache contents and if the instance name and instance iden‐
                  tifiers match then the state in the cache (**active** or **inac‐**
                  **tive**) is not changed. Should a mismatch be found (same in‐
                  stance name and different instance identifier, or same in‐
                  stance identifier and different instance name, or some but
                  not all of the instance identifier, the instance name and
                  the ``hint'' match) then the entry from the external file
                  is ignored and a warning is issued on _stderr_.  Typically a
                  PMDA would only perform this operation once per execution.
    
           PMDA_CACHE_SAVE
                  If any instance has been added to, or deleted from, the in‐
                  stance domain since the last PMDA_CACHE_LOAD, PM‐
                  DA_CACHE_SAVE, PMDA_CACHE_WRITE or PMDA_CACHE_SYNC opera‐
                  tion, the _entire_ cache is written to the external file as a
                  bulk operation.  This operation is provided for PMDAs that
                  are _not_ interested in using **pmdaCachePurge** and simply want
                  the external file to reflect the set of known instances
                  without accurate details of when they were last marked **ac‐**
                  **tive**.
    
                  Returns the number of instances saved to the external file,
                  else 0 if the external file was already up to date.
    
           PMDA_CACHE_WRITE
                  A variant of PMDA_CACHE_SAVE where the _entire_ cache is un‐
                  conditionally written to the external file as a bulk opera‐
                  tion, independent of any previous cache operations or the
                  state of the cache.
    
           PMDA_CACHE_STRINGS
                  Annotates this cache as being a special-purpose cache used
                  for string de-duplication in PMDAs exporting large numbers
                  of string valued metrics.  This can be used to reduce the
                  memory footprint of the PMDA (duplicate strings hash to the
                  same bucket, and are stored in memory once only).  Key com‐
                  parisons are not terminated at the first space but rather
                  the entire string is used for matching.  These are spe‐
                  cialised caches not useful for general purpose instance do‐
                  main handling.
    
           PMDA_CACHE_SYNC
                  Within an instance domain, if any instance has been added
                  to, or deleted from, or marked **active** since the last PM‐
                  DA_CACHE_LOAD, PMDA_CACHE_SAVE, PMDA_CACHE_WRITE or PM‐
                  DA_CACHE_SYNC operation, the _entire_ cache is written to the
                  external file as a bulk operation.  This operation is simi‐
                  lar to PMDA_CACHE_SAVE, but will save the instance domain
                  more frequently so the timestamps more accurately match the
                  semantics expected by **pmdaCachePurge**.
    
                  Returns the number of instances saved to the external file,
                  else 0 if the external file was already synchronized.
    
           PMDA_CACHE_CHECK
                  Returns 1 if a cache exists for the specified instance do‐
                  main, else 0.
    
           PMDA_CACHE_REUSE
                  When a new instance is added to the cache, the default
                  strategy is to assign instance identifiers in a monotonic
                  increasing manner.  Once the maximum possible instance
                  identifier value has been assigned, the strategy changes to
                  one where starting from 0, the next available unused in‐
                  stance identifier will be used.  Calling **pmdaCacheOp** with
                  PMDA_CACHE_REUSE forces an irreversible change to a second
                  (reuse) strategy where the next unallocated instance iden‐
                  tifier will be used.  This may be useful in cases where
                  there is a desire to restrict the allocated instance iden‐
                  tifiers to smaller values.  The prevailing strategy will be
                  saved and restored across PMDA_CACHE_SAVE and PM‐
                  DA_CACHE_LOAD operations.  If **pmdaCacheStoreKey** is ever
                  used, the associated instance domain will be changed to PM‐
                  DA_CACHE_REUSE mode.
    
           PMDA_CACHE_REORG
                  Reorganize the cache to allow faster retrieval of **active**
                  entries, at the cost of slower retrieval for **inactive** en‐
                  tries, and reclaim any culled entries.  The cache may be
                  internally re-organized as entries are added, so this oper‐
                  ation is not required for most PMDAs.
    
           PMDA_CACHE_WALK_REWIND
                  Prepares for a traversal of the cache in ascending instance
                  identifier sequence.
    
           PMDA_CACHE_WALK_NEXT
                  Fetch the next **active** instance identifier from the cache.
                  Requires a prior call using PMDA_CACHE_WALK_REWIND and will
                  return -1 when all instances have been processed.
    
                  Only one cache walk can be active at any given time, nest‐
                  ing calls to PMDA_CACHE_WALK_NEXT and PM‐
                  DA_CACHE_WALK_REWIND will interfere with each other.
    
           PMDA_CACHE_ACTIVE
                  Changes **every inactive** entry in the cache to be marked **ac‐**
                  **tive**.
    
           PMDA_CACHE_INACTIVE
                  Changes **every active** entry in the cache to be marked **inac‐**
                  **tive**.
    
           PMDA_CACHE_CULL
                  Remove **every** entry from the cache.
    
           PMDA_CACHE_SIZE
                  Return the number of entries in the cache (includes **active** ,
                  **inactive** and any culled entries that have not yet been re‐
                  claimed).
    
           PMDA_CACHE_SIZE_ACTIVE
                  Return the number of **active** entries in the cache.
    
           PMDA_CACHE_SIZE_INACTIVE
                  Return the number of **inactive** entries in the cache.
    
           PMDA_CACHE_DUMP
                  Dump the current state of the cache on _stderr_.
    
           PMDA_CACHE_DUMP_ALL
                  Like PMDA_CACHE_DUMP, but also dump the internal hashing
                  structures used to support lookup by instance name, lookup
                  by instance identifier and the collision statistics for
                  ``hint'' hashing from **pmdaCacheStoreKey**.
    
           **pmdaCacheOp** returns a non-negative value on success, and failure
           is indicated by a negative return value (suitable for decoding
           with pmErrStr(3)).
    


## OTHER CONSIDERATIONS



           When the **pmdaCache** routines are used for particular instance do‐
           main, pmdaInstance(3) and the instance domain enumeration behind
           pmdaFetch(3) will attempt to extract instance domain information
           from the cache, thereby avoiding reference to the **pmdaIndom** data
           structures that have historically been used to define instance do‐
           mains and service instance requests.  A PMDA can adopt a hybrid
           approach and choose to implement some instance domains via the
           traditional **pmdaIndom** method, and others via the **pmdaCache** ap‐
           proach, however attempts to manage the _same_ instance domain by
           both methods will result in the **pmdaCache** method silently prevail‐
           ing.
    
           If **all** instances in a PMDA are to be serviced from a **pmdaCache**
           then a **pmdaIndom** is not required, and the pmdaInit(3) call becomes
    
                 **pmdaInit(dp, NULL, 0, metrictab, nmetrics);**
    
           However, the PMDA will need to explicitly initialize the **indom**
           field of the **pmDesc** in the _metrictab_ entries, as this cannot be
           done by pmdaInit(3) if _indomtab_ is missing entries for the in‐
           stance domains maintained in the cache.
    
           Independent of how the instance domain is being maintained, to re‐
           fresh an instance domain prior to a fetch or an instance domain
           operation, the standard methods of a ``wrapper'' to the
           pmdaInstance(3) and pmdaFetch(3) methods should be used.
    
           Refer to the **simple** PMDA source code for an example use of the **pm‐**
           **daCache** routines.
    
           When using **pmdaCacheStoreKey** , if there is a desire to ensure the
           given ``hint'' generates the same initial instance identifier
           across all platforms, then the caller should ensure the endian and
           word-size issues are considered, e.g. if the natural data struc‐
           ture used for the _key_ is an array of 32-bit integers, then
           htonl(3) should be used on each element of the array before call‐
           ing **pmdaCacheStoreKey** or **pmdaCacheLookupKey**.
    


## INSTANCE NAME MATCHING



           The following table summarizes the ``short name'' matching seman‐
           tics for an instance domain (caches other than PMDA_CACHE_STRINGS
           style).
                ┌─────────┬─────────────────┬────────────────────────┐
                │ name in │ **pmdaCacheLookup** │ result                 │
                │ cache   │ name            │                        │
                ├─────────┼─────────────────┼────────────────────────┤
                │ foodle  │ foo             │ no match (PM_ERR_INST) │
                │ foo     │ foodle          │ no match (PM_ERR_INST) │
                │ foo     │ foo             │ match                  │
                │ foo bar │ foo             │ match on short name    │
                │         │                 │ (instance identifier)  │
                │ foo bar │ foo bar         │ match on full name     │
                │         │                 │ (instance identifier)  │
                │ foo     │ foo bar         │ bad match (-EINVAL)    │
                │ foo bar │ foo blah        │ bad match (-EINVAL)    │
                └─────────┴─────────────────┴────────────────────────┘
    


## FILES



           Cache persistence uses files with names constructed from the _indom_
           within the **$PCP_VAR_DIR/config/pmda** directory.
    


* * *