cs-cache 20250111
A few caching data structures and other lossy things with capped sizes.
Keywords: python2, python3

A few caching data structures and other lossy things with capped sizes.

*Latest release 20250111*:
Prevent automatic creation of the default ConvCache instance on import, which tries to make its cache directory.

## <a name="cachedmethod"></a>`cachedmethod(*da, **dkw)`

Decorator to cache the result of an instance or class method
and keep a revision counter for changes.

The cached values are stored on the instance (`self`).
The revision counter supports the `@revised` decorator.

This decorator may be used in 2 modes.
Directly:

    @cachedmethod
    def method(self, ...)

or indirectly:

    @cachedmethod(poll_delay=0.25)
    def method(self, ...)

Optional keyword arguments:
* `attr_name`: the basis name for the supporting attributes.
  Default: the name of the method.
* `poll_delay`: minimum time between polls; after the first
  access, subsequent accesses before the `poll_delay` has elapsed
  will return the cached value.
  Default: `None`, meaning the value never becomes stale.
* `sig_func`: a signature function, which should be significantly
  cheaper than the method. If the signature is unchanged, the
  cached value will be returned. The signature function
  expects the instance (`self`) as its first parameter.
  Default: `None`, meaning no signature function;
  the first computed value will be kept and never updated.
* `unset_value`: the value to return before the method has been
  called successfully.
  Default: `None`.

If the method raises an exception, this will be logged and
the method will return the previously cached value,
unless there is not yet a cached value
in which case the exception will be reraised.

If the signature function raises an exception
then a log message is issued and the signature is considered unchanged.

An example use of this decorator might be to keep a "live"
configuration data structure, parsed from a configuration
file which might be modified after the program starts. One
might provide a signature function which called `os.stat()` on
the file to check for changes before invoking a full read and
parse of the file.

*Note*: use of this decorator requires the `cs.pfx` module.

## <a name="CachingMapping"></a>Class `CachingMapping(cs.resources.MultiOpenMixin, collections.abc.MutableMapping)`

A caching front end for another mapping.
This is intended as a generic superclass for a proxy to a
slower mapping such as a database or remote key value store.

Note that this subclasses `MultiOpenMixin` to start/stop the worker `Thread`.
Users must enclose use of a `CachingMapping` in a `with` statement.
If subclasses also subclass `MultiOpenMixin` their `startup_shutdown`
method needs to also call our `startup_shutdown` method.

Example:

    class Store:
      """ A key value store with a slower backend.
      """
      def __init__(self, mapping:Mapping):
        self.mapping = CachingMapping(mapping)

    .....
    S = Store(slow_mapping)
    with S:
      ... work with S ...

*`CachingMapping.__init__(self, mapping: Mapping, *, max_size=1024, queue_length=1024, delitem_bg: Optional[Callable[[Any], cs.result.Result]] = None, setitem_bg: Optional[Callable[[Any, Any], cs.result.Result]] = None, missing_fallthrough: bool = False)`*:
Initialise the cache.

Parameters:
* `mapping`: the backing store, a mapping
* `max_size`: optional maximum size for the cache, default 1024
* `queue_length`: option size for the queue to the worker, default 1024
* `delitem_bg`: optional callable to queue a delete of a
  key in the backing store; if unset then deleted are
  serialised in the worker thread
* `setitem_bg`: optional callable to queue setting the value
  for a key in the backing store; if unset then deleted are
  serialised in the worker thread
* `missing_fallthrough`: is true (default `False`) always
  fall back to the backing mapping if a key is not in the cache

*`CachingMapping.flush(self)`*:
Wait for outstanding requests in the queue to complete.
Return the UNIX time of completion.

*`CachingMapping.items(self)`*:
Generator yielding `(k,v)` pairs.

*`CachingMapping.keys(self)`*:
Generator yielding the keys.

## <a name="ConvCache"></a>Class `ConvCache(cs.fs.HasFSPath)`

A cache for conversions of file contents such as thumbnails
or transcoded media, etc. This keeps cached results in a file
tree based on a content key, whose default function is
`cs.hashutils.file_checksum('blake3')`.

*`ConvCache.__init__(self, fspath: Optional[str] = None, content_key_func=None)`*:
Initialise a `ConvCache`.

Parameters:
* `fspath`: optional base path of the cache, default from
  `ConvCache.DEFAULT_CACHE_BASEPATH`;
  if this does not exist it will be created using `os.mkdir`
* `content_key_func`: optional function to compute a key
  from the contents of a file, default `cs.hashindex.file_checksum`
  (the blake3 hash of the contents)

*`ConvCache.content_key(self, srcpath)`*:
Return a content key for the filesystem path `srcpath`.

*`ConvCache.content_subpath(self, srcpath) -> str`*:
Return the content key based subpath component.

This default assumes the content key is a hash code and
breaks it hex representation into a 3 level hierarchy
such as `'d6/d9/c510785c468c9aa4b7bda343fb79'`.

*`ConvCache.convof(self, srcpath, conv_subpath, conv_func, *, ext=None, force=False) -> str`*:
Return the filesystem path of the cached conversion of
`srcpath` via `conv_func`.

Parameters:
* `srcpath`: the source filesystem path
* `conv_subpath`: a name for the conversion which encompasses
  the salient aspaects such as `'png/64/64'` for a 64x64 pixel
  thumbnail in PNG format
* `conv_func`: a callable of the form `conv_func(srcpath,dstpath)`
  to convert the contents of `srcpath` and write the result
  to the filesystem path `dstpath`
* `ext`: an optional filename extension, default from the
  first component of `conv_subpath`
* `force`: option flag to require conversion even if the
  cache has an entry

## <a name="convof"></a>`convof(srcpath, conv_subpath, conv_func, *, ext=None, force=False)`

`ConvCache.convof` using the default cache.

## <a name="file_based"></a>`file_based(*da, **dkw)`

A decorator which caches a value obtained from a file.

In addition to all the keyword arguments for `@cs.cache.cachedmethod`,
this decorator also accepts the following arguments:
* `attr_name`: the name for the associated attribute, used as
  the basis for the internal cache value attribute
* `filename`: the filename to monitor.
  Default from the `._{attr_name}__filename` attribute.
  This value will be passed to the method as the `filename` keyword
  parameter.
* `poll_delay`: delay between file polls, default `DEFAULT_POLL_INTERVAL`.
* `sig_func`: signature function used to encapsulate the relevant
  information about the file; default
  cs.filestate.FileState({filename}).

If the decorated function raises OSError with errno == ENOENT,
this returns None. Other exceptions are reraised.

## <a name="file_property"></a>`file_property(*da, **dkw)`

A property whose value reloads if a file changes.

## <a name="LRU_Cache"></a>Class `LRU_Cache`

A simple least recently used cache.

Unlike `functools.lru_cache`
this provides `on_add` and `on_remove` callbacks.

*`LRU_Cache.__init__(self, max_size, *, on_add=None, on_remove=None)`*:
Initialise the LRU_Cache with maximum size `max`,
additon callback `on_add` and removal callback `on_remove`.

*`LRU_Cache.__delitem__(self, key)`*:
Delete the specified `key`, calling the on_remove callback.

*`LRU_Cache.__setitem__(self, key, value)`*:
Store the item in the cache. Prune if necessary.

*`LRU_Cache.flush(self)`*:
Clear the cache.

*`LRU_Cache.get(self, key, default=None)`*:
Mapping method: get value for `key` or `default`.

*`LRU_Cache.items(self)`*:
Items from the cache.

*`LRU_Cache.keys(self)`*:
Keys from the cache.

## <a name="lru_cache"></a>`lru_cache(max_size=None, cache=None, on_add=None, on_remove=None)`

Enhanced workalike of @functools.lru_cache.

# Release Log



*Release 20250111*:
Prevent automatic creation of the default ConvCache instance on import, which tries to make its cache directory.

*Release 20250103*:
* Pull in @cachedmethod from cs.deco for dependency reasons.
* Move file_based and @file_property from cs.fileutils to cs.cache.

*Release 20241122*:
* convof, ConvCache.convof: new options force=False parameter to bypass existing cache entries.
* ConvCache.convof: check the conv_subpath using cs.fs.validate_rpath.

*Release 20240422.1*:
ConvCache docstring update.

*Release 20240422*:
New ConvCache and convof: a cache for conversions of file contents such as thumbnails or transcoded media.

*Release 20240412*:
* New CachingMapping, a caching front end for another mapping.
* LRU_Cache: add keys() and items().

*Release 20181228*:
Initial PyPI release.