smemo 0.8
Explicit session memoization
Requires Python: >=3.9

# smemo: Explicit session memoization

Memoization of function return values is a very common optimization
technique.  Even the standard Python 3 library provides multiple
facilities for it, e.g., `functools.cached_property` and
`functools.lru_cache`.  Needless to say, many user-contributed
facilities are also available.

Most of them focuses on ensuring that the cache is invisible to the
user.  When you call a function decorated by `functools.lru_cache`,
you don't really need to know that the function value is cached.  That
is usually a big merit, but at times this is a big drawback.

The problem is on invalidation.  Most libraries spent a lot of efforts
to keep track of them magically so that we don't need to worry about
them.  But at times, all these work are not necessary or outright
counterproductive.  The run of your program might be divided into
parts, or sessions, where each session is benefited by having return
values of functions, usually many of them, saved rather than
recomputed.  But after a session, the saved values of the function is
of little value or even may be invalid, so there is not much to be
gained by keeping them in memory.

In such cases, most other caching tools either do not help at all, or
requires you to carefully find the functions being used by the session
and invalidate their caches one by one.

This module uses a more explicit approach.  A "session" object is
used, providing the cache required by the memoization process.  The
availability of this object makes a few operations, tricky in other
libraries, to become trivial.  They include:

  * Invalidation of all caches of a session.
  * One single function uses different caches for different calls.
  * Fine-grain control of whether values of a function should skip
    caching.
  * Injection of values to a particular function for a session.

In fact, the implementation of the module is so simple that the core
of it can be done in just a couple of hours.

## Usage

Functions requiring caching should be written like this:

    @smemo.cached
    def efib(session: smemo.BaseSession, n: int,
             a0: float, a1: float) -> float:
        "Return an extended Fibonacci number where efib[0] = a0, efib[1] = a1"
        if n == 0:
            return a0
        if n == 1:
            return a1
        return efib(session, n - 1, a0, a1) + efib(session, n - 2, a0, a1)

Note that it explicitly includes the session object as the first
argument.  All functions decorated by @smemo.cached does the same.

It can then be called like this:

    session = smemo.Session()
    print(efib(session, 5, 1, 3))

After the call, the session contains entries like `{((2, 1, 3), ()):
(4, None)}`, the key contains a hashable representation of the
positional and keyword arguments, while the values are the return
value or exception raised.  The value can be observed without
triggering a call to the inner function, by:

    print(session.get_cache(efib, 5, 1, 3))

The returned value of the function decorated by `@smemo.cached` goes
through `copy.deepcopy`, so that the caller can freely manipulate the
returned object without affecting the cache.  If that is undesirable
(usually because it slows things down or you really want the actual
object returned), you would replace `@smemo.cached` by
`@smemo.rcached`.

A simpler cache is used for functions that takes just the session
argument.  We will not forward function arguments and keyword
arguments in this case, and exceptions are not cached.  This is to
speed up the common use case where the session is used as a global
objects store.  In this case the `simple_get_cache` function is used
instead of `get_cache`.

For many purposes these are good enough.  But at times some of the
values (e.g., 3 and 1 above) are really constants for a session, and
you can save a lot of space by not including them in the keys of the
cache.  In such cases you can rewrite your function as follows:

    @smemo.cached
    def efib(session: smemo.BaseSession, n: int) -> float:
        "Return an extended Fibonacci number where efib[0] = a0, efib[1] = a1"
        if n == 0:
            return session.getval('a0')  # Same as smemo.getter(session, 'a0')
        if n == 1:
            return session.getval('a1')
        return efib(session, n - 1) + efib(session, n - 2)

You call your function like this instead:

    session = smemo.Session()
    session.putval(1, 'a0')
    session.putval(3, 'a1')
    print(efib(session, 5))

This is a trivial function to implement.  It just composes other
session facilities with a "getter" function, the latter simply raises
an exception.  On the other hand, this is a very useful mechanism,
because it enables you to use memoization where previously you might
not even think you want to.  For example, you can read a whole config
file and put it to your session.  Now your functions can freely use
the configuration values, and you don't need to worry about having
cache which has huge entries, and at the same time you don't need to
worry about cross-talk when your function is called concurrently with
different sessions.

## Nested session

At times you may have some of your data to live for a shorter lifetime
than the rest.  In such case, you might find it beneficial to have a
hierarchy of sessions.  The Session class `__init__` takes a "parent"
BaseSession argument.  When cache is fetched and no entry is found,
this parent will be consulted.  Whenever cache entry is set it is
local.  This way you can populate some entry in a global session, and
create a "sub-session" which will hold any additional cache entries.
With an additional `restrict` argument, you can also specify that only
some functions will be cached in the new objects, all other functions
are handled solely in the parent.

## Cache control

The big differentiation between smemo and other solutions is cache
control.  We do not do implicit cached control like LRU at all, all
cached values are kept there until explicitly invalidated.  In fact,
we don't even store those information (e.g., when is a value last
needed), so the overheads in our module is quite small.

Having a session object, explicit cache control can be done trivially.
For example, you can clear the cache either for one call, for one
function or for all functions, by the following respectively:

    session.invalidate(efib, 7)
    session.invalidate_all(efib)
    session.invalidate_all()

For the first call, the call needs to match the calling arguments, but
mypy will not be able to catch it if there is a type error.  An
alternative method can be used instead, which does trigger mypy to
catch a type error:

    efib(session.inv, 7)

Because it is just another call to efib, it has to match the type
correctly to pass mypy.

For the last `session.invalidate` call, it actually doesn't invalidate
all function values cached.  If a function is marked as "persistent",
the values cached for it can only be invalidated more explicitly.
This is done by using `@smemo.gcached(persistent=True)` instead of
`@smemo.cached`, or (more commonly) using `@smemo.prcached()` instead
of `@smemo.rcached`.  Under the hood, `getval` and `putval` is just
cache of a function, where the "persistent" mechanism is used to avoid
getting values removed.

You can force a value into the cache, like one of these:

    session.cache(efib, 5.0, 7)
    session.cache_exc(efib, RuntimeError('My error'), 7)

In this case, it causes efib(7, session) to return 5.0 and raises "My
error" respectively.  This is very useful during unit tests, where at
times you don't want your functions to actually be called.  Again, we
have alternative methods which allows mypy to catch type errors:

    efib(session.setcache(5.0), 7)
    efib(session.setcache(exception=RuntimeError('My error')), 7)

So, if you want to avoid `getval` and `setval` (which mypy cannot
check for type errors), you can create functions returning the correct
type and mark them as persistent instead:

    @smemo.gcached(ref=True, persistent=True)
    def mydict(session: smemo.BaseSession, a: str) -> typing.Dict[str, float]:
        raise RuntimeError(f'No value set for {a}')

Then you can put and get values as follows:

    mydict(session.setcache({'pi': 3.1415936, 'e': 2.7182818}), 'const')
    print(mydict(session, 'const')['pi'])

Note that this mechanism, when combined with nested sessions, can be
used as a poor man dependency injection mechanism.  All you need to do
is to pass a session when you invoke any top-level function, and
obtain all dependencies from the session.  The main program would
populate the main session and perhaps create sub-sessions as needed,
before calling the top-level functions.  Your unit tests can use mock
objects instead of actual objects as desired.

Finally, you can call a function without caching it:

    efib(session.callonly, 7)

The skipping of caching does not extend to the calls made by efib.  If
you want that, you can do the following:

    with session.nocache:
        print(efib(session, 7))

Caching would be disabled for all the duration of the above efib call.

## Persistence keys

The `persistent` keyword argument of `gcached()` mentioned above also
allows the argument to be a string rather than just `True`.  In this
case the argument is called a persistence key, or "pkey" in short.  It
is used like `@gcached(persistent='my_key')`.  Just like a regular
persistent function, the cached value can be invalidated by explicitly
invalidating a function, but not by using `invalidate_all()`.  But
another method `invalidate_by_pkey()` can be used to invalidate the
function, together with all other functions tagged with the same pkey.
For example, you might have some functions which do not need to be
invalidated with `invalidate_all()`, but can be invalidated when a
specific event happens which triggers `invalidate_by_pkey()`.

## BaseSession interface

All the functionality of the memoization system is provided by the
BaseSession interface, which the Session class is written against.  If
you want your own facilities, you might want to provide your own
implementation.  It is a rather simple interface:

  * `get_cache(func, *args, **kwargs)`

      * When the user calls the defined function, the function
        object and the positional/keyword arguments are passed to
        this method to find a cache entry, which should be in the
        form `(ret_val, exception)`, where the `ret_val` is used if
        `exception` is None.

  * `simple_get_cache(func)`

      * Same as `get_cache`, used if `func` takes the session argument
        only.

  * `pre_call(func, args, kwargs)`

      * When cache entry is not found or is skipped, this function is
        called to determine the session to call the actual function.
        If None is returned, the actual function is not called, and it
        is treated as None being returned.

  * `simple_pre_call(func)`

      * Same as `pre_call`, used if `func` takes the session argument
        only.

  * `cache_exc(func, _exc, *args, **kwargs)`

      * When the actual call generates an exception, this function is
        called to ask the session object to cache the resulting
        exception.  Note that we usually define the method to have the
        exception argument named like `_exc`, because the name could
        clash with names in `kwargs`.

  * `cache_val(func, _val, *args, **kwargs)`

      * When the actual call returns a value, this function is called
        to ask the session object to cache the result.  Note that we
        usually define the method to have the exception argument named
        like `_val`, because the name could clash with names in
        `kwargs`.

Cache control is actually done by creative use of the interface.  For
example, `session.inv` is a BaseSession object which returns None upon
`get_cache` or `pre_call`, but invalidate cache entry upon
`cache_val`.  And `session.setcache` returns an object which is just
the same as `session.inv` but instead of invalidating the cache, it
forces a return value or exception to the cache.