Repository: Pylons/pyramid

Issue:
Cache identity
Seems like caching identity is a popular request, but has been difficult to figure out how to properly deal with it.  Opening this issue for discussion.   Some ideas off the top of my head:

- Bust cache on request.authenticated_userid change
- Bust cache on remember/forget
- Have `identify` return a two-tuple of the identity and a boolean on whether to cache or not.
- Have a `request.authenticated_identity` and `request.cached_identity`
- Have a manual `bust_identity_cache()` function


Discussion:
If an identity for an authed employee is cached, and the employee gets fired then deleted or disabled in the system, I would want that fired employee's identity cache to get busted. Does that use case apply to this issue?
I think it's overblown a bit. At least in my apps (which are similar to the tutorial approach) the user is cached for the lifetime of every request. This is fine for 99% of apps (imo). It's only in situations where long-running requests are in play where the user's information may change during a request. My own apps do not mutate the credentials or bust them. There's a `request.user`, or `request.access_token` and it remains the same throughout the request - it does not change when forget is called.

With this in mind, my suggestion is normally to just cache the info. This works for every request in almost every situation without much thought. Everything except for login/logout.

The two cases I have seen that are odd are around remember/forget. The decisions here are application-specific, and not something that I think Pyramid can solve directly - it just needs to provide a way for the application to do what it wants.

1) login - the response to a login request may want to use credentials from the newly logged-in user. However, it is not correct to just replace the credentials on the current request. For example, there may have been credentials supplied in the request, in order to get an access token for another user. This can happen when authentications applications, or allowing admins to login as users, or other stuff. In these scenarios you do not want to just replace the user loaded in the request with the new user - it kind of depends on what is done as part of login.

2) logout - the response to a logout request may want to prevent certain data from leaking as part of logout. For example a user has the ability to logout and we don't want the remainder of the request to know it's for a user. This is rare - normally a logout should be a POST + redirect, or a simple json response.

In both scenarios above it's ok most of the time to just cache the original request identity for the duration of the request. If it's not then the policy can hook remember/forget and run a `del request.authenticated_identity` or update it to a new value like `None` or anything else.

To be clear the proposal is basically:

1) The result of `ISecurityPolicy.authenticated_identity` is cached on the request object as `request.authenticated_identity`.

2) This value is then available as `request.authenticated_identity` throughout the app, including in the security policy methods like `permits`, `remember`, and `forget`.

3) On boundaries, the app may mutate the cached value. For example `request.authenticated_identity = something_else`.

Are there scenarios that this simple approach is not handling?
> For example, there may have been credentials supplied in the request, in order to get an access token for another user. This can happen when authentications applications, or allowing admins to login as users, or other stuff. In these scenarios you do not want to just replace the user loaded in the request with the new user - it kind of depends on what is done as part of login.

If all we do is bust the cache, the correct identity will be loaded next time `request.authenticated_identity` is accessed.  Worst case, the identity didn't change and reloading it is a bit of unnecessary extra work.  So I think we should bust the cache on remember/forget, as that's less of a footgun then relying on users to manage the cache themselves.
Downside of cache busting is that is remember/forget can't mutate cached `authenticated_identity`, because it will immediately be forgotten.  Only use-case I can think of for mutating would be efficiency?  Since remember/forget are generally just called on login and logout, I'm okay sacrificing efficiency for the sake of a more straightforward security policy implementation.
It’s not well defined how bust it. For example if you delete it on remember then it’s just going to get loaded again with the value in the request - same value as before remember. The policy needs to do it however it wants to do it. 
Hmm, I see your point.  I'm good with your proposal.
I'm walking this back for like the 100th time - unless someone feels super strongly I think this should be done inside the policy itself and not done by the framework. I think it'll just be confusing and unclear without some naming changes, etc. It won't fly to have `request.authenticated_identity` automatically cached by pyramid but to have `request.authenticated_userid` un-cached. I think Pyramid needs to treat them both as-is, and the policy itself can handle caching if it desires.

**It's really really close to just removing `authenticated_identity` and `identify` as it's no longer required anywhere** - however, I think it's probably a useful abstraction. If we remove it we basically **have** to provide a way to access the security policy publicly. The reason I say this is:

1) We **want** users to define authentication sources inside the policy.
2) Users probably want to cache the identity.

These two constraints are directly competing because the authentication source, `remember`, and `forget` **need to share code and implementation details**.

There's only a couple ways to do this while maintaining bw-compat and none of them are sitting well with me other than our current impl where all the details are left up to the policy itself.

Anyone wanna counterpoint me to keep this open?
The more I think about it the more I agree—caching is a concern of the policy. The framework caching identity would be a huge convenience, but does blur lines and confuse concepts around the policy. 

Maybe we’re approaching this from the wrong angle. Instead of worrying if and how the framework should cache identity, maybe we should focus in making implementing caching in the policy less awkward. An `identity_cache` decorator or something in Pyramid would go a long ways. Or at least some documentation giving a reference implementation for a caching policy. 

— Theron Luhn

> On Dec 24, 2019, at 2:31 PM, Michael Merickel <notifications@github.com> wrote:
> 
> ﻿
> I'm walking this back for like the 100th time - unless someone feels super strongly I think this should be done inside the policy itself and not done by the framework. I think it'll just be confusing and unclear without some naming changes, etc. It won't fly to have request.authenticated_identity automatically cached by pyramid but to have request.authenticated_userid un-cached. I think Pyramid needs to treat them both as-is, and the policy itself can handle caching if it desires.
> 
> It's really really close to just removing authenticated_identity and identify as it's no longer required anywhere - however, I think it's probably a useful abstraction. If we remove it we basically have to provide a way to access the security policy publicly. The reason I say this is:
> 
> We want users to define authentication sources inside the policy.
> Users probably want to cache the identity.
> These two constraints are directly competing because the authentication source, remember, and forget need to share code and implementation details.
> 
> There's only a couple ways to do this while maintaining bw-compat and none of them are sitting well with me other than our current impl where all the details are left up to the policy itself.
> 
> Anyone wanna counterpoint me to keep this open?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.

This direction seems fine.  If we get a helper decorator, or some guidance in the docs, then security policy authors can easily implement cached identity with even less code than we did with older Pyramid.
I think Pyramid has a general issue of not providing a clear pattern for caching per-request data if you want the source of the data external to something directly computed on the request (using `reify` or `config.add_request_method`).

I accomplish this easily with `pyramid_services` where I use a per-request service container that can cache things based on type but the overhead of instantiating it per-request is actually fairly expensive and not something I want to put into Pyramid.

The el-cheapo approach is to just do things like:

```python
bar = getattr(request, '_foo', marker)
if bar is marker:
    bar = something_expensive(request)
    request._foo = bar
return bar
```

But this could be done more cleanly with some sort of simple cache object:

```python
class MySecurityPolicy:
    def __init__(self):
        self.identity_cache = Cache()

    def identify(self, request):
        identity = self.identity_cache.get(request)
        if identity is self.identity_cache.NO_VALUE:
            identity = load_identity_from_db(request)
            self.identity_cache.set(request, identity)
        return identity
```

This could be done pretty easily using a weakref on the request or via `request.add_finished_callback` to free up any circular references. I'm sure there's alternatively some third party library to point people to if it isn't worth Pyramid shipping something.
It feels like the simple cache object example above is less nice to write and less efficient than an old-school project-specific reified request property.
The reified request property is explicitly harder to implement now because we don't expose `unauthenticated_userid` any longer. The reified request property will not have access to the helper objects the policy uses to identify and authenticate the user.
Yeah I mean the goal is to put everything in the policy and `request.user` can be something like `config.add_request_method(lambda request: request.authenticated_identity, 'user', property=True)`. Or if the identity is not a user object, then something that picks the user out of the identity, etc. This was the goal of `authenticated_identity` - to allow the policy to perform all of the work without some roundabout `unauthenticated_userid`. If it's not sitting well then can talk about it. :-)
I was thinking about weakref, didn't think of `request.add_finished_callback`...  That's clever.

I am a big +1 on adding some sort of `Cache` or `RequestStore` class.  Would be very useful for security policies, obviously, but also could be useful in general.  There are cases where I'd like to store something per-request, but would like to keep it more self-contained than dumping it into the request object.

I would also like to give users an option for simple policies that's as easy as `@reify`, some sort of `@cached_identity` or something.  I think that should help address @merwok's concerns.
I don't have a plan to do anything about this.
I'd like to try to put something together, would you consider a PR for this?

Unfortunately I'm pretty slammed for the next couple weeks, so I don't think I'll be able to make it into the alpha.
I'd consider something... it'll need to come with good narrative docs / support or I'll probably just argue to put it into an external package. I'm on the fence how general it should be as well - specific to identity or general request cache or multiple apis or what.
Yeah, I’ve been thinking about that. My current thought it should be identity-specific. It’d be hard to get a general-purpose utility done right and agreed upon by everybody. Start with some easier that gets concrete results.
The new API is quite nice, and with `request.authentified_identity` we are close to removing the need for a custom `user` (or other name) request property added in each project.  The fine point is that it’s the security policy that can do the caching, but the cached value needs to be tied to a request object; this is not too hard, but not obvious at first sight, and too small for a reusable package.  I will try to make a PR soon with one of the ideas written in this thread.
Revising this after adding a reified request property in my test project.

It’s not that bad!  Can be a one-liner; it’s still clean as most of the work is happening in the security policy; it’s not bad to have security policy methods depend on the custom request property (you can define both of these in the same `includeme` function, making the relationship clear to readers); it gives you the flexibility to build something on top of `authenticated_userid` or `authenticated_identity` (just switched from using only the former to using both!).

I am not sure now that adding a whole request cache class to save one line is worth it.  Maybe we see how people write security policies in the wild after the new versions is shipped.  Maybe other use cases bring back the idea of a generic request cache class that can cache identity and other things.  (This does seem like a great idea for a third-party module, an analog of how pyramid-services can help you clean up code for utilities stored in the registry.)

@luhn what do you think now?  Would you still like a `cached_identity` decorator in the core?  If you think so, and can add a couple details (i.e. is this a class decorator for security policies that wraps the `identify` method?), then I could try a PR.  If not, I am OK to see this idea rejected for 2.0.
My thoughts mirror [mmerickel's on refiy](https://github.com/Pylons/pyramid/pull/3557#discussion_r363824413)—it might work for certain use cases, but its not good practice and doesn't cover `request.authenticated_identity`.

My thoughts for cache_identity were something like:

```python
def cached_identity(func):
    cache = dict()

    @functools.wraps(func)
    def wrapped(request):
        if request in cache:
            return cache[request]
        identity = func(request)
        cache[request] = identity
        request.add_finalizer(clear_cache)
        return identity

    def clear_cache(request):
        del cache[request]

    return wrapped

class MySecurityPolicy:
    @cached_identity
    def identity(request):
        # ...

    # ...
```

If we don't go for `@cached_identity`, we should at the very least have caching sample code in the documentation, since its such a common use case.  So we have to maintain the code either way :P
Here is my current approach - need to write tests and turn it into a PR but it does work as advertised. Just don't ask how.

```python
class RequestCache:
    """
    A store that caches values during for the lifecycle of a request.

    Instantiate a cache object and use it to decorate methods or functions
    that accept a request parameter. Any other arguments are not used as
    cache keys so make sure they are constant across calls per-request.

    Wrapping methods:

    .. code-block:: python

        class SecurityPolicy:
            identity_cache = RequestCache()

            @identity_cache
            def authenticated_identity(self, request):
                result = ...  # do some expensive computations
                return result

    Wrapping functions:

    .. code-block:: python

        user_cache = RequestCache()

        @user_cache
        def get_user(request):
            result = ...  # do some expensive computations
            return result

    It's possible to inspect and influence the cache during runtime using
    :meth:`.get`, :meth:`.set` and :meth:`.clear`.

    """
    NO_VALUE = Sentinel('NO_VALUE')

    def __init__(self):
        self.store = dict()

    def auto_adapt(decorator):
        class FunctionOrMethodAdapter:
            def __init__(self, inst, func):
                self.inst = inst
                self.__wrapped__ = func

            def __call__(self, *args, **kwargs):
                return decorator(self.inst, self.__wrapped__)(*args, **kwargs)

            def __get__(self, instance, owner):
                return decorator(self.inst, self.__wrapped__.__get__(instance, owner))

        def adapt(inst, fn):
            return FunctionOrMethodAdapter(inst, fn)
        return adapt

    @auto_adapt
    def __call__(self, fn):
        """
        Decorate a method or function.

        """
        @functools.wraps(fn)
        def wrapper(request, *args, **kwargs):
            result = self.get(request)
            if result is self.NO_VALUE:
                result = fn(request, *args, **kwargs)
                self.set(request, result)
                request.add_finished_callback(self.clear)
            return result
        return wrapper

    del auto_adapt

    def clear(self, request):
        """
        Delete the value from the cache.

        The cached value is returned or :attr:`.NO_VALUE`.

        """
        return self.store.pop(request, self.NO_VALUE)

    def get(self, request, default=NO_VALUE):
        """
        Return the value from the cache.

        The cached value is returned or ``default``.

        """
        return self.store.get(request, default)

    def set(self, request, value):
        """
        Update the cache with a new value.

        """
        self.store[request] = value
```

Resolving patch:
diff --git a/src/pyramid/paster.py b/src/pyramid/paster.py
--- a/src/pyramid/paster.py
+++ b/src/pyramid/paster.py
@@ -107,6 +107,12 @@ def bootstrap(config_uri, request=None, options=None):
 
        Added the ability to use the return value as a context manager.
 
+    .. versionchanged:: 2.0
+
+       Request finished callbacks added via
+       :meth:`pyramid.request.Request.add_finished_callback` will be invoked
+       by the ``closer``.
+
     """
     app = get_app(config_uri, options=options)
     env = prepare(request)
diff --git a/src/pyramid/request.py b/src/pyramid/request.py
--- a/src/pyramid/request.py
+++ b/src/pyramid/request.py
@@ -1,4 +1,6 @@
 from collections import deque
+import functools
+import weakref
 from webob import BaseRequest
 from zope.interface import implementer
 from zope.interface.interface import InterfaceClass
@@ -17,6 +19,7 @@
 from pyramid.util import (
     InstancePropertyHelper,
     InstancePropertyMixin,
+    Sentinel,
     bytes_,
     text_,
 )
@@ -330,3 +333,134 @@ def apply_request_extensions(request, extensions=None):
         InstancePropertyHelper.apply_properties(
             request, extensions.descriptors
         )
+
+
+class RequestLocalCache:
+    """
+    A store that caches values during for the lifecycle of a request.
+
+    Wrapping Functions
+
+    Instantiate and use it to decorate functions that accept a request
+    parameter. The result is cached and returned in subsequent invocations
+    of the function.
+
+    .. code-block:: python
+
+        @RequestLocalCache()
+        def get_user(request):
+            result = ...  # do some expensive computations
+            return result
+
+        value = get_user(request)
+
+        # manipulate the cache directly
+        get_user.cache.clear(request)
+
+    The cache instance is attached to the resulting function as the ``cache``
+    attribute such that the function may be used to manipulate the cache.
+
+    Wrapping Methods
+
+    A method can be used as the creator function but it needs to be bound to
+    an instance such that it only accepts one argument - the request. An easy
+    way to do this is to bind the creator in the constructor and then use
+    :meth:`.get_or_create`:
+
+    .. code-block:: python
+
+        class SecurityPolicy:
+            def __init__(self):
+                self.identity_cache = RequestLocalCache(self.load_identity)
+
+            def load_identity(self, request):
+                result = ...  # do some expensive computations
+                return result
+
+            def authenticated_identity(self, request):
+                return self.identity_cache.get_or_create(request)
+
+    The cache maintains a weakref to each request and will release the cached
+    values when the request is garbage-collected. However, in most scenarios,
+    it will release resources earlier via
+    :meth:`pyramid.request.Request.add_finished_callback`.
+
+    .. versionadded:: 2.0
+
+    """
+
+    NO_VALUE = Sentinel('NO_VALUE')
+
+    def __init__(self, creator=None):
+        self._store = weakref.WeakKeyDictionary()
+        self._creator = creator
+
+    def __call__(self, fn):
+        @functools.wraps(fn)
+        def wrapper(request):
+            return wrapper.cache.get_or_create(request, fn)
+
+        wrapper.cache = self
+        self._creator = fn
+        return wrapper
+
+    def get_or_create(self, request, creator=None):
+        """
+        Return the value from the cache. Compute if necessary.
+
+        If no value is cached then execute the creator, cache the result,
+        and return it.
+
+        The creator may be passed in as an argument or bound to the cache
+        by decorating a function or supplied as a constructor argument.
+
+        """
+        result = self._store.get(request, self.NO_VALUE)
+        if result is self.NO_VALUE:
+            if creator is None:
+                creator = self._creator
+                if creator is None:
+                    raise ValueError(
+                        'no creator function has been registered with the '
+                        'cache or supplied to "get_or_create"'
+                    )
+            result = creator(request)
+            self.set(request, result)
+        return result
+
+    def get(self, request, default=NO_VALUE):
+        """
+        Return the value from the cache.
+
+        The cached value is returned or ``default``.
+
+        """
+        return self._store.get(request, default)
+
+    def set(self, request, value):
+        """
+        Update the cache with a new value.
+
+        """
+        already_set = request in self._store
+        self._store[request] = value
+
+        # avoid registering the callback more than once
+        if not already_set:
+            request.add_finished_callback(self._store.pop)
+
+    def clear(self, request):
+        """
+        Delete the value from the cache.
+
+        The cached value is returned or :attr:`.NO_VALUE`.
+
+        """
+        old_value = self.NO_VALUE
+        if request in self._store:
+            old_value = self._store[request]
+
+            # keep a value in the store so that we don't register another
+            # finished callback when set is invoked
+            self._store[request] = self.NO_VALUE
+        return old_value
diff --git a/src/pyramid/scripting.py b/src/pyramid/scripting.py
--- a/src/pyramid/scripting.py
+++ b/src/pyramid/scripting.py
@@ -74,6 +74,12 @@ def prepare(request=None, registry=None):
 
        Added the ability to use the return value as a context manager.
 
+    .. versionchanged:: 2.0
+
+       Request finished callbacks added via
+       :meth:`pyramid.request.Request.add_finished_callback` will be invoked
+       by the ``closer``.
+
     """
     if registry is None:
         registry = getattr(request, 'registry', global_registries.last)
@@ -94,6 +100,8 @@ def prepare(request=None, registry=None):
     apply_request_extensions(request)
 
     def closer():
+        if request.finished_callbacks:
+            request._process_finished_callbacks()
         ctx.end()
 
     root_factory = registry.queryUtility(


Test patch:
diff --git a/tests/test_request.py b/tests/test_request.py
--- a/tests/test_request.py
+++ b/tests/test_request.py
@@ -535,10 +535,6 @@ def test_it_override_extensions(self):
         self.assertEqual(request.foo('abc'), 'abc')
 
 
-class Dummy(object):
-    pass
-
-
 class Test_subclassing_Request(unittest.TestCase):
     def test_subclass(self):
         from pyramid.interfaces import IRequest
@@ -598,14 +594,124 @@ class RequestSub(Request):
         self.assertTrue(IRequest.implementedBy(RequestSub))
 
 
+class TestRequestLocalCache(unittest.TestCase):
+    def _makeOne(self, *args, **kwargs):
+        from pyramid.request import RequestLocalCache
+
+        return RequestLocalCache(*args, **kwargs)
+
+    def test_it_works_with_functions(self):
+        a = [0]
+
+        @self._makeOne()
+        def foo(request):
+            a[0] += 1
+            return a[0]
+
+        req1 = DummyRequest()
+        req2 = DummyRequest()
+        self.assertEqual(foo(req1), 1)
+        self.assertEqual(foo(req2), 2)
+        self.assertEqual(foo(req1), 1)
+        self.assertEqual(foo(req2), 2)
+        self.assertEqual(len(req1.finished_callbacks), 1)
+        self.assertEqual(len(req2.finished_callbacks), 1)
+
+    def test_clear_works(self):
+        a = [0]
+
+        @self._makeOne()
+        def foo(request):
+            a[0] += 1
+            return a[0]
+
+        req = DummyRequest()
+        self.assertEqual(foo(req), 1)
+        self.assertEqual(len(req.finished_callbacks), 1)
+        foo.cache.clear(req)
+        self.assertEqual(foo(req), 2)
+        self.assertEqual(len(req.finished_callbacks), 1)
+
+    def test_set_overrides_current_value(self):
+        a = [0]
+
+        @self._makeOne()
+        def foo(request):
+            a[0] += 1
+            return a[0]
+
+        req = DummyRequest()
+        self.assertEqual(foo(req), 1)
+        self.assertEqual(len(req.finished_callbacks), 1)
+        foo.cache.set(req, 8)
+        self.assertEqual(foo(req), 8)
+        self.assertEqual(len(req.finished_callbacks), 1)
+        self.assertEqual(foo.cache.get(req), 8)
+
+    def test_get_works(self):
+        cache = self._makeOne()
+        req = DummyRequest()
+        self.assertIs(cache.get(req), cache.NO_VALUE)
+        cache.set(req, 2)
+        self.assertIs(cache.get(req), 2)
+
+    def test_creator_in_constructor(self):
+        def foo(request):
+            return 8
+
+        cache = self._makeOne(foo)
+        req = DummyRequest()
+        result = cache.get_or_create(req)
+        self.assertEqual(result, 8)
+
+    def test_decorator_overrides_creator(self):
+        def foo(request):  # pragma: no cover
+            raise AssertionError
+
+        cache = self._makeOne(foo)
+
+        @cache
+        def bar(request):
+            return 8
+
+        req = DummyRequest()
+        result = cache.get_or_create(req)
+        self.assertEqual(result, 8)
+
+    def test_get_or_create_overrides_creator(self):
+        cache = self._makeOne()
+
+        @cache
+        def foo(request):  # pragma: no cover
+            raise AssertionError
+
+        req = DummyRequest()
+        result = cache.get_or_create(req, lambda r: 8)
+        self.assertEqual(result, 8)
+
+    def test_get_or_create_with_no_creator(self):
+        cache = self._makeOne()
+        req = DummyRequest()
+        self.assertRaises(ValueError, cache.get_or_create, req)
+
+
+class Dummy(object):
+    pass
+
+
 class DummyRequest(object):
     def __init__(self, environ=None):
         if environ is None:
             environ = {}
         self.environ = environ
+        self.response_callbacks = []
+        self.finished_callbacks = []
 
     def add_response_callback(self, callback):
-        self.response_callbacks = [callback]
+        self.response_callbacks.append(callback)
+
+    def add_finished_callback(self, callback):
+        self.finished_callbacks.append(callback)
 
     def get_response(self, app):
         return app
diff --git a/tests/test_scripting.py b/tests/test_scripting.py
--- a/tests/test_scripting.py
+++ b/tests/test_scripting.py
@@ -1,3 +1,4 @@
+from collections import deque
 import unittest
 
 
@@ -162,6 +163,20 @@ def closer():
         self.assertEqual(request.context, root)
         self.assertEqual(request.registry, registry)
 
+    def test_closer_invokes_finished_callbacks(self):
+        finish_called = [False]
+
+        def finished_callback(request):
+            finish_called[0] = True
+
+        request = DummyRequest({})
+        request.registry = self._makeRegistry()
+        info = self._callFUT(request=request)
+        request.add_finished_callback(finished_callback)
+        closer = info['closer']
+        closer()
+        self.assertTrue(finish_called[0])
+
 
 class Test__make_request(unittest.TestCase):
     def _callFUT(self, path='/', registry=None):
@@ -234,6 +249,15 @@ class DummyRequest(object):
 
     def __init__(self, environ):
         self.environ = environ
+        self.finished_callbacks = deque()
+
+    def add_finished_callback(self, cb):
+        self.finished_callbacks.append(cb)
+
+    def _process_finished_callbacks(self):
+        while self.finished_callbacks:
+            cb = self.finished_callbacks.popleft()
+            cb(self)
 
 
 class DummyExtensions: