Repository: Pylons/pyramid

Issue:
ICacheBuster interface leaks some implementation details
The `ICacheBuster` interface has a method, `token(pathspec)` which is used to generate a token for the given `pathspec`. As far as I can tell the only place this method is used is for calling code to generate a token and then to immediate pass it into `ICacheBuster.pregenerate`.

I feel like baking this `token()` method into the `ICacheBuster` interface is leaking the implementation details of the built in cache busting methods into the interface. Conceptually what you need in the cache busting interface is a way to convert and URL back and forth between the non-cachebusting and the cachebusting form. However this API assumes that Pyramid is the tool that is generating the token or the cache buster URLs. This doesn't always hold true though, for instance you might have a static asset pipeline outside of Pyramid that generates a manifest file and then your `ICacheBuster` simply needs to look up the path in that manifest file, but since the `token()` method is mandatory, you're forced to write a no-op method when you don't need this method.

Suggested on IRC was, I think, a better way of expressing this:
- Drop the `ICacheBuster.token(pathspec)` method.
- Modify `ICacheBuster.pregenerate(token, subpath, kw)` to `ICacheBuster.pregenerate(pathspec, subpath, kw)`.
- Modify `PathSegmentMd5CacheBuster` so that they can accept a token generating function (e.g. `PathSegmentMd5CacheBuster(tokenizer=Md5AssetTokenGenerator())` and they'll simply call that inside of their `pregenerate` function.

Since this API hasn't been released yet it may be a good idea to review it before there are backwards compatibility considerations to take into account.



Discussion:
I think it'd be nice to have generic `PathSegmentCacheBuster` and `QueryStringCacheBuster` objects that are not coupled to md5. We can do this pretty easily via composition + mixins with very minimal changes to the current codebase.

``` python
class PathSegmentCacheBuster(object):
    def __init__(self, tokenizer=None):
        if tokenizer is not None:
            self.tokenize = tokenizer

    def pregenerate(self, pathspec, subpath, kw):
        token = self.tokenize(pathspec)
        return (token,) + subpath, kw

    def match(self, subpath):
        return subpath[1:]

class MD5AssetTokenGenerator(object):
    def tokenize(self, pathspec):
        return hashlib.md5(pathspec).hexdigest()

class PathSegmentMD5CacheBuster(PathSegmentCacheBuster, MD5AssetTokenGenerator):
    def __init__(self):
        PathSegmentCacheBuster.__init__(self)
        MD5AssetTokenGenerator.__init__(self)
```

I really like the above, composition is great and it gives all of niceties of the current API without forcing people who aren't generating tokens in Pyramid to have a no-op `token()` method.


Resolving patch:
diff --git a/pyramid/config/views.py b/pyramid/config/views.py
--- a/pyramid/config/views.py
+++ b/pyramid/config/views.py
@@ -1980,9 +1980,9 @@ def add(self, config, name, spec, **extra):
             cb = self._default_cachebust()
         if cb:
             def cachebust(subpath, kw):
-                token = cb.token(spec + subpath)
                 subpath_tuple = tuple(subpath.split('/'))
-                subpath_tuple, kw = cb.pregenerate(token, subpath_tuple, kw)
+                subpath_tuple, kw = cb.pregenerate(
+                    spec + subpath, subpath_tuple, kw)
                 return '/'.join(subpath_tuple), kw
         else:
             cachebust = None
diff --git a/pyramid/interfaces.py b/pyramid/interfaces.py
--- a/pyramid/interfaces.py
+++ b/pyramid/interfaces.py
@@ -1192,18 +1192,11 @@ class ICacheBuster(Interface):
 
     .. versionadded:: 1.6
     """
-    def token(pathspec):
-        """
-        Computes and returns a token string used for cache busting.
-        ``pathspec`` is the path specification for the resource to be cache
-        busted.  """
-
-    def pregenerate(token, subpath, kw):
+    def pregenerate(pathspec, subpath, kw):
         """
         Modifies a subpath and/or keyword arguments from which a static asset
-        URL will be computed during URL generation.  The ``token`` argument is
-        a token string computed by
-        :meth:`~pyramid.interfaces.ICacheBuster.token` for a particular asset.
+        URL will be computed during URL generation.  The ``pathspec`` argument
+        is the path specification for the resource to be cache busted.
         The ``subpath`` argument is a tuple of path elements that represent the
         portion of the asset URL which is used to find the asset.  The ``kw``
         argument is a dict of keywords that are to be passed eventually to
diff --git a/pyramid/static.py b/pyramid/static.py
--- a/pyramid/static.py
+++ b/pyramid/static.py
@@ -174,7 +174,7 @@ class Md5AssetTokenGenerator(object):
     def __init__(self):
         self.token_cache = {}
 
-    def token(self, pathspec):
+    def tokenize(self, pathspec):
         # An astute observer will notice that this use of token_cache doesn't
         # look particularly thread safe.  Basic read/write operations on Python
         # dicts, however, are atomic, so simply accessing and writing values
@@ -192,38 +192,54 @@ def token(self, pathspec):
             self.token_cache[pathspec] = token = _generate_md5(pathspec)
         return token
 
-class PathSegmentMd5CacheBuster(Md5AssetTokenGenerator):
+class PathSegmentCacheBuster(object):
     """
     An implementation of :class:`~pyramid.interfaces.ICacheBuster` which
-    inserts an md5 checksum token for cache busting in the path portion of an
-    asset URL.  Generated md5 checksums are cached in order to speed up
-    subsequent calls.
+    inserts a token for cache busting in the path portion of an asset URL.
+
+    To use this class, subclass it and provide a ``tokenize`` method which
+    accepts a ``pathspec`` and returns a token.
 
     .. versionadded:: 1.6
     """
-    def pregenerate(self, token, subpath, kw):
+    def pregenerate(self, pathspec, subpath, kw):
+        token = self.tokenize(pathspec)
         return (token,) + subpath, kw
 
     def match(self, subpath):
         return subpath[1:]
 
-class QueryStringMd5CacheBuster(Md5AssetTokenGenerator):
+class PathSegmentMd5CacheBuster(PathSegmentCacheBuster,
+                                Md5AssetTokenGenerator):
+    """
+    An implementation of :class:`~pyramid.interfaces.ICacheBuster` which
+    inserts an md5 checksum token for cache busting in the path portion of an
+    asset URL.  Generated md5 checksums are cached in order to speed up
+    subsequent calls.
+
+    .. versionadded:: 1.6
+    """
+    def __init__(self):
+        super(PathSegmentMd5CacheBuster, self).__init__()
+
+class QueryStringCacheBuster(object):
     """
     An implementation of :class:`~pyramid.interfaces.ICacheBuster` which adds
-    an md5 checksum token for cache busting in the query string of an asset
-    URL.  Generated md5 checksums are cached in order to speed up subsequent
-    calls.
+    a token for cache busting in the query string of an asset URL.
 
     The optional ``param`` argument determines the name of the parameter added
     to the query string and defaults to ``'x'``.
 
+    To use this class, subclass it and provide a ``tokenize`` method which
+    accepts a ``pathspec`` and returns a token.
+
     .. versionadded:: 1.6
     """
     def __init__(self, param='x'):
-        super(QueryStringMd5CacheBuster, self).__init__()
         self.param = param
 
-    def pregenerate(self, token, subpath, kw):
+    def pregenerate(self, pathspec, subpath, kw):
+        token = self.tokenize(pathspec)
         query = kw.setdefault('_query', {})
         if isinstance(query, dict):
             query[self.param] = token
@@ -231,7 +247,23 @@ def pregenerate(self, token, subpath, kw):
             kw['_query'] = tuple(query) + ((self.param, token),)
         return subpath, kw
 
-class QueryStringConstantCacheBuster(QueryStringMd5CacheBuster):
+class QueryStringMd5CacheBuster(QueryStringCacheBuster,
+                                Md5AssetTokenGenerator):
+    """
+    An implementation of :class:`~pyramid.interfaces.ICacheBuster` which adds
+    an md5 checksum token for cache busting in the query string of an asset
+    URL.  Generated md5 checksums are cached in order to speed up subsequent
+    calls.
+
+    The optional ``param`` argument determines the name of the parameter added
+    to the query string and defaults to ``'x'``.
+
+    .. versionadded:: 1.6
+    """
+    def __init__(self, param='x'):
+        super(QueryStringMd5CacheBuster, self).__init__(param=param)
+
+class QueryStringConstantCacheBuster(QueryStringCacheBuster):
     """
     An implementation of :class:`~pyramid.interfaces.ICacheBuster` which adds
     an arbitrary token for cache busting in the query string of an asset URL.
@@ -245,9 +277,8 @@ class QueryStringConstantCacheBuster(QueryStringMd5CacheBuster):
     .. versionadded:: 1.6
     """
     def __init__(self, token, param='x'):
+        super(QueryStringConstantCacheBuster, self).__init__(param=param)
         self._token = token
-        self.param = param
 
-    def token(self, pathspec):
+    def tokenize(self, pathspec):
         return self._token
-


Test patch:
diff --git a/pyramid/tests/test_config/test_views.py b/pyramid/tests/test_config/test_views.py
--- a/pyramid/tests/test_config/test_views.py
+++ b/pyramid/tests/test_config/test_views.py
@@ -3995,7 +3995,7 @@ def test_add_viewname_with_renderer(self):
     def test_add_cachebust_default(self):
         config = self._makeConfig()
         inst = self._makeOne()
-        inst._default_cachebust = DummyCacheBuster
+        inst._default_cachebust = lambda: DummyCacheBuster('foo')
         inst.add(config, 'view', 'mypackage:path', cachebust=True)
         cachebust = config.registry._static_url_registrations[0][3]
         subpath, kw = cachebust('some/path', {})
@@ -4014,7 +4014,7 @@ def test_add_cachebust_custom(self):
         config = self._makeConfig()
         inst = self._makeOne()
         inst.add(config, 'view', 'mypackage:path',
-                 cachebust=DummyCacheBuster())
+                 cachebust=DummyCacheBuster('foo'))
         cachebust = config.registry._static_url_registrations[0][3]
         subpath, kw = cachebust('some/path', {})
         self.assertEqual(subpath, 'some/path')
@@ -4127,10 +4127,10 @@ def __permitted__(self, context, request):
         """ """
 
 class DummyCacheBuster(object):
-    def token(self, pathspec):
-        return 'foo'
-    def pregenerate(self, token, subpath, kw):
-        kw['x'] = token
+    def __init__(self, token):
+        self.token = token
+    def pregenerate(self, pathspec, subpath, kw):
+        kw['x'] = self.token
         return subpath, kw
 
 def parse_httpdate(s):
diff --git a/pyramid/tests/test_static.py b/pyramid/tests/test_static.py
--- a/pyramid/tests/test_static.py
+++ b/pyramid/tests/test_static.py
@@ -393,13 +393,13 @@ def _makeOne(self):
         return cls()
 
     def test_package_resource(self):
-        fut = self._makeOne().token
+        fut = self._makeOne().tokenize
         expected = '76d653a3a044e2f4b38bb001d283e3d9'
         token = fut('pyramid.tests:fixtures/static/index.html')
         self.assertEqual(token, expected)
 
     def test_filesystem_resource(self):
-        fut = self._makeOne().token
+        fut = self._makeOne().tokenize
         expected = 'd5155f250bef0e9923e894dbc713c5dd'
         with open(self.fspath, 'w') as f:
             f.write("Are we rich yet?")
@@ -407,7 +407,7 @@ def test_filesystem_resource(self):
         self.assertEqual(token, expected)
 
     def test_cache(self):
-        fut = self._makeOne().token
+        fut = self._makeOne().tokenize
         expected = 'd5155f250bef0e9923e894dbc713c5dd'
         with open(self.fspath, 'w') as f:
             f.write("Are we rich yet?")
@@ -425,11 +425,11 @@ class TestPathSegmentMd5CacheBuster(unittest.TestCase):
     def _makeOne(self):
         from pyramid.static import PathSegmentMd5CacheBuster as cls
         inst = cls()
-        inst.token = lambda pathspec: 'foo'
+        inst.tokenize = lambda pathspec: 'foo'
         return inst
 
     def test_token(self):
-        fut = self._makeOne().token
+        fut = self._makeOne().tokenize
         self.assertEqual(fut('whatever'), 'foo')
 
     def test_pregenerate(self):
@@ -448,11 +448,11 @@ def _makeOne(self, param=None):
             inst = cls(param)
         else:
             inst = cls()
-        inst.token = lambda pathspec: 'foo'
+        inst.tokenize = lambda pathspec: 'foo'
         return inst
 
     def test_token(self):
-        fut = self._makeOne().token
+        fut = self._makeOne().tokenize
         self.assertEqual(fut('whatever'), 'foo')
 
     def test_pregenerate(self):
@@ -490,7 +490,7 @@ def _makeOne(self, param=None):
         return inst
 
     def test_token(self):
-        fut = self._makeOne().token
+        fut = self._makeOne().tokenize
         self.assertEqual(fut('whatever'), 'foo')
 
     def test_pregenerate(self):