Repository: django-cms/django-cms

Issue:
show_placeholder doesn't add CSS
I use my plugins:

**3columns.html:**

``` html
{% extends 'mainframe.html' %}
{% load cms_tags placeholder_tags %}
{% block content %}
<div class="c3-left clearfix">{% block column-left %}{% placeholder "left_column" or %}{% show_placeholder "left_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div>
<div class="c3-center clearfix">{% block column-center %}{% placeholder "content" %}{% endblock %}</div>
<div class="c3-right clearfix">{% block column-right %}{% placeholder "right_column" or %}{% show_placeholder "right_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div>
{% endblock %}
```

**plugin.html:**

``` html
{% load i18n sekizai_tags %}
{% addtoblock "css" %}<link rel="stylesheet" href="{{ STATIC_URL }}basic_jobs_trades_countries_listing_plugin/css/style.css" type="text/css" media="screen" />{% endaddtoblock %}
<div class="small-column-closed-wrapper basic-jobs-listing-plugin trades">
    <h2></h2>
    <div class="content">
        <ul class=>
            {% for trade in trades %}
                <li>
                    <a href="#">{{ trade.name }}</a>
                </li>
            {% endfor %}
        </ul>
    </div>
</div>
<div class="small-column-closed-wrapper basic-jobs-listing-plugin countries">
    <h2></h2>
    <div class="content">
        <ul class=>
            {% for country in countries %}
                <li>
                    <a href="#">{{ country.name }}</a>
                </li>
            {% endfor %}
        </ul>
    </div>
</div>
```

The problem is, after server restart all is ok, but after next page reload (subpage, not index) CSS declared in plugin is not being added. When I change :
`html{% show_placeholder "left_column" 'homepage' %}` to `html{% show_uncached_placeholder "left_column" 'homepage' %}` ALL IS OK.

show_placeholder doesn't add CSS
I use my plugins:

**3columns.html:**

``` html
{% extends 'mainframe.html' %}
{% load cms_tags placeholder_tags %}
{% block content %}
<div class="c3-left clearfix">{% block column-left %}{% placeholder "left_column" or %}{% show_placeholder "left_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div>
<div class="c3-center clearfix">{% block column-center %}{% placeholder "content" %}{% endblock %}</div>
<div class="c3-right clearfix">{% block column-right %}{% placeholder "right_column" or %}{% show_placeholder "right_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div>
{% endblock %}
```

**plugin.html:**

``` html
{% load i18n sekizai_tags %}
{% addtoblock "css" %}<link rel="stylesheet" href="{{ STATIC_URL }}basic_jobs_trades_countries_listing_plugin/css/style.css" type="text/css" media="screen" />{% endaddtoblock %}
<div class="small-column-closed-wrapper basic-jobs-listing-plugin trades">
    <h2></h2>
    <div class="content">
        <ul class=>
            {% for trade in trades %}
                <li>
                    <a href="#">{{ trade.name }}</a>
                </li>
            {% endfor %}
        </ul>
    </div>
</div>
<div class="small-column-closed-wrapper basic-jobs-listing-plugin countries">
    <h2></h2>
    <div class="content">
        <ul class=>
            {% for country in countries %}
                <li>
                    <a href="#">{{ country.name }}</a>
                </li>
            {% endfor %}
        </ul>
    </div>
</div>
```

The problem is, after server restart all is ok, but after next page reload (subpage, not index) CSS declared in plugin is not being added. When I change :
`html{% show_placeholder "left_column" 'homepage' %}` to `html{% show_uncached_placeholder "left_column" 'homepage' %}` ALL IS OK.



Discussion:
Problem is cached content forgets about sekizai. We need to make caching
also include the sekizai blocks somehow. Arguably a sekizai issue.
On Jan 8, 2013 4:23 PM, "Robert Węglarek" notifications@github.com wrote:

> I use my plugins:
> 
> _3columns.html:_
> 
> {% extends 'mainframe.html' %}
> {% load cms_tags placeholder_tags %}
> {% block content %}<div class="c3-left clearfix">{% block column-left %}{% placeholder "left_column" or %}{% show_placeholder "left_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div><div class="c3-center clearfix">{% block column-center %}{% placeholder "content" %}{% endblock %}</div><div class="c3-right clearfix">{% block column-right %}{% placeholder "right_column" or %}{% show_placeholder "right_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div>
> {% endblock %}
> 
> _plugin.html:_
> 
> {% load i18n sekizai_tags %}
> {% addtoblock "css" %}<link rel="stylesheet" href="{{ STATIC_URL }}basic_jobs_trades_countries_listing_plugin/css/style.css" type="text/css" media="screen" />{% endaddtoblock %}<div class="small-column-closed-wrapper basic-jobs-listing-plugin trades">
>     <h2></h2>
>     <div class="content">
>         <ul class=>
>             {% for trade in trades %}
>                 <li>
>                     <a href="#">{{ trade.name }}</a>
>                 </li>
>             {% endfor %}
>         </ul>
>     </div></div><div class="small-column-closed-wrapper basic-jobs-listing-plugin countries">
>     <h2></h2>
>     <div class="content">
>         <ul class=>
>             {% for country in countries %}
>                 <li>
>                     <a href="#">{{ country.name }}</a>
>                 </li>
>             {% endfor %}
>         </ul>
>     </div></div>
> 
> The problem is, after server restart all is ok, but after next page reload
> (subpage, not index) CSS declared in plugin is not being added. When I
> change :
> html{% show_placeholder "left_column" 'homepage' %} to html{%
> show_uncached_placeholder "left_column" 'homepage' %} ALL IS OK.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/divio/django-cms/issues/1577.

Okay now I'm at my laptop explain me the situation a little:

`show_placeholder` by default caches it's result. Therefore, the template is only executed when it hits a cold cache. Since sekizai uses templates, it's blocks are only injected when the template is executed. If `show_placeholder` hits a warm cache, it just renders the contents from the cache, not executing the templates, thus not triggering sekizai.

The solutions has two steps (the first being optional):
1. Sekizai implements a way to "watch" for data being added. This will most likely be a context manager.
2. django CMS caches the sekizai data along the rendered HTML.

Basically in CMS land this would turn  `_show_placeholder_for_page` into something like this:

``` python

def _show_placeholder_for_page(context, placeholder_name, page_lookup, lang=None,
        site=None, cache_result=True):
    """
    Shows the content of a page with a placeholder name and given lookup
    arguments in the given language.
    This is useful if you want to have some more or less static content that is
    shared among many pages, such as a footer.

    See _get_page_by_untyped_arg() for detailed information on the allowed types
    and their interpretation for the page_lookup argument.
    """
    validate_placeholder_name(placeholder_name)

    request = context.get('request', False)
    site_id = get_site_id(site)

    if not request:
        return {'content': ''}
    if lang is None:
        lang = get_language_from_request(request)

    content = None

    if cache_result:
        base_key = _get_cache_key('_show_placeholder_for_page', page_lookup, lang, site_id)
        cache_key = _clean_key('%s_placeholder:%s' % (base_key, placeholder_name))
        cached = cache.get(cache_key)
        if cached:
            content = cached['content']
            sekizai.inject_diff(cached['sekizai']) # inject sekizai data into context

    if not content:
        page = _get_page_by_untyped_arg(page_lookup, request, site_id)
        if not page:
            return {'content': ''}
        try:
            placeholder = page.placeholders.get(slot=placeholder_name)
        except PlaceholderModel.DoesNotExist:
            if settings.DEBUG:
                raise
            return {'content': ''}
        differ = sekizai.Differ(context) # start watching the context
        content = render_placeholder(placeholder, context, placeholder_name)
        sekizai_diff = differ.get_diff() # get the sekizai changes
    if cache_result:
        cache.set(cache_key, {'content': content, 'sekizai': sekizai_diff}, settings.CMS_CACHE_DURATIONS['content'])

    if content:
        return {'content': mark_safe(content)}
    return {'content': ''}
```

Which obviously needs sekizai to implement those APIs (I think it's better I do it there rather than the CMS implementing it itself).

If this sounds reasonable I'll starting implementing the required APIs in sekizai.

Is it included in 2.4b?

no this was not included in 2.4b1

@digi604 before the next beta/rc, this really has to be resolved. I'll see how much I can get done tonight to solve this issue.

Problem is cached content forgets about sekizai. We need to make caching
also include the sekizai blocks somehow. Arguably a sekizai issue.
On Jan 8, 2013 4:23 PM, "Robert Węglarek" notifications@github.com wrote:

> I use my plugins:
> 
> _3columns.html:_
> 
> {% extends 'mainframe.html' %}
> {% load cms_tags placeholder_tags %}
> {% block content %}<div class="c3-left clearfix">{% block column-left %}{% placeholder "left_column" or %}{% show_placeholder "left_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div><div class="c3-center clearfix">{% block column-center %}{% placeholder "content" %}{% endblock %}</div><div class="c3-right clearfix">{% block column-right %}{% placeholder "right_column" or %}{% show_placeholder "right_column" 'homepage' %}{% endplaceholder %}{% endblock %}</div>
> {% endblock %}
> 
> _plugin.html:_
> 
> {% load i18n sekizai_tags %}
> {% addtoblock "css" %}<link rel="stylesheet" href="{{ STATIC_URL }}basic_jobs_trades_countries_listing_plugin/css/style.css" type="text/css" media="screen" />{% endaddtoblock %}<div class="small-column-closed-wrapper basic-jobs-listing-plugin trades">
>     <h2></h2>
>     <div class="content">
>         <ul class=>
>             {% for trade in trades %}
>                 <li>
>                     <a href="#">{{ trade.name }}</a>
>                 </li>
>             {% endfor %}
>         </ul>
>     </div></div><div class="small-column-closed-wrapper basic-jobs-listing-plugin countries">
>     <h2></h2>
>     <div class="content">
>         <ul class=>
>             {% for country in countries %}
>                 <li>
>                     <a href="#">{{ country.name }}</a>
>                 </li>
>             {% endfor %}
>         </ul>
>     </div></div>
> 
> The problem is, after server restart all is ok, but after next page reload
> (subpage, not index) CSS declared in plugin is not being added. When I
> change :
> html{% show_placeholder "left_column" 'homepage' %} to html{%
> show_uncached_placeholder "left_column" 'homepage' %} ALL IS OK.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/divio/django-cms/issues/1577.

Okay now I'm at my laptop explain me the situation a little:

`show_placeholder` by default caches it's result. Therefore, the template is only executed when it hits a cold cache. Since sekizai uses templates, it's blocks are only injected when the template is executed. If `show_placeholder` hits a warm cache, it just renders the contents from the cache, not executing the templates, thus not triggering sekizai.

The solutions has two steps (the first being optional):
1. Sekizai implements a way to "watch" for data being added. This will most likely be a context manager.
2. django CMS caches the sekizai data along the rendered HTML.

Basically in CMS land this would turn  `_show_placeholder_for_page` into something like this:

``` python

def _show_placeholder_for_page(context, placeholder_name, page_lookup, lang=None,
        site=None, cache_result=True):
    """
    Shows the content of a page with a placeholder name and given lookup
    arguments in the given language.
    This is useful if you want to have some more or less static content that is
    shared among many pages, such as a footer.

    See _get_page_by_untyped_arg() for detailed information on the allowed types
    and their interpretation for the page_lookup argument.
    """
    validate_placeholder_name(placeholder_name)

    request = context.get('request', False)
    site_id = get_site_id(site)

    if not request:
        return {'content': ''}
    if lang is None:
        lang = get_language_from_request(request)

    content = None

    if cache_result:
        base_key = _get_cache_key('_show_placeholder_for_page', page_lookup, lang, site_id)
        cache_key = _clean_key('%s_placeholder:%s' % (base_key, placeholder_name))
        cached = cache.get(cache_key)
        if cached:
            content = cached['content']
            sekizai.inject_diff(cached['sekizai']) # inject sekizai data into context

    if not content:
        page = _get_page_by_untyped_arg(page_lookup, request, site_id)
        if not page:
            return {'content': ''}
        try:
            placeholder = page.placeholders.get(slot=placeholder_name)
        except PlaceholderModel.DoesNotExist:
            if settings.DEBUG:
                raise
            return {'content': ''}
        differ = sekizai.Differ(context) # start watching the context
        content = render_placeholder(placeholder, context, placeholder_name)
        sekizai_diff = differ.get_diff() # get the sekizai changes
    if cache_result:
        cache.set(cache_key, {'content': content, 'sekizai': sekizai_diff}, settings.CMS_CACHE_DURATIONS['content'])

    if content:
        return {'content': mark_safe(content)}
    return {'content': ''}
```

Which obviously needs sekizai to implement those APIs (I think it's better I do it there rather than the CMS implementing it itself).

If this sounds reasonable I'll starting implementing the required APIs in sekizai.

Is it included in 2.4b?

no this was not included in 2.4b1

@digi604 before the next beta/rc, this really has to be resolved. I'll see how much I can get done tonight to solve this issue.


Resolving patch:
diff --git a/cms/templatetags/cms_tags.py b/cms/templatetags/cms_tags.py
--- a/cms/templatetags/cms_tags.py
+++ b/cms/templatetags/cms_tags.py
@@ -19,6 +19,7 @@
 from django.utils.translation import ugettext_lazy as _, get_language
 from itertools import chain
 import re
+from sekizai.helpers import Watcher, get_varname
 
 
 register = template.Library()
@@ -368,6 +369,15 @@ def get_context(self, context, cl, spec):
         return {'title': spec.title(), 'choices': unique_choices}
 
 
+def _restore_sekizai(context, changes):
+    varname = get_varname()
+    sekizai_container = context[varname]
+    for key, values in changes.items():
+        sekizai_namespace = sekizai_container[key]
+        for value in values:
+            sekizai_namespace.append(value)
+
+
 def _show_placeholder_for_page(context, placeholder_name, page_lookup, lang=None,
         site=None, cache_result=True):
     """
@@ -389,26 +399,30 @@ def _show_placeholder_for_page(context, placeholder_name, page_lookup, lang=None
     if lang is None:
         lang = get_language_from_request(request)
 
-    content = None
-
     if cache_result:
         base_key = _get_cache_key('_show_placeholder_for_page', page_lookup, lang, site_id)
         cache_key = _clean_key('%s_placeholder:%s' % (base_key, placeholder_name))
-        content = cache.get(cache_key)
-
-    if not content:
-        page = _get_page_by_untyped_arg(page_lookup, request, site_id)
-        if not page:
-            return {'content': ''}
-        try:
-            placeholder = page.placeholders.get(slot=placeholder_name)
-        except PlaceholderModel.DoesNotExist:
-            if settings.DEBUG:
-                raise
-            return {'content': ''}
-        content = render_placeholder(placeholder, context, placeholder_name)
+        cached_value = cache.get(cache_key)
+        if isinstance(cached_value, dict): # new style
+            _restore_sekizai(context, cached_value['sekizai'])
+            return {'content': cached_value['content']}
+        elif isinstance(cached_value, basestring): # old style
+            return {'content': cached_value}
+
+    page = _get_page_by_untyped_arg(page_lookup, request, site_id)
+    if not page:
+        return {'content': ''}
+    try:
+        placeholder = page.placeholders.get(slot=placeholder_name)
+    except PlaceholderModel.DoesNotExist:
+        if settings.DEBUG:
+            raise
+        return {'content': ''}
+    watcher = Watcher(context)
+    content = render_placeholder(placeholder, context, placeholder_name)
+    changes = watcher.get_changes()
     if cache_result:
-        cache.set(cache_key, content, get_cms_setting('CACHE_DURATIONS')['content'])
+        cache.set(cache_key, {'content': content, 'sekizai': changes}, get_cms_setting('CACHE_DURATIONS')['content'])
 
     if content:
         return {'content': mark_safe(content)}
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,7 +33,7 @@
         'south>=0.7.2',
         'html5lib',
         'django-mptt>=0.5.1,<0.5.3',
-        'django-sekizai>=0.6.1',
+        'django-sekizai>=0.7',
     ],
     tests_require=[
         'django-reversion>=1.6',


Test patch:
diff --git a/cms/test_utils/project/templates/alt_plugin_templates/show_placeholder/cms/plugins/text.html b/cms/test_utils/project/templates/alt_plugin_templates/show_placeholder/cms/plugins/text.html
new file mode 100644
--- /dev/null
+++ b/cms/test_utils/project/templates/alt_plugin_templates/show_placeholder/cms/plugins/text.html
@@ -0,0 +1 @@
+{% load sekizai_tags %}{% addtoblock "js" %}JAVASCRIPT{% endaddtoblock %}
diff --git a/cms/tests/templatetags.py b/cms/tests/templatetags.py
--- a/cms/tests/templatetags.py
+++ b/cms/tests/templatetags.py
@@ -1,7 +1,10 @@
 from __future__ import with_statement
 import copy
-from cms.api import create_page, create_title
+from django.test import RequestFactory, TestCase
+import os
+from cms.api import create_page, create_title, add_plugin
 from cms.models.pagemodel import Page, Placeholder
+from cms.plugins.text.cms_plugins import TextPlugin
 from cms.templatetags.cms_tags import (get_site_id, _get_page_by_untyped_arg,
         _show_placeholder_for_page)
 from cms.test_utils.fixtures.templatetags import TwoPagesFixture
@@ -9,13 +12,11 @@
 from cms.test_utils.util.context_managers import SettingsOverride
 from cms.utils import get_cms_setting
 from cms.utils.plugins import get_placeholders
-from django.conf import settings
 from django.contrib.sites.models import Site
 from django.core import mail
 from django.core.exceptions import ImproperlyConfigured
 from django.http import HttpRequest
 from django.template import RequestContext, Context
-from unittest import TestCase
 from django.template.base import Template
 from django.utils.html import escape
 
@@ -60,13 +61,6 @@ class FakeRequest(object):
 
 
 class TemplatetagDatabaseTests(TwoPagesFixture, SettingsOverrideTestCase):
-
-    def setUp(self):
-        self._prev_DEBUG = settings.DEBUG
-
-    def tearDown(self):
-        settings.DEBUG = self._prev_DEBUG
-
     def _getfirst(self):
         return Page.objects.public().get(title_set__title='first')
 
@@ -129,19 +123,19 @@ def test_show_placeholder_for_page_placeholder_does_not_exist(self):
         Verify ``show_placeholder`` correctly handles being given an
         invalid identifier.
         """
-        settings.DEBUG = True # So we can see the real exception raised
-        request = HttpRequest()
-        request.REQUEST = {}
-        request.session = {}
-        self.assertRaises(Placeholder.DoesNotExist,
-                          _show_placeholder_for_page,
-                          RequestContext(request),
-                          'does_not_exist',
-                          'myreverseid')
-        settings.DEBUG = False # Now test the non-debug output
-        content = _show_placeholder_for_page(RequestContext(request),
-                                            'does_not_exist', 'myreverseid')
-        self.assertEqual(content['content'], '')
+        with SettingsOverride(DEBUG=True):
+            request = HttpRequest()
+            request.REQUEST = {}
+            request.session = {}
+            self.assertRaises(Placeholder.DoesNotExist,
+                              _show_placeholder_for_page,
+                              RequestContext(request),
+                              'does_not_exist',
+                              'myreverseid')
+        with SettingsOverride(DEBUG=False):
+            content = _show_placeholder_for_page(RequestContext(request),
+                                                'does_not_exist', 'myreverseid')
+            self.assertEqual(content['content'], '')
 
     def test_untranslated_language_url(self):
         """ Tests page_language_url templatetag behavior when used on a page
@@ -192,3 +186,21 @@ def test_untranslated_language_url(self):
             context['request'].current_page = page_3.publisher_public
             res = tpl.render(context)
             self.assertEqual(res,"/de/")
+
+
+class NoFixtureDatabaseTemplateTagTests(TestCase):
+    def test_cached_show_placeholder_sekizai(self):
+        from cms.test_utils import project
+        template_dir = os.path.join(os.path.dirname(project.__file__), 'templates', 'alt_plugin_templates', 'show_placeholder')
+        page = create_page('Test', 'col_two.html', 'en')
+        placeholder = page.placeholders.all()[0]
+        add_plugin(placeholder, TextPlugin, 'en', body='HIDDEN')
+        request = RequestFactory().get('/')
+        with SettingsOverride(TEMPLATE_DIRS=[template_dir]):
+            template = Template("{% load cms_tags sekizai_tags %}{% show_placeholder slot page 'en' 1 %}{% render_block 'js' %}")
+            context = RequestContext(request, {'page': page, 'slot': placeholder.slot})
+            output = template.render(context)
+            self.assertIn('JAVASCRIPT', output)
+            context = RequestContext(request, {'page': page, 'slot': placeholder.slot})
+            output = template.render(context)
+            self.assertIn('JAVASCRIPT', output)
diff --git a/test_requirements/requirements_base.txt b/test_requirements/requirements_base.txt
--- a/test_requirements/requirements_base.txt
+++ b/test_requirements/requirements_base.txt
@@ -6,7 +6,7 @@ django-classy-tags>=0.3.4.1
 South>=0.7.2
 html5lib>=0.90
 django-mptt==0.5.2
-django-sekizai>=0.6.1
+django-sekizai>=0.7
 argparse
 dj-database-url
 transifex-client