Repository: django-compressor/django-compressor

Issue:
Recursion error when extending Django admin templates and compressing offline
Compressor throws a recursion error when trying to compress offline and admin extension templates are present:

```
$ python /www/360i/artifact/manage.py compress

Traceback (most recent call last):
  File "/www/360i/artifact/manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
    utility.execute()
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 342, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/management/commands/compress.py", line 286, in handle
    self.compress(sys.stdout, **options)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/management/commands/compress.py", line 181, in compress
    nodes = list(parser.walk_nodes(template))
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 150, in walk_nodes
    for node in self.walk_nodes(node, original):
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 146, in walk_nodes
    for node in self.get_nodelist(node, original):
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 128, in get_nodelist
    return handle_extendsnode(node, block_context=None, original=original)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 41, in handle_extendsnode
    return handle_extendsnode(node, block_context, original)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 41, in handle_extendsnode
    return handle_extendsnode(node, block_context, original)
  { ... CONTINUED ... }
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 30, in handle_extendsnode
    context = Context()
  File "/www/360i/local/lib/python2.7/site-packages/django/template/context.py", line 151, in __init__
    self.render_context = RenderContext()
  File "/www/360i/local/lib/python2.7/site-packages/django/template/context.py", line 34, in __init__
    self._reset_dicts(dict_)
RuntimeError: maximum recursion depth exceeded
```

This is due to the template extending a template that isn't defined in the application (admin/base_site.html is in [django.contrib.admin.templates](https://github.com/django/django/tree/master/django/contrib/admin/templates/admin)):

``` django
{% extends "admin/base_site.html" %}

{% block title %}360i{% endblock %}

{% block branding %}
<h1 id="site-name"><a href="{% url 'admin:index' %}">360i</a></h1>
{% endblock %}
```

Perhaps I missed a way to tell compressor to ignore certain templates?

Recursion error when extending Django admin templates and compressing offline
Compressor throws a recursion error when trying to compress offline and admin extension templates are present:

```
$ python /www/360i/artifact/manage.py compress

Traceback (most recent call last):
  File "/www/360i/artifact/manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
    utility.execute()
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 342, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/www/360i/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/management/commands/compress.py", line 286, in handle
    self.compress(sys.stdout, **options)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/management/commands/compress.py", line 181, in compress
    nodes = list(parser.walk_nodes(template))
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 150, in walk_nodes
    for node in self.walk_nodes(node, original):
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 146, in walk_nodes
    for node in self.get_nodelist(node, original):
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 128, in get_nodelist
    return handle_extendsnode(node, block_context=None, original=original)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 41, in handle_extendsnode
    return handle_extendsnode(node, block_context, original)
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 41, in handle_extendsnode
    return handle_extendsnode(node, block_context, original)
  { ... CONTINUED ... }
  File "/www/360i/local/lib/python2.7/site-packages/compressor/offline/django.py", line 30, in handle_extendsnode
    context = Context()
  File "/www/360i/local/lib/python2.7/site-packages/django/template/context.py", line 151, in __init__
    self.render_context = RenderContext()
  File "/www/360i/local/lib/python2.7/site-packages/django/template/context.py", line 34, in __init__
    self._reset_dicts(dict_)
RuntimeError: maximum recursion depth exceeded
```

This is due to the template extending a template that isn't defined in the application (admin/base_site.html is in [django.contrib.admin.templates](https://github.com/django/django/tree/master/django/contrib/admin/templates/admin)):

``` django
{% extends "admin/base_site.html" %}

{% block title %}360i{% endblock %}

{% block branding %}
<h1 id="site-name"><a href="{% url 'admin:index' %}">360i</a></h1>
{% endblock %}
```

Perhaps I missed a way to tell compressor to ignore certain templates?



Discussion:
Does it work with compressor disabled? This can happen due to template name conflict - for example http://stackoverflow.com/questions/25263277/django-template-extending-itself-maximum-recursion-depth-exceeded-when-trying-t

Yes, everything works correctly with compressor disabled.

If I remove these templates, offline compression works properly (manage.py compress).

My current workaround in my build script is to move these admin templates out of my templates directory, run compress and then copy them back.

Having the same issue with offline compression.

Seems to happen if a template is overriding and extending a template with the same name, for example `{% extends "admin/index.html" %}` in `admin/index.html` (in the project).

Does it work with compressor disabled? This can happen due to template name conflict - for example http://stackoverflow.com/questions/25263277/django-template-extending-itself-maximum-recursion-depth-exceeded-when-trying-t

Yes, everything works correctly with compressor disabled.

If I remove these templates, offline compression works properly (manage.py compress).

My current workaround in my build script is to move these admin templates out of my templates directory, run compress and then copy them back.

Having the same issue with offline compression.

Seems to happen if a template is overriding and extending a template with the same name, for example `{% extends "admin/index.html" %}` in `admin/index.html` (in the project).


Resolving patch:
diff --git a/compressor/offline/django.py b/compressor/offline/django.py
--- a/compressor/offline/django.py
+++ b/compressor/offline/django.py
@@ -6,31 +6,25 @@
 from django.template.base import Node, VariableNode, TextNode, NodeList
 from django.template.defaulttags import IfNode
 from django.template.loader import get_template
-from django.template.loader_tags import ExtendsNode, BlockNode, BlockContext
+from django.template.loader_tags import BLOCK_CONTEXT_KEY, ExtendsNode, BlockNode, BlockContext
 
 
 from compressor.exceptions import TemplateSyntaxError, TemplateDoesNotExist
 from compressor.templatetags.compress import CompressorNode
 
 
-def handle_extendsnode(extendsnode, block_context=None, original=None):
+def handle_extendsnode(extendsnode, context):
     """Create a copy of Node tree of a derived template replacing
     all blocks tags with the nodes of appropriate blocks.
     Also handles {{ block.super }} tags.
     """
-    if block_context is None:
-        block_context = BlockContext()
+    if BLOCK_CONTEXT_KEY not in context.render_context:
+        context.render_context[BLOCK_CONTEXT_KEY] = BlockContext()
+    block_context = context.render_context[BLOCK_CONTEXT_KEY]
     blocks = dict((n.name, n) for n in
                   extendsnode.nodelist.get_nodes_by_type(BlockNode))
     block_context.add_blocks(blocks)
 
-    # Note: we pass an empty context when we find the parent, this breaks
-    # inheritance using variables ({% extends template_var %}) but a refactor
-    # will be needed to support that use-case with multiple offline contexts.
-    context = Context()
-    if original is not None:
-        context.template = original
-
     compiled_parent = extendsnode.get_parent(context)
     parent_nodelist = compiled_parent.nodelist
     # If the parent template has an ExtendsNode it is not the root.
@@ -38,7 +32,7 @@ def handle_extendsnode(extendsnode, block_context=None, original=None):
         # The ExtendsNode has to be the first non-text node.
         if not isinstance(node, TextNode):
             if isinstance(node, ExtendsNode):
-                return handle_extendsnode(node, block_context, original)
+                return handle_extendsnode(node, context)
             break
     # Add blocks of the root template to block context.
     blocks = dict((n.name, n) for n in
@@ -122,10 +116,17 @@ def render_nodelist(self, template, context, node):
     def render_node(self, template, context, node):
         return node.render(context, forced=True)
 
-    def get_nodelist(self, node, original=None):
+    def get_nodelist(self, node, original):
         if isinstance(node, ExtendsNode):
             try:
-                return handle_extendsnode(node, block_context=None, original=original)
+                context = Context()
+                context.template = original
+                # TODO: We are passing an empty context when finding base
+                # templates. This does not work when extending using
+                # variables ({% extends template_var %}).
+                # A refactor might be needed to support that use-case with
+                # multiple offline contexts.
+                return handle_extendsnode(node, context)
             except template.TemplateSyntaxError as e:
                 raise TemplateSyntaxError(str(e))
             except template.TemplateDoesNotExist as e:
@@ -144,7 +145,8 @@ def walk_nodes(self, node, original=None):
         if original is None:
             original = node
         for node in self.get_nodelist(node, original):
-            if isinstance(node, CompressorNode) and node.is_offline_compression_enabled(forced=True):
+            if isinstance(node, CompressorNode) \
+                    and node.is_offline_compression_enabled(forced=True):
                 yield node
             else:
                 for node in self.walk_nodes(node, original):


Test patch:
diff --git a/compressor/tests/test_offline.py b/compressor/tests/test_offline.py
--- a/compressor/tests/test_offline.py
+++ b/compressor/tests/test_offline.py
@@ -1,5 +1,6 @@
 from __future__ import with_statement, unicode_literals
 import copy
+import django
 import io
 import os
 import sys
@@ -7,11 +8,12 @@
 from importlib import import_module
 
 from mock import patch
-from unittest import SkipTest
+from unittest import SkipTest, skipIf
 
 from django.core.management.base import CommandError
 from django.template import Template, Context
 from django.test import TestCase
+from django.test.utils import override_settings
 from django.utils import six
 
 from compressor.cache import flush_offline_manifest, get_offline_manifest
@@ -629,3 +631,27 @@ def _test_offline(self, engine):
         rendered_template = self._render_template(engine)
         result = (result[0], result[2])
         self.assertEqual(rendered_template, ''.join(result) + '\n')
+
+
+@skipIf(django.VERSION < (1, 9), "Needs Django >= 1.9, recursive templates were fixed in Django 1.9")
+class OfflineCompressExtendsRecursionTestCase(OfflineTestCaseMixin, TestCase):
+    """
+        Test that templates extending templates with the same name
+        (e.g. admin/index.html) don't cause an infinite test_extends_recursion
+    """
+    templates_dir = 'test_extends_recursion'
+    engines = ('django',)
+
+    INSTALLED_APPS = [
+        'django.contrib.admin',
+        'django.contrib.auth',
+        'django.contrib.contenttypes',
+        'django.contrib.staticfiles',
+        'compressor',
+    ]
+
+    @override_settings(INSTALLED_APPS=INSTALLED_APPS)
+    def _test_offline(self, engine):
+        count, result = CompressCommand().compress(
+            log=self.log, verbosity=self.verbosity, engine=engine)
+        self.assertEqual(count, 1)
diff --git a/compressor/tests/test_templates/test_extends_recursion/admin/index.html b/compressor/tests/test_templates/test_extends_recursion/admin/index.html
new file mode 100644
--- /dev/null
+++ b/compressor/tests/test_templates/test_extends_recursion/admin/index.html
@@ -0,0 +1 @@
+{% extends "admin/index.html" %}
diff --git a/compressor/tests/test_templates/test_extends_recursion/test_compressor_offline.html b/compressor/tests/test_templates/test_extends_recursion/test_compressor_offline.html
new file mode 100644
--- /dev/null
+++ b/compressor/tests/test_templates/test_extends_recursion/test_compressor_offline.html
@@ -0,0 +1,5 @@
+{% load compress %}
+
+{% compress js%}
+    <script type="text/javascript">alert('test');</script>
+{% endcompress %}