Repository: django-compressor/django-compressor

Issue:
TemplateDoesNotExist is raised when using native Jinja2 support in Django 1.8
Django 1.8 has native Jinja2 template support built-in. Therefore, I am not using Jingo, Coffin or django-jinja. In using the native Jinja2 template support option, however, there seems to be some pretty significant underlying differences in the way that context data is handled and in how Django locates Django app templates when the native Jinja2 is enabled.

These are my Django settings for the templates:

```
...
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.jinja2.Jinja2',
        'DIRS': [
            str(DJANGO_APPS_ROOT_DIR.path('templates')),
        ],
        'APP_DIRS': True,
        'OPTIONS': {
            'environment': 'oiaccount.common.template.jinja2.conf.environment',
            'extensions': [
                'jinja2.ext.i18n',
                'jinja2.ext.autoescape',
                'compressor.contrib.jinja2ext.CompressorExtension'
            ]
        }
    },
]
...
```

Using these settings and attempting to use django-compressor will raise the following exception when attempting to render a template. For example, this is the exception raised when the parser encounters a `{% compress css %}` block:

`django.template.base.TemplateDoesNotExist: compressor/css_file.html`

**Workaround:**
To fix this issue, I had to modify the django_compressor package in two (2) ways. If both these changes are not done, a `TemplateDoesNotExist` exception will be raised.
1. When the `TEMPLATES[APP_DIRS]` Django setting is True, Django looks into each Django app installed for each app's necessary templates. However, when Jinja2 is enabled, it looks for a `jinja2` top-level directory in the app. Therefore, I had to add a copy of the django-compressor template set into `/compressor/jinja2/compressor`. See the picture below.
   ![screen shot 2015-07-20 at 1 24 13 pm](https://cloud.githubusercontent.com/assets/654607/8783619/b63e2b88-2ee2-11e5-8f59-ead6d07e4b09.png)
2. Unfortunately, probably due to the fact that `context_instance` argument is deprecated starting in Django 1.8 (https://docs.djangoproject.com/en/1.8/topics/http/shortcuts/#render-to-response), enabling Jinja2 seems to totally prevent the `compressor/css_file.html` from being found. To fix this, change:

```
return render_to_string(template_name, context_instance=final_context)
```

at https://github.com/django-compressor/django-compressor/blob/develop/compressor/base.py#L345
to

```
return render_to_string(template_name, context=final_context.flatten())
```

**Caveats:**
1. The additional copy needed of the template set seems pretty rough around the edges, but I have yet to figure out a better method.
2. I am not sure if this negatively impacts the situation where Django Templates (DTL) and not Jinja2 templates are used.

Any comments on my approaches?

I can make a pull request if the changes that I have made are ultimately acceptable.



Discussion:
:+1: I'm affected by this behaviour too.

@adamjcook or @La0 can you make a pull request with those changes, to see if that breaks anything ? Then we'll take care about adding tests. Thanks.

@diox Sure. I can try to get to this in the next couple of days. I was working with this quite a while ago so I will have to spend some time putting a pull request together.

Actually i made it work without any code modifications.
You need to have 2 templates rendering engines:
- the vanilla Django engine, for admin, and "legacy" apps, looking in apps for paths.
- the Jinja engine, for your custom apps.

It looks like this in my settings:

``` python
TEMPLATES = [
    # Django templates for apps
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
        ],
        'APP_DIRS': True,
        'OPTIONS' : {
            'context_processors': [
                'django.contrib.auth.context_processors.auth',
                'django.template.context_processors.i18n',
                'django.template.context_processors.media',
                'django.template.context_processors.static',
                'django.template.context_processors.tz',
            ],
        }
    },

    # Jinja for our templates
    {
        'BACKEND': 'django.template.backends.jinja2.Jinja2',
        'DIRS': [
            os.path.join(BASE_DIR, 'templates'),
        ],
        #'APP_DIRS': False,
        'OPTIONS': {
            'environment' : 'project.jinja.environment',
            'extensions' : [
                'jinja2.ext.i18n',
                'jinja2.ext.with_',
                'jinja2.ext.autoescape',
                'django_jinja.builtins.extensions.CsrfExtension',
                'django_jinja.builtins.extensions.UrlsExtension',
                'django_jinja.builtins.extensions.StaticFilesExtension',
                'django_jinja.builtins.extensions.DjangoFiltersExtension',
                'compressor.contrib.jinja2ext.CompressorExtension',
            ],
        },
    },
]

```

Maybe we could/should just document that then ?

@La0 Did you test this on Django 1.8 or Django 1.9?

Django 1.8.5

Ok. Quite possibly an aspect of this was fixed in subsequent Django releases. I _think_ I wrote this issue while working with native Jinja templates in Django 1.8.1 or 1.8.2 - I cannot remember since it was a couple of months ago and the project that I was working on has moved from django-compressor to something else.

Can either of you still reproduce with django 1.8+ and django-compressor 2.0 ?

@diox Since I first raised this issue, I have moved away from the project where I was using django-compressor on so I am not really in a position to help right away. I will let you know though if I get another opportunity to help if this issue is not closed before then.

@diox 

I am using `Django==1.8.8` and `django-compressor==2.0`

I am encountering this issue with the following TEMPLATES setting:

```
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.jinja2.Jinja2',
        'DIRS': [
            os.path.join(BASE_DIR, 'jinja2'),
        ],
        'APP_DIRS': True,
        'OPTIONS': {
            'environment': 'project.jinja2.environment',
            'extensions': [
                'compressor.contrib.jinja2ext.CompressorExtension',
            ],
        },
    },
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            os.path.join(BASE_DIR, 'templates'),
        ],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]
```

`python manage.py compress --engine jinja2` fails with:

> Could not find config for 'Jinja2' in settings.TEMPLATES. COMPRESS_JINJA2_GET_ENVIRONMENT() may need to be defined in settings

My environment file `project.jinja2.environment` is as follows:

```
from __future__ import absolute_import
from django.conf import settings
from django.contrib.staticfiles.finders import find
from django.contrib.staticfiles.storage import staticfiles_storage
from django.core.urlresolvers import reverse
from jinja2 import Environment
from jinja2.ext import with_

def environment(**options):
    options.setdefault("extensions", []).append(with_)
    env = Environment(**options)
    env.globals.update({
        'static': staticfiles_storage.url,
        'static_find': find,
        'url': reverse,
        'DEBUG': settings.DEBUG,
    })
    return env
```

So I was tinkering with this and if I set alias to "jinja2" instead of "Jinja2" then the env is returned:

```
$ python manage.py shell
>>> from django.template import engines
>>> e = engines["Jinja2"]
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/django/template/utils.py", line 93, in __getitem__
    "in settings.TEMPLATES".format(alias))
InvalidTemplateEngineError: Could not find config for 'Jinja2' in settings.TEMPLATES
>>> e = engines["jinja2"]
>>> e.env
<jinja2.environment.Environment object at 0x7f5245344a90>
>>> 
```

Is the capitalization in the `JINJA2_GET_ENVIRONMENT` function defined [here](https://github.com/django-compressor/django-compressor/blob/a9ab8a13d8dd7817352daf31d804ec77eefb31e4/compressor/conf.py#L76) a typo?

So here is a simple function to put in settings:

```
def COMPRESS_JINJA2_GET_ENVIRONMENT():
    from django.template import engines
    return engines["jinja2"].env
```

I don't have time to investigate but yes, that looks like a typo. I wonder how test pass... Could you submit a pull request to fix it ? Thanks.

@diox Since the tests are passing I am not sure how to go about the change.  Do you think anyone is relying on the "Jinja2" key working or should it simply replace "Jinja2" with "jinja2".  Or we could try both keys?  Or maybe iterate the engines and compare the keys lower cased against "jinja2" and then use the first matching key?

I'd need to investigate :) I don't know where the engine name `django.template.loader._engine_list()` returns come from. Submit the pull request to just do the replacement, and I'll have a look later.


Resolving patch:
diff --git a/compressor/conf.py b/compressor/conf.py
--- a/compressor/conf.py
+++ b/compressor/conf.py
@@ -73,13 +73,10 @@ class CompressorConf(AppConf):
 
     # Returns the Jinja2 environment to use in offline compression.
     def JINJA2_GET_ENVIRONMENT():
-        alias = 'Jinja2'
+        alias = 'jinja2'
         try:
-            from django.template.loader import _engine_list
-            engines = _engine_list(alias)
-            if engines:
-                engine = engines[0]
-                return engine.env
+            from django.template import engines
+            return engines[alias].env
         except InvalidTemplateEngineError:
             raise InvalidTemplateEngineError(
                 "Could not find config for '{}' "


Test patch: