internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: django-cms/django-filer

Issue:
omit unicode character in filename
When I upload a file using django-filer and cmsplugin_filer, unicode character in file name is omitted.For instance, my original file name with Korean, '한글과English.docx', becomes 'English.docx' only. I didn't get any error message and my encoding settings are  all uf8 (db(postgresql), LC_ALL, LANG, nginx, django). 

It becomes huge problem when I used unicode character only especially mobile. I can't download file with extension only in mobile devices.

I tried to figure out what's the problem, but I can't find the reason. Could you test and find solution please?



Discussion:
The reason that unicode characters are removed, is because many filesystems don't support them.
We "downcast" things like üäö to uao. But of course this does not work for Korean.
I'm not sure what a clean solution would be here, since we can't allow unicode filenames by default. Maybe an optional setting?

By using something like https://github.com/miurahr/unihandecode?


Resolving patch:
diff --git a/filer/utils/files.py b/filer/utils/files.py
--- a/filer/utils/files.py
+++ b/filer/utils/files.py
@@ -1,12 +1,12 @@
 #-*- coding: utf-8 -*-
 from __future__ import unicode_literals
 
-import os
+import os, sys
 from django.utils.text import get_valid_filename as get_valid_filename_django
-from django.template.defaultfilters import slugify
+from django.template.defaultfilters import slugify as slugify_django
 from django.http.multipartparser import ChunkIter, exhaust, \
     StopFutureHandlers, SkipFile, StopUpload
-
+from unidecode import unidecode
 
 class UploadException(Exception):
     pass
@@ -100,6 +100,13 @@ def handle_upload(request):
     return upload, filename, is_raw
 
 
+if sys.version_info < (3, ):
+    def slugify(string):
+        return slugify_django(unidecode(unicode(string)))
+else:
+    def slugify(string):
+        return slugify_django(unidecode(string))
+
 def get_valid_filename(s):
     """
     like the regular get_valid_filename, but also slugifies away
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -29,6 +29,7 @@ def read(fname):
         'easy-thumbnails>=1.0',
         'django-mptt==0.5.2,==0.6,==0.6.1',
         'django_polymorphic>=0.2',
+        'Unidecode>=0.04',
     ),
     include_package_data=True,
     zip_safe=False,


Test patch:
Repository: django-cms/django-filer

Issue:
Missing zoom argument in thumbnail_processors.scale_and_crop_with_subject_location
In my humble point of view, the argument `zoom` has to be passed through to the underlying `processors.scale_and_crop` function:
https://github.com/stefanfoulis/django-filer/blob/develop/filer/thumbnail_processors.py#L48

otherwise zooming into thumbnails does not work.



Discussion:

Resolving patch:
diff --git a/filer/thumbnail_processors.py b/filer/thumbnail_processors.py
--- a/filer/thumbnail_processors.py
+++ b/filer/thumbnail_processors.py
@@ -45,7 +45,7 @@ def scale_and_crop_with_subject_location(im, size, subject_location=False, zoom=
     subject_location = normalize_subject_location(subject_location)
     if not (subject_location and crop):
         # use the normal scale_and_crop
-        return processors.scale_and_crop(im, size, crop=crop,
+        return processors.scale_and_crop(im, size, zoom=zoom, crop=crop,
                                          upscale=upscale, **kwargs)
 
     # for here on we have a subject_location and cropping is on


Test patch:
Repository: django-cms/django-filer

Issue:
[Django>=1.7] FilerFolderField admin widget hidden
If you use Django 1.7 and have a FilerFolderField in a model, the corresponding admin widget will not show up.

That’s happening since django/django@dc3d2ac98c1bcfad74d3e9523caf07e7e9fb15aa. In `fieldset.html`, a `hidden` class is added to a form row if all the contained widgets have `is_hidden = True`.

Setting `is_hidden` to `False` works fine. I’m not sure whether this attribute is useful somewhere else. I found it used 7 times in Django, and always refer to showing the field and its label. Since it’s not really a hidden field, I suggest we do that change.



Discussion:

Resolving patch:
diff --git a/filer/fields/folder.py b/filer/fields/folder.py
--- a/filer/fields/folder.py
+++ b/filer/fields/folder.py
@@ -18,7 +18,6 @@
 class AdminFolderWidget(ForeignKeyRawIdWidget):
     choices = None
     input_type = 'hidden'
-    is_hidden = True
 
     def render(self, name, value, attrs=None):
         obj = self.obj_for_value(value)


Test patch:
Repository: django-cms/django-filer

Issue:
CircularDependencyError when using custom Image model
093d07357ee13d4ea830db136ef037180824ddae added a migration dependency to the swappable `Image` model.

But a custom `Image` model inherits from `File`, so `filer.0001_initial` needs to be applied before the custom `Image` model. It of course leads to a `CircularDependencyError`.

The solution is to remove that dependency. No django-filer model depends on `Image`, so it can be removed safely.



Discussion:

Resolving patch:
diff --git a/filer/migrations_django/0001_initial.py b/filer/migrations_django/0001_initial.py
--- a/filer/migrations_django/0001_initial.py
+++ b/filer/migrations_django/0001_initial.py
@@ -13,7 +13,6 @@ class Migration(migrations.Migration):
     dependencies = [
         ('auth', '0001_initial'),
         migrations.swappable_dependency(settings.AUTH_USER_MODEL),
-        migrations.swappable_dependency(FILER_IMAGE_MODEL or 'filer.models.imagemodels.Image'),
         ('contenttypes', '0001_initial'),
     ]
 


Test patch:
Repository: django-cms/django-filer

Issue:
Need to add migrations to go with Django 1.10 changes



Discussion:
So this is going to vary on the Django version, not sure there is a clean way to do it.


Resolving patch:
diff --git a/filer/migrations/0007_auto_20161016_1055.py b/filer/migrations/0007_auto_20161016_1055.py
new file mode 100644
--- /dev/null
+++ b/filer/migrations/0007_auto_20161016_1055.py
@@ -0,0 +1,25 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.10.2 on 2016-10-16 08:55
+from __future__ import unicode_literals
+
+from django.db import migrations
+from filer.utils.compatibility import GTE_DJANGO_1_10
+
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('filer', '0006_auto_20160623_1627'),
+    ]
+
+    if GTE_DJANGO_1_10:
+        # This migrations is required in Django 1.10, but will error on Django 1.9 and below
+        # As the migration below does not alter the database schema it's safe to skip it
+        # on 1.9 and below
+        operations = [
+            migrations.AlterModelOptions(
+                name='image',
+                options={'default_manager_name': 'objects', 'verbose_name': 'image', 'verbose_name_plural': 'images'},
+            ),
+        ]


Test patch:
Repository: django-cms/django-filer

Issue:
Fixed canonical urls
`%s` is not supported format string for `strftime`, it is not portable, it may silently produce a wrong result for an aware `datetime` object, it fails if an input is in UTC but local timezone is not UTC. The only reason it's working (on unix-like systems) is because Python is passing the information to system's `strftime`.

Fixes #896 



Discussion:

Resolving patch:
diff --git a/filer/models/filemodels.py b/filer/models/filemodels.py
--- a/filer/models/filemodels.py
+++ b/filer/models/filemodels.py
@@ -4,11 +4,13 @@
 
 import hashlib
 import os
+from datetime import datetime
 
 from django.conf import settings
 from django.core import urlresolvers
 from django.core.files.base import ContentFile
 from django.db import models
+from django.utils import timezone
 from django.utils.translation import ugettext_lazy as _
 
 from . import mixins
@@ -256,13 +258,17 @@ def url(self):
             r = ''
         return r
 
+    @property
+    def canonical_time(self):
+        return int((self.uploaded_at - datetime(1970, 1, 1, tzinfo=timezone.utc)).total_seconds())
+
     @property
     def canonical_url(self):
         url = ''
         if self.file and self.is_public:
             try:
                 url = urlresolvers.reverse('canonical', kwargs={
-                    'uploaded_at': self.uploaded_at.strftime('%s'),
+                    'uploaded_at': self.canonical_time,
                     'file_id': self.id
                 })
             except urlresolvers.NoReverseMatch:
diff --git a/filer/views.py b/filer/views.py
--- a/filer/views.py
+++ b/filer/views.py
@@ -12,7 +12,6 @@ def canonical(request, uploaded_at, file_id):
     Redirect to the current url of a public file
     """
     filer_file = get_object_or_404(File, pk=file_id, is_public=True)
-    if (uploaded_at != filer_file.uploaded_at.strftime('%s') or
-            not filer_file.file):
+    if (not filer_file.file or int(uploaded_at) != filer_file.canonical_time):
         raise Http404('No %s matches the given query.' % File._meta.object_name)
     return redirect(filer_file.url)


Test patch:
Repository: django-cms/django-filer

Issue:
Django 1.11 support
When will be next release with Django 1.11.x support?


Discussion:
I am also interested in this, has anyone tried running the tests in 1.11?

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -26,10 +26,10 @@ def read(fname):
     author_email='stefan@foulis.ch',
     packages=find_packages(),
     install_requires=(
-        'Django>=1.8,<1.10.999',  # Django is known to use rc versions
-        'easy-thumbnails>=2,<2.4',
+        'Django>=1.8,<1.11.999',  # Django is known to use rc versions
+        'easy-thumbnails>=2,<2.5',
         'django-mptt>=0.6,<0.9',  # the exact version depends on Django
-        'django_polymorphic>=0.7,<1.1',
+        'django_polymorphic>=0.7,<1.4',
         'Unidecode>=0.04,<0.05',
     ),
     include_package_data=True,
@@ -44,11 +44,10 @@ def read(fname):
         'Programming Language :: Python',
         'Topic :: Internet :: WWW/HTTP',
         'Framework :: Django',
-        'Framework :: Django :: 1.5',
-        'Framework :: Django :: 1.6',
-        'Framework :: Django :: 1.7',
         'Framework :: Django :: 1.8',
         'Framework :: Django :: 1.9',
+        'Framework :: Django :: 1.10',
+        'Framework :: Django :: 1.11',
         'Programming Language :: Python :: 2.6',
         'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3.4',


Test patch:
diff --git a/test_requirements/django-1.11.txt b/test_requirements/django-1.11.txt
--- a/test_requirements/django-1.11.txt
+++ b/test_requirements/django-1.11.txt
@@ -1,4 +1,4 @@
-django>=1.10,<1.11
+django>=1.11,<2.0
 django-mptt>=0.8,<0.9
 djangocms-helper>=0.9.2,<0.10
 coverage
Repository: django-cms/django-filer

Issue:
Canonical URLs computation changed between 1.26 and 1.27
Hi,

Commit 878cafe82a5d2482ca4b571f6bca8321b55261db changed the behavior of canonical URLs.

`self.uploaded_at.strftime('%s')` does not equals `int((self.uploaded_at - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds())`. Thus, former permalinks to filer images of files are broken.

The fix for me was to change `int((self.uploaded_at - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds())` to `int((self.uploaded_at - datetime(1970, 1, 1, 1, tzinfo=pytz.utc)).total_seconds())` to get the old behavior.

It is quite a serious problem for me as I have thousands of links broken.

What do you think?

Thanks



Discussion:

Resolving patch:
diff --git a/filer/models/filemodels.py b/filer/models/filemodels.py
--- a/filer/models/filemodels.py
+++ b/filer/models/filemodels.py
@@ -281,9 +281,9 @@ def url(self):
     @property
     def canonical_time(self):
         if settings.USE_TZ:
-            return int((self.uploaded_at - datetime(1970, 1, 1, tzinfo=timezone.utc)).total_seconds())
+            return int((self.uploaded_at - datetime(1970, 1, 1, 1, tzinfo=timezone.utc)).total_seconds())
         else:
-            return int((self.uploaded_at - datetime(1970, 1, 1)).total_seconds())
+            return int((self.uploaded_at - datetime(1970, 1, 1, 1)).total_seconds())
 
     @property
     def canonical_url(self):


Test patch:
Repository: django-cms/django-filer

Issue:
Out of memory error fix.
As described by @leonsmith in issue #973, 10000 value is causing timeout and eventually killing process.


Discussion:

Resolving patch:
diff --git a/filer/models/abstract.py b/filer/models/abstract.py
--- a/filer/models/abstract.py
+++ b/filer/models/abstract.py
@@ -22,7 +22,7 @@ class BaseImage(File):
     DEFAULT_THUMBNAILS = {
         'admin_clipboard_icon': {'size': (32, 32), 'crop': True,
                                  'upscale': True},
-        'admin_sidebar_preview': {'size': (SIDEBAR_IMAGE_WIDTH, 10000), 'upscale': True},
+        'admin_sidebar_preview': {'size': (SIDEBAR_IMAGE_WIDTH, 0), 'upscale': True},
         'admin_directory_listing_icon': {'size': (48, 48),
                                          'crop': True, 'upscale': True},
         'admin_tiny_icon': {'size': (32, 32), 'crop': True, 'upscale': True},


Test patch:
Repository: django-cms/django-filer

Issue:
hide "Save as New" button
A click on "Save as New" will raise into PermissionDenied, because filer.admin.permissions.PrimitivePermissionAwareModelAdmin.has_add_permission() will always return False, see: https://github.com/divio/django-filer/blob/develop/filer/admin/permissions.py#L9

So, the "Save as New" button should not be exists...


Discussion:
Confirmed

Resolving patch:
diff --git a/filer/admin/fileadmin.py b/filer/admin/fileadmin.py
--- a/filer/admin/fileadmin.py
+++ b/filer/admin/fileadmin.py
@@ -26,12 +26,6 @@ class FileAdmin(PrimitivePermissionAwareModelAdmin):
     raw_id_fields = ('owner',)
     readonly_fields = ('sha1', 'display_canonical')
 
-    # save_as hack, because without save_as it is impossible to hide the
-    # save_and_add_another if save_as is False. To show only save_and_continue
-    # and save in the submit row we need save_as=True and in
-    # render_change_form() override add and change to False.
-    save_as = True
-
     form = FileAdminChangeFrom
 
     @classmethod


Test patch:
Repository: django-cms/django-filer

Issue:
django-mptt<0.9
Is there any good reason to limit django-mptt to `<0.9`?


Discussion:
`django-mptt 0.9` supports Django 1.11 and 2.0 but removed tests of supporting 1.9, 1.10.
@czpython, check tests and move the django-mptt limit to `<0.10` if everything is ok and release new minor version. Everybody wants to upgrade their django to 1.11 at least ;)

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -28,7 +28,7 @@ def read(fname):
     install_requires=(
         'Django>=1.8,<1.11.999',  # Django is known to use rc versions
         'easy-thumbnails>=2,<3.0',
-        'django-mptt>=0.6,<0.9',  # the exact version depends on Django
+        'django-mptt>=0.6,<0.10',  # the exact version depends on Django
         'django_polymorphic>=0.7,<2.1',
         'Unidecode>=0.04,<0.05',
     ),


Test patch:
diff --git a/test_requirements/django-1.11.txt b/test_requirements/django-1.11.txt
--- a/test_requirements/django-1.11.txt
+++ b/test_requirements/django-1.11.txt
@@ -1,6 +1,6 @@
 django>=1.11,<2.0
-django-mptt>=0.8,<0.9
-djangocms-helper>=0.9.2,<0.10
+django-mptt>=0.8,<0.10
+djangocms-helper>=1.0,<1.2
 django_polymorphic>=2.0,<2.1
 coverage
Repository: django-cms/django-filer

Issue:
Uncaught ReferenceError: django is not defined
When I try to create a new model with the content creation wizard that has a FilerImageField, I get an `Uncaught ReferenceError: django is not defined` at:

    VM1967 dropzone.init.js:11
    VM1969 popup_handling.js:46
    VM1970 widget.js:4
    (index):155

Why do I get this and how can I avoid it?

### Reproducible with: 

#### modify `aldryn_newsblog/cms_wizards.py`:


    class CreateNewsBlogArticleForm(BaseFormMixin, TranslatableModelForm):
        # ...
        class Meta:
            model = Article
            fields = ['title', 'app_config', 'featured_image']
        # ...

#### or: create a test `foo` app:

My `models.py` file:

    from django.db import models
    from django.core.urlresolvers import reverse
    from filer.fields.image import FilerImageField

    class Foo(models.Model):
        slug = models.SlugField(unique=True)
        image = FilerImageField(related_name='foo_picture', null=True)

        def get_absolute_url(self):
            return reverse('foo', kwargs={'slug': self.slug})

My `forms.py` file:

    from django import forms
    from .models import Foo

    class FooWizardForm(forms.ModelForm):
        class Meta:
            model = Foo
            exclude = []

My `cms_wizards.py` file:

    from cms.wizards.wizard_base import Wizard
    from cms.wizards.wizard_pool import wizard_pool
    from .forms import FooWizardForm

    class FooWizard(Wizard):
        pass

    foo_wizard = FooWizard(
        title="Foo",
        weight=200,
        form=FooWizardForm,
        description="Create a new Foo"
    )

    wizard_pool.register(foo_wizard)

My `views.py` file:

    from django.views.generic import DetailView
    from .models import Foo

    class FooView(DetailView):
        model = Foo

My `urls.py` file:

    from foo.views import FooView
    from django.conf import settings
    from django.contrib import admin
    from django.conf.urls import url, include
    from django.conf.urls.static import static

    urlpatterns = [
        url(r'^admin/', admin.site.urls),
        url(r'^foo/(?P<slug>[-\w]+)/$', FooView.as_view(), name='foo'),
        url(r'^', include('cms.urls'))
    ] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)


Discussion:

Resolving patch:
diff --git a/filer/fields/file.py b/filer/fields/file.py
--- a/filer/fields/file.py
+++ b/filer/fields/file.py
@@ -87,6 +87,8 @@ class Media(object):
             ]
         }
         js = (
+            'admin/js/vendor/jquery/jquery.js',
+            'admin/js/jquery.init.js',
             'filer/js/libs/dropzone.min.js',
             'filer/js/addons/dropzone.init.js',
             'filer/js/addons/popup_handling.js',


Test patch:
Repository: django-cms/django-filer

Issue:
Missing object for Django 2.0 and up for obj_for_value method
Filer 1.4.1 uses a deprecated `to` property which was deprecated in Django 1.9 and removed in Django 2.0. Mentioned property is using in obj_for_value method present in AdminFileWidget and AdminFolderWidget:
```
    def obj_for_value(self, value):
        try:
            key = self.rel.get_related_field().name
            obj = self.rel.to._default_manager.get(**{key: value})
        except:
            obj = None
        return obj
```
I prepared a fix which uses just model instead of using the deprecated property for Django 1.9+. 

Here is my PR: #1117 


Discussion:

Resolving patch:
diff --git a/filer/fields/file.py b/filer/fields/file.py
--- a/filer/fields/file.py
+++ b/filer/fields/file.py
@@ -7,6 +7,7 @@
 from django import forms
 from django.contrib.admin.sites import site
 from django.contrib.admin.widgets import ForeignKeyRawIdWidget
+from django.core.exceptions import ObjectDoesNotExist
 from django.db import models
 from django.template.loader import render_to_string
 from django.utils.http import urlencode
@@ -75,8 +76,11 @@ def label_for_value(self, value):
     def obj_for_value(self, value):
         try:
             key = self.rel.get_related_field().name
-            obj = self.rel.to._default_manager.get(**{key: value})
-        except:
+            if LTE_DJANGO_1_8:
+                obj = self.rel.to._default_manager.get(**{key: value})
+            else:
+                obj = self.rel.model._default_manager.get(**{key: value})
+        except ObjectDoesNotExist:
             obj = None
         return obj
 
diff --git a/filer/fields/folder.py b/filer/fields/folder.py
--- a/filer/fields/folder.py
+++ b/filer/fields/folder.py
@@ -6,13 +6,14 @@
 from django import forms
 from django.contrib.admin.sites import site
 from django.contrib.admin.widgets import ForeignKeyRawIdWidget
+from django.core.exceptions import ObjectDoesNotExist
 from django.db import models
 from django.template.loader import render_to_string
 from django.utils.http import urlencode
 from django.utils.safestring import mark_safe
 
 from ..models import Folder
-from ..utils.compatibility import reverse, truncate_words
+from ..utils.compatibility import LTE_DJANGO_1_8, reverse, truncate_words
 from ..utils.model_label import get_model_label
 
 
@@ -73,8 +74,11 @@ def label_for_value(self, value):
     def obj_for_value(self, value):
         try:
             key = self.rel.get_related_field().name
-            obj = self.rel.to._default_manager.get(**{key: value})
-        except:
+            if LTE_DJANGO_1_8:
+                obj = self.rel.to._default_manager.get(**{key: value})
+            else:
+                obj = self.rel.model._default_manager.get(**{key: value})
+        except ObjectDoesNotExist:
             obj = None
         return obj
 


Test patch:
Repository: django-cms/django-filer

Issue:
Django 2.1 render() got an unexpected keyword argument 'renderer'
I'm getting an error:
`render() got an unexpected keyword argument 'renderer'`

Django from version 2.1 requires `renderer` argument for render method:

> Support for Widget.render() methods without the renderer argument is removed.

`AdminFolderWidget` and `AdminFileWidget` classes are missing `renderer` argument - so it should be added to support Django 2.1 and up.

Here is my PR to fix that: #...

Filer: 1.4.1
Django: 2.1


Discussion:

Resolving patch:
diff --git a/filer/fields/file.py b/filer/fields/file.py
--- a/filer/fields/file.py
+++ b/filer/fields/file.py
@@ -23,7 +23,7 @@
 class AdminFileWidget(ForeignKeyRawIdWidget):
     choices = None
 
-    def render(self, name, value, attrs=None):
+    def render(self, name, value, attrs=None, render=None):
         obj = self.obj_for_value(value)
         css_id = attrs.get('id', 'id_image_x')
         related_url = None
diff --git a/filer/fields/folder.py b/filer/fields/folder.py
--- a/filer/fields/folder.py
+++ b/filer/fields/folder.py
@@ -21,7 +21,7 @@ class AdminFolderWidget(ForeignKeyRawIdWidget):
     input_type = 'hidden'
     is_hidden = False
 
-    def render(self, name, value, attrs=None):
+    def render(self, name, value, attrs=None, renderer=None):
         obj = self.obj_for_value(value)
         css_id = attrs.get('id')
         css_id_folder = "%s_folder" % css_id


Test patch: