Repository: matthewwithanm/django-imagekit

Issue #391: I/O operation on closed file
Created: 2016-10-25T14:44:33Z
Getting this issue when creating a new image.  I simply have a few specs on an ImageField like this:
    big_image = ImageSpecField(
        [SmartResize(width=900, height=600)],
        source='original_image',
        format='JPEG',
        options={'quality': 75}
    )

Exception:
ValueError` at /api/v1/pictures/pictures/

I/O operation on closed file.
Request Method: POST
Request URL:    https://foobar.com/api/v1/pictures/pictures/
Django Version: 1.10.2
Exception Type: ValueError
Exception Value:    
I/O operation on closed file.
Exception Location: /home/foobar/.virtualenvs/foobar/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py in _generate, line 102

Later part of the stack trace, somehow the "File" gets set to none?:
/home/foobar/.virtualenvs/foobar/local/lib/python2.7/site-packages/imagekit/cachefiles/backends.py in generate_now
            file._generate()
Local Vars

Variable    Value
file    
<ImageCacheFile: CACHE/images/pictures/2016/10/25/foobar.jpg>
force   
False
self    
<imagekit.cachefiles.backends.Simple object at 0x7f9e2e91ce50>
/home/foobar/.virtualenvs/foobar/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py in _generate
        content.seek(0)
Local Vars

Variable    Value
actual_name 
u'CACHE/images/pictures/2016/10/25/foobar.jpg'
content 
<File: None>
self    
<ImageCacheFile: CACHE/images/pictures/2016/10/25/foobar.jpg>

I have a setup with S3 and this started happening when I updated the boto library to boto3.
Reverting back seemed to help. I can provide more info if needed.

Duplicate issue #465: Programmatically `.save()` makes admin page broken
Created: 2018-05-09T13:56:25Z
Simplified traceback is
KeyError: 'small_thumbnail'  During handling of the above exception, another exception occurred: 
ValueError: I/O operation on closed file.
Internal Server Error: /admin/vouchers/voucher/
Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/django/db/models/options.py", line 566, in get_field
        return self.fields_map[field_name] KeyError: 'small_thumbnail'  During handling of the above exception, another exception occurred:  
        Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/utils.py", line 276, in lookup_field     
        f = _get_non_gfk_field(opts, name)
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/utils.py", line 307, in _get_non_gfk_field     
        field = opts.get_field(name)
    File "/usr/local/lib/python3.6/site-packages/django/db/models/options.py", line 568, in get_field     
        raise FieldDoesNotExist("%s has no field named '%s'" % (self.object_name, field_name)) django.core.exceptions.FieldDoesNotExist: 
            Voucher has no field named 'small_thumbnail'  During handling of the above exception, another exception occurred:  
        Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 35, in inner     
        response = get_response(request)
    File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 158,  in _get_response     
        response = self.process_exception_by_middleware(e, request)
    File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 156, in _get_response     
        response = response.render()
    File "/usr/local/lib/python3.6/site-packages/django/template/response.py", line 106, in render     
            self.content = self.rendered_content
    File "/usr/local/lib/python3.6/site-packages/django/template/response.py", line 83, in rendered_content     
        content = template.render(context, self._request)
    File "/usr/local/lib/python3.6/site-packages/django/template/backends/django.py", line 61, in render     
        return self.template.render(context)
    File "/usr/local/lib/python3.6/site-packages/opbeat/instrumentation/packages/base.py", line 63, in __call__     
        args, kwargs)
    File "/usr/local/lib/python3.6/site-packages/opbeat/instrumentation/packages/base.py", line 222, in call_if_sampling     
        return self.call(module, method, wrapped, instance, args, kwargs)
    File "/usr/local/lib/python3.6/site-packages/opbeat/instrumentation/packages/django/template.py", line 18, in call     
        return wrapped(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 175, in render     
        return self._render(context)
    File "/usr/local/lib/python3.6/site-packages/django/test/utils.py", line 98, in instrumented_test_render     
        return self.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/loader_tags.py", line 155, in render     
        return compiled_parent._render(context)
    File "/usr/local/lib/python3.6/site-packages/django/test/utils.py", line 98, in instrumented_test_render     
        return self.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/loader_tags.py", line 155, in render     
        return compiled_parent._render(context)
    File "/usr/local/lib/python3.6/site-packages/django/test/utils.py", line 98, in instrumented_test_render     
        return self.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/loader_tags.py", line 155, in render     
        return compiled_parent._render(context)
    File "/usr/local/lib/python3.6/site-packages/django/test/utils.py", line 98, in instrumented_test_render     
        return self.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/loader_tags.py", line 67, in render     
        result = block.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/loader_tags.py", line 67, in render     
        result = block.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/library.py", line 214, in render     
        _dict = self.func(*resolved_args, **resolved_kwargs)
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/templatetags/admin_list.py", line 326, in result_list     
        'results': list(results(cl))}
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/templatetags/admin_list.py", line 302, in results     
        yield ResultList(None, items_for_result(cl, res, None))
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/templatetags/admin_list.py", line 293, in __init__     
        super().__init__(*items)
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/templatetags/admin_list.py", line 212, in items_for_result     
        f, attr, value = lookup_field(field_name, result, cl.model_admin)
    File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/utils.py", line 285, in lookup_field     
        value = attr(obj)
    File "/usr/local/lib/python3.6/site-packages/imagekit/admin.py", line 39, in __call__     
        'original_image': original_image,
    File "/usr/local/lib/python3.6/site-packages/django/template/loader.py", line 62, in render_to_string     
        return template.render(context, request)
    File "/usr/local/lib/python3.6/site-packages/django/template/backends/django.py", line 61, in render     
        return self.template.render(context)
    File "/usr/local/lib/python3.6/site-packages/opbeat/instrumentation/packages/base.py", line 63, in __call__     
        args, kwargs)
    File "/usr/local/lib/python3.6/site-packages/opbeat/instrumentation/packages/base.py", line 222, in call_if_sampling     
        return self.call(module, method, wrapped, instance, args, kwargs)
    File "/usr/local/lib/python3.6/site-packages/opbeat/instrumentation/packages/django/template.py", line 18, in call     
        return wrapped(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 175, in render     
        return self._render(context)
    File "/usr/local/lib/python3.6/site-packages/django/test/utils.py", line 98, in instrumented_test_render     
        return self.nodelist.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 943, in render     
        bit = node.render_annotated(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/base.py", line 910, in render_annotated     
        return self.render(context)
    File "/usr/local/lib/python3.6/site-packages/django/template/defaulttags.py", line 313, in render     
        if match:
    File "/usr/local/lib/python3.6/site-packages/imagekit/cachefiles/__init__.py", line 131, in __bool__     
        existence_required.send(sender=self, file=self)
    File "/usr/local/lib/python3.6/site-packages/django/dispatch/dispatcher.py", line 178, in send     
        for receiver in self._live_receivers(sender)
    File "/usr/local/lib/python3.6/site-packages/django/dispatch/dispatcher.py", line 178, in <
        listcomp>     for receiver in self._live_receivers(sender)
    File "/usr/local/lib/python3.6/site-packages/imagekit/registry.py", line 53, in existence_required_receiver     
        self._receive(file, 'on_existence_required')
    File "/usr/local/lib/python3.6/site-packages/imagekit/registry.py", line 61, in _receive     
        call_strategy_method(file, callback)
    File "/usr/local/lib/python3.6/site-packages/imagekit/utils.py", line 166, in call_strategy_method     
        fn(file)
    File "/usr/local/lib/python3.6/site-packages/imagekit/cachefiles/strategies.py", line 15, in on_existence_required     
        file.generate()
    File "/usr/local/lib/python3.6/site-packages/imagekit/cachefiles/__init__.py", line 94, in generate     
        self.cachefile_backend.generate(self, force)
    File "/usr/local/lib/python3.6/site-packages/imagekit/cachefiles/backends.py", line 109, in generate     
        self.generate_now(file, force=force)
    File "/usr/local/lib/python3.6/site-packages/imagekit/cachefiles/backends.py", line 96, in generate_now     
        file._generate()
    File "/usr/local/lib/python3.6/site-packages/imagekit/cachefiles/__init__.py", line 103, in _generate     
        content.seek(0) ValueError: I/O operation on closed file.
requirements.txt
django==2.0.5
django-imagekit==4.0.2

utils.py
def give_voucher(userprofile: UserProfile, profile_created: bool):
    """
    Give the voucher to mobile_phone user if profile_created
    :param mobile_phone:
    :param profile_created:
    :return:
    """
    voucher_record = get_vouchered_people(userprofile.mobile_phone)
    logger.info(f"profile_created: {profile_created} voucher_record: {voucher_record}")
    if profile_created and voucher_record is not None:
        # New user give voucher. For integrity reason I have to use get_or_create()
        # No worry. It does not create authenticated user to the system
        # I intentionally put my identity to not freak out people
        admin, created = User.objects.get_or_create(username='admin', email='sarit@elcolie.com')

        # Config require 3rd parties therefore I decided to put it here rather than `base.py`
        bkk = timezone('Asia/Bangkok')
        expiry_date = datetime(2018, 5, 31, 23, 59, tzinfo=bkk)
        expiry_utc = expiry_date.astimezone(pytz.utc)

        # Use name from the given records XLSX since platform unable to supply firstname&lastname
        description = f"Thank you {voucher_record.get('name')} for your continued support of Hunter Poke! As a token of appreciation Hunter would like to give you a free 500Baht voucher to continue your healthy & delicious way of eating :)"
        company = Company.objects.get(name__icontains='hunter poke')
        voucher = Voucher.objects.create(
            amount=Money('500', 'THB'),
            expiry_date=expiry_utc,
            company=company,
            customer=userprofile.user,
            title=f"Hunter Poke VIP Customer Voucher",
            description=description,
            created_by=admin,
            updated_by=admin,
        )
        filename = str(uuid.uuid4()) + '.jpg'
        with open(str(settings.BASE_DIR('apps/vouchers/S__10838070.jpg')), 'rb') as file:
            data = File(file)
            voucher.image.save(filename, data, True)
            voucher.save()

        voucher.branches.set([branch for branch in company.branches.all()])
        logging.info(f"{voucher} has been created")
models.py
class Voucher(AbstractThumbnail, AbstractTimestamp):
    image = models.ImageField(default='uc.png', upload_to='vouchers')
    thumbnail = models.ImageField(default='uc.png', upload_to='voucher_thumbnails')
    small_thumbnail = ImageSpecField(source='image',
                                     processors=[ResizeToFill(settings.THUMBNAIL['small']['size_x'],
                                                              settings.THUMBNAIL['small']['size_y'])],
                                     format=settings.THUMBNAIL_FORMAT,
                                     options={'quality': settings.THUMBNAIL_QUALITY})
    medium_thumbnail = ImageSpecField(source='image',
                                      processors=[ResizeToFill(settings.THUMBNAIL['medium']['size_x'],
                                                               settings.THUMBNAIL['medium']['size_y'])],
                                      format=settings.THUMBNAIL_FORMAT,
                                      options={'quality': settings.THUMBNAIL_QUALITY})
    large_thumbnail = ImageSpecField(source='image',
                                     processors=[ResizeToFill(settings.THUMBNAIL['large']['size_x'],
                                                              settings.THUMBNAIL['large']['size_y'])],
                                     format=settings.THUMBNAIL_FORMAT,
                                     options={'quality': settings.THUMBNAIL_QUALITY})
admin.py
from django.contrib import admin
from imagekit.admin import AdminThumbnail
from reversion.admin import VersionAdmin

from poinkbackend.apps.vouchers.models import Voucher


@admin.register(Voucher)
class VoucherAdmin(VersionAdmin):
    small_thumbnail = AdminThumbnail(image_field='small_thumbnail')
    medium_thumbnail = AdminThumbnail(image_field='medium_thumbnail')
    large_thumbnail = AdminThumbnail(image_field='large_thumbnail')
    list_display = [
        'id',
        'amount',
        'expiry_date',
        'company',
        'customer',
        'order',
        'image',
        'thumbnail',
        'small_thumbnail',
        'medium_thumbnail',
        'large_thumbnail',
        'title',
        'description',
        'created_at',
        'updated_at',
        'created_by',
        'updated_by',
    ]
    list_display_links = ['id', 'amount']
    search_fields = ['title', 'description']
I confirm file is there and I can save the record with image
When I use djangoAdmin page do save. The thumbnails are fine.
Problem:
When I use utils.py to create my record. Django admin page will call thumbnails and raises IOError file not found
Question:
Where am I wrong?

Closing comment (2018-05-14T07:48:34Z): Ok this is duplicate of #391