sebleier/django-redis-cache — Issue #32: Models are missing dynamic attributes (e.g. django-thumbs) if fetched from cache
Author: turingmachine
Created: 2012-02-24 15:41:32+00:00
State: open

This only happens under certain circumstances:

* only on collections (i.e. filter) not single objects
* only if the collection is fetched via an iterator, i.e. for entry in queryset:
* it never fails if the query is evaluated at once (i.e list(queryset)
* the collection must have at least 100 objects for this error to occur

Given the following model which has a ImageWithThumbsField (provided by django-thumbs)

```python
class Debug(caching.base.CachingMixin, models.Model):
text = models.CharField(max_length=512, blank=True, null=True)
img = ImageWithThumbsField(
verbose_name='image',
upload_to='uploads/projects/',
styles=({'w': 100, 'h': 100},),
thumb_method=generate_thumb_square_top,
blank=True, null=True
)
objects = caching.base.CachingManager()

```

and the following debug management command:

```python
from django.core.management.base import NoArgsCommand
class Command(NoArgsCommand):
def __init__(self):
pass

def handle_noargs(self, **options):
from django.core.cache import cache
from atizo.apps.debug.models import Debug

def _add_objects(count):
print "add %d" % count
n = 0
while n < count:
Debug().save()
n += 1

def _fetch_via_list():
print "via list()"
cache.clear()
# populate cache
query = list(Debug.objects.all())
for entry in query: img = entry.img.url_100x100
# fetch from cache
query = list(Debug.objects.all())
for entry in query: img = entry.img.url_100x100

def _fetch_via_iterator():
print "via iterator"
cache.clear()
# populate cache
query = Debug.objects.all()
for entry in query: img = entry.img.url_100x100
# fetch from cache
query = Debug.objects.all()
try:
for entry in query: img = entry.img.url_100x100 # <- will fail if there are more than 100 obj's
except:
print "fail!"

Debug.objects.all().delete()
cache.clear()

_add_objects(1)
_fetch_via_list() # success
_fetch_via_iterator() # success

_add_objects(98) # total 99
_fetch_via_list() # success
_fetch_via_iterator() # success

_add_objects(1) # total 100
_fetch_via_list() # success
_fetch_via_iterator() # fail

_add_objects(270) # total 270
_fetch_via_list() # success
```

django.db.model.query has a constant `CHUNK_SIZE = 100`. If changed to 200, the problem will only appear with collections of 200 objects or bigger.

Also there is a difference in the pickled cache entry:

cache entry that was populated with `query = list(Debug.objects.all())`:
http://pastie.org/3446872

cache entry that was populated with `for entry in query: img = entry.img.url_100x100`:
http://pastie.org/3446860