jbalogh/django-cache-machine — Issue #19: Models are missing dynamic attributes (e.g. django-thumbs) if fetched from cache
Author: turingmachine
Created: 2012-02-24 15:52:41+00:00
State: open

This only happens under certain circumstances:

* only on collections (i.e. filter) not single objects
* only if the collection is fetched via an iterator, i.e. for entry in queryset:
* it never fails if the query is evaluated at once (i.e list(queryset)
* the collection must have at least 100 objects for this error to occur

I'm using the fork of theatlantic with redis, but I've verified that this problem also exists in jbalogh/master with memcached.

Given the following model which has a ImageWithThumbsField (provided by django-thumbs)

```python
class Debug(caching.base.CachingMixin, models.Model):
text = models.CharField(max_length=512, blank=True, null=True)
img = ImageWithThumbsField(
verbose_name='image',
upload_to='uploads/projects/',
styles=({'w': 100, 'h': 100},),
thumb_method=generate_thumb_square_top,
blank=True, null=True
)
objects = caching.base.CachingManager()

```

and the following debug management command:

```python
from django.core.management.base import NoArgsCommand
class Command(NoArgsCommand):
def __init__(self):
pass

def handle_noargs(self, **options):
from django.core.cache import cache
from atizo.apps.debug.models import Debug

def _add_objects(count):
print "add %d" % count
n = 0
while n < count:
Debug().save()
n += 1

def _fetch_via_list():
print "via list()"
cache.clear()
# populate cache
query = list(Debug.objects.all())
for entry in query: img = entry.img.url_100x100
# fetch from cache
query = list(Debug.objects.all())
for entry in query: img = entry.img.url_100x100

def _fetch_via_iterator():
print "via iterator"
cache.clear()
# populate cache
query = Debug.objects.all()
for entry in query: img = entry.img.url_100x100
# fetch from cache
query = Debug.objects.all()
try:
for entry in query: img = entry.img.url_100x100 # <- will fail if there are more than 100 obj's
except:
print "fail!"

Debug.objects.all().delete()
cache.clear()

_add_objects(1)
_fetch_via_list() # success
_fetch_via_iterator() # success

_add_objects(98) # total 99
_fetch_via_list() # success
_fetch_via_iterator() # success

_add_objects(1) # total 100
_fetch_via_list() # success
_fetch_via_iterator() # fail

_add_objects(270) # total 370
_fetch_via_list() # success
```

django.db.model.query has a constant `CHUNK_SIZE = 100`. If changed to 200, the problem will only appear with collections of 200 objects or bigger.

Also there is a difference in the pickled cache entry:

cache entry that was populated with `query = list(Debug.objects.all())`:
http://pastie.org/3446872

cache entry that was populated with `for entry in query: img = entry.img.url_100x100`:
http://pastie.org/3446860