richleland/django-cumulus — pull request #91 by michelts (2013-06-13 17:12:45+00:00)
Title: Filelike behavior

The `read` method's chunk_size parameter is now optional and, if blank or negative, returns the whole file.

There is a new method `chunks` that returns the storage_object get iterator, see the [explanation in the Pyrax docs](https://github.com/rackspace/pyrax/blob/master/docs/cloud_files.md#retrieving-downloading-stored-objects) and in [Djangp docs](https://docs.djangoproject.com/en/dev/ref/files/file/#django.core.files.File.chunks).

This fixes a hidden problem: if I iterate over the file (line by line), I would get only 64kb of data (the django's default chunk_size for the chunks method).

Now, chunks returns an iterator with chunk_size (defaults to 64kb) and the whole file would be got in a snippet like bellow:

```python
file = self.thing.document.file
for line in file:
print line
```

#### Unable to call read twice

When a chunk_size is defined, Pyrax get method returns a generator and it must be spent until subsequent calls. A new get call would get the content from the beginning (see the above link). I cover it in the tests.

I didn't find a way to fix it. Even if I cache the iterator, I can't change his chunk_size and then, I can't do:

```python
file = self.thing.document.file
file.read(4)
file.read(3)
```

**Note that this isn't possible even with the current behavior of SwiftClientStorageFile.**

What should I do about it? May I cache the iterator and avoid this behavior raising value error when read is call with different chunk_sizes?