jdriscoll/django-photologue — pull request #90 by celiao (2014-06-05 22:30:39+00:00)
Title: Extend to Django Storage API and Amazon AWS S3

### Intro
The Photologue code in this Pull Request makes use of the Django Storage API and allows the use of Amazon AWS S3 for remote storage.

### Changed and New Files
The following files have changed or are new:
- `photologue/models.py` - converted Python file operations to Django Storage API where necessary; added `delete()` method to `Watermark` class, similar to that found in the `Photo` class.
- `photologue/tests/factories.py` - added `ImageModelFactory` as instructed in http://factoryboy.readthedocs.org/en/v2.3.1/orms.html?highlight=from_path#the-djangomodelfactory-subclass, since `ImageModel` is an abstract class.
- `photologue/tests/test_effect.py` - converted Python file operations to Django Storage API where necessary.
- `photologue/tests/test_photo.py` - converted Python file operations to Django Storage API where necessary; added two `self.pl4.delete()` calls for proper cleanup.
- `example_project/requirements.txt` - added `boto` and `django-storage` for S3 use.
- `example_project/example_project/s3utils.py` - defined S3 storage types and locations.
- `example_project/example_project/settings.py` - added logic to check for remote storage.
- `example_project/example_project/settings_s3boto.py` - get environment variables and set global variables specific to S3 configuration. Secret keys are stored in environment variables for security.

### Local vs. Remote Storage
By default, Photologue will use local storage.
https://docs.djangoproject.com/en/dev/topics/files/#file-storage
https://docs.djangoproject.com/en/dev/ref/files/storage/#module-django.core.files.storage

To configure Photologue for remote storage, `PHOTOLOGUE_REMOTE_STORAGE` needs to be set. See the next section on how to use Amazon AWS S3 for remote storage.

### Using Amazon AWS S3
Use the following steps to configure the Photologue `example_project` (or your project!) for remote storage on S3.
1. Signup for Amazon AWS S3 at http://aws.amazon.com/s3/.
2. Create a Bucket on S3 to store your media and static files.
3. Set the environment variables:
- `PHOTOLOGUE_REMOTE_STORAGE` - equal to `s3boto` for S3.
- `AWS_ACCESS_KEY_ID` - issued to your account by S3.
- `AWS_SECRET_ACCESS_KEY` - issued to your account by S3.
- `AWS_STORAGE_BUCKET_NAME` - name of your bucket on S3.
- `AWS_REGION` - (optional) set if using a S3 region other than `US Standard`.
4. Note that the value of `PHOTOLOGUE_REMOTE_STORAGE` controls whether local or remote storage is used. Unset `PHOTOLOGUE_REMOTE_STORAGE` to revert to local storage.
5. To copy your static files into your S3 Bucket, type `python manage.py collectstatic` in the `example_project` directory.
5. Give it a try!

### Testing
The Photologue unit tests were run and pass for both local and remote Amazon AWS S3 storage. The unit tests were run by typing `python manage.py test photologue` in the `example_project` directory.

### Comments
- The `makedirs()` and `removedirs()` calls were removed because the Django Storage API takes care of creating directories when necessary, and remote storage sites may use keys rather than directories.
- When the unit tests were run with S3, they were noticeably slower than with local storage. It would be useful to investigate how to speed up the code or configuration for S3 remote storage.
- This code can be easily extended to cover other remote storage sites.

### Other Observations
- The `EXIF/date_taken` code in `Photo.save()` was not changed, but has issues.
- The code that assigns `date_taken` should follow a `super...save()` call on the model, since per the Django documentation on ImageField and FileField: 'the actual file name used on disk cannot be relied on until after the model has been saved.' https://docs.djangoproject.com/en/dev/ref/models/fields/#filefield.
- Then, when `date_taken` is assigned, a `super...save()` should be called again, so it is stored to the database. In this case, `date_taken` will be assigned and saved properly, as proved by informal tests.
- However, for some reason, when the unit tests are performed with multiple calls to `super...save()` in one method, `IntegrityError: PRIMARY KEY must be unique` errors are issued. I believe this is due to the use of `factory_boy` with an abstract model. I tried adding the class `ImageModelFactory` as instructed in the `factory_boy` documentation, but that didn't suppress the error messages.
- Also, when the code assigning `date_taken` is executed, a `Runtime Warning:` message is issued to the runserver window. The `datetime` calculation needs to be passed to `make_aware` prior to being assigned to `date_taken`.
- These issues needs further investigation.
- Note that `Photo.delete()` and `Watermark.delete()` are not called on individual object instances when deleted in bulk via admin actions (i.e., the `Delete selected ...` dropdown menu). https://docs.djangoproject.com/en/dev/topics/db/queries/#deleting-objects. If so desired, the admin action `delete_selected` could be overridden, the queryset `delete()` could be overridden, a cron job could delete orphan files, or the orphan files could be deleted manually.