bmihelac/django-import-export — pull request #44 by pjdelport (2013-09-11 07:09:22+00:00)
Title: Prevent queryset caching when exporting

By default, Django querysets will cache result rows as they are loaded. This can waste a large amount of memory when iteratively consuming a large queryset, like `Resource.export()` does. In my case, trying to export in the order of tens of thousands rows will quickly consume gigabytes of memory, before running into swap, making the export practically unusable.

Calling [`iterator()`](https://docs.djangoproject.com/en/1.6/ref/models/querysets/#iterator) on the queryset fixes this behavior, and makes the large exports in question succeed without a problem.

The proposed change makes using `iterator()` the default behavior of `Resource.export()`, which should avoid the resource exhaustion problem without changing API behavior (particularly for code that overrides `ModelResource.get_queryset()` or `ExportMixin.get_export_queryset()`), and should not have any adverse effects.

(Note: This change does not affect `ModelInstanceLoader.get_queryset()`, which does not iterative access.)