one minor correction is that this is a celery task, so it will be running on the celery worker machine (last I understood, that would be a different execution environment than the app servers).

But yes, I agree with paging being better also.