Package: django-import-export-celery
django-import-export-celery: process slow django imports and exports in celerydjango-import-export-celery helps you process long running imports and exports in celery.Basic installationSet up celeryto work with your project.Add'import_export_celery'to yourINSTALLED_APPSsettings variableAdd'author.middlewares.AuthorDefaultBackendMiddleware'to yourMIDDLEWARE_CLASSESConfigure the location of your celery module setupIMPORT_EXPORT_CELERY_INIT_MODULE = "projectname.celery"Setting up imports with celeryA fully configured example project can be found in the example directory of this repository.Perform the basic setup procedure described above.Configure the IMPORT_EXPORT_CELERY_MODELS variable.def resource():  # Optional
    from myapp.models import WinnerResource
    return WinnerResource


IMPORT_EXPORT_CELERY_MODELS = {
    "Winner": {
        'app_label': 'winners',
        'model_name': 'Winner',
        'resource': resource,  # Optional
    }
}The available parameters areapp_label,model_name, andresource. ‘resource’ should be a function which returns a django-import-exportResource.DoneBy default a dry run of the import is initiated when the import object is created. To instead import the file immediately without a dry-run set theIMPORT_DRY_RUN_FIRST_TIMEtoFalseIMPORT_DRY_RUN_FIRST_TIME = FalsePreforming an importYou will find an example django application that uses django-import-export-celery for importing data. There are instructions for running the example application in the example directory’s README file. Once you have it running, you can perform an import with the following steps.Navigate to the example applications admin page:Navigate to the ImportJobs table:Create a new import job. There is an example import CSV file in the example/example-data directory. Select that file. Select csv as the file format. We’ll be importing to the Winner’s model table.Select “Save and continue editing” to save the import job and refresh until you see that a “Summary of changes made by this import” file has been created.You can view the summary if you want. Your import has NOT BEEN PERFORMED YET!Return to the import-jobs table, select the import job we just created, and select the “Perform import” action from the actions drop down.In a short time, your imported Winner object should show up in your Winners table.Setting up exportsAs with imports, a fully configured example project can be found in theexampledirectory.Add aexport_resource_classesclassmethod to the model you want to export.@classmethod
def export_resource_classes(cls):
    return {
        'winners': ('Winners resource', WinnersResource),
        'winners_all_caps': ('Winners with all caps column resource', WinnersWithAllCapsResource),
    }This should return a dictionary of tuples. The keys should be unique unchanging strings, the tuples should consist of aresourceand a human friendly description of that resource.Add thecreate_export_job_actionto the model’sModelAdmin.from django.contrib import admin
from import_export_celery.admin_actions import create_export_job_action

from . import models


@admin.register(models.Winner)
class WinnerAdmin(admin.ModelAdmin):
    list_display = (
        'name',
    )

    actions = (
        create_export_job_action,
    )To customise export queryset you need to addget_export_querysetto theModelResource.class WinnersResource(ModelResource):
    class Meta:
        model = Winner

    def get_export_queryset(self):
        """To customise the queryset of the model resource with annotation override"""
        return self.Meta.model.objects.annotate(device_type=Subquery(FCMDevice.objects.filter(
                user=OuterRef("pk")).values("type")[:1])Done!Performing exports with celeryPerform the basic setup procedure described in the first section.Open up the object list for your model in django admin, select the objects you wish to export, and select theExport with celeryadmin action.Select the file format and resource you want to use to export the data.Save the modelYou will receive an email when the export is done, click on the link in the emailClick on the link near the bottom of the page titledExported file.Excluding export file formats in the admin siteAll available file formats to export are taken from theTablib project.To exclude or disable file formats from the admin site, configureIMPORT_EXPORT_CELERY_EXCLUDED_FORMATSdjango settings variable. This variable is a list of format strings written in lower case.IMPORT_EXPORT_CELERY_EXCLUDED_FORMATS = ["csv", "xls"]Customizing File Storage BackendDefine a custom storage backend by adding theIMPORT_EXPORT_CELERY_STORAGEto your Django settings. For instance:IMPORT_EXPORT_CELERY_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"Customizing email template for export job completion emailBy default this is the subject and template used to send the emailSubject: 'Django: Export job completed'
Email template: 'email/export_job_completion.html'The default email template can be foundhereThe default email subject and template can be customized by overriding these values from django settings:-EXPORT_JOB_COMPLETION_MAIL_SUBJECT="Your custom subject"
EXPORT_JOB_COMPLETION_MAIL_TEMPLATE="path_to_folder/your_custom_template.html"The email template will get some context variables that you can use to customize your template.{
    export_job: The current instance of ExportJob model
    app_label: export_job.app_label
    model: export_job.model
    link: A link to go to the export_job instance on django admin
}For developers of this libraryYou can enter a preconfigured dev environment by first runningmakeand then launching./develop.shto get into a docker compose environment packed withredis,celery,postgresand everything you need to run and test django-import-export-celery.Before submitting a PR please runflake8and (in the examples directory)python3 manange.py test.Please note, that you need to restart celery for changes to propogate to the workers. Do this withdocker-compose down celery,docker-compose up celery.Commercial supportCommercial support is provided bygradesta s.r.o.Creditsdjango-import-export-celerywas developed by the Czech non-profitauto*mat z.s..