Package: django-s3file
django-s3fileA lightweight file upload input for Django and Amazon S3.Django-S3File allows you to upload files directly AWS S3 effectively
bypassing your application server. This allows you to avoid long running
requests from large file uploads. This is particularly helpful for if
you run your service on AWS Lambda or Heroku where you have a hard
request limit.Featureslightweight: less 200 linesno JavaScript or Python dependencies (no jQuery)easy integrationworks just like the built-inextendable JavaScript APIFor the NerdssequenceDiagram
    autonumber
    actor Browser
    participant S3
    participant Middleware
    Browser->>Django: GET form view
    activate Django
    Django->>Browser: RESPONSE w/ presigned POST URL & signed middleware key
    deactivate Django
    Browser->>S3: POST large file
    activate S3
    S3->>Browser: RESPONSE AWS S3 key
    Browser->>Middleware: POST AWS S3 key (signed)
    activate Middleware
    Middleware->>S3: GET AWS S3 key
    S3->>Middleware: RESPONSE large file promise
    deactivate S3
    Middleware->>Django: request incl. large file promise
    deactivate Middleware
    activate Django
    opt only if files is procssed by Django
        Django-->>S3: GET large file
        activate S3
        S3-->>Django: RESPONSE large file
        deactivate S3
    end
    Django->>Browser: RESPONSE success
    deactivate DjangoIn a nutshell, we can bypass Django completely and have AWS handle
the upload or any processing. Of course, if you want to do something
with your file in Django, you can do so, just like before, with the
added advantage, that your file is served from within your datacenter.InstallationMake sure you haveAmazon S3
storagesetup correctly.Just install S3file usingpip.pipinstalldjango-s3file# orpipenvinstalldjango-s3fileAdd the S3File app and middleware in your settings:# settings.pyINSTALLED_APPS=('...','s3file','...',)MIDDLEWARE=('...','s3file.middleware.S3FileMiddleware','...',)UsageS3File automatically replaces Django'sClearableFileInputwidget, you
do not need to alter your code at all.TheClearableFileInputwidget is only than automatically replaced when
theDEFAULT_FILE_STORAGEsetting is set todjango-storages'S3Boto3Storageor the dummyFileSystemStorageis enabled.Setting up the AWS S3 bucketUpload folderS3File uploads to a single folder. Files are later moved by Django when
they are saved to theupload_tolocation.It is recommended tosetup
expirationfor that folder, to ensure that old and unused file uploads don't add up
and produce costs.The default folder name is:tmp/s3fileYou can change it by changing
theS3FILE_UPLOAD_PATHsetting.CORS policyYou will need to allowPOSTfrom all origins. Just add the following
to your CORS policy.[{"AllowedHeaders":["*"],"AllowedMethods":["POST"],"AllowedOrigins":["*"],"ExposeHeaders":[],"MaxAgeSeconds":3000}]Progress BarS3File does emit progress signals that can be used to display some kind
of progress bar. Signals namedprogressare emitted for both each
individual file input as well as for the form as a whole.The progress signal carries the following details:console.log(event.detail){progress:0.4725307607171312// total upload progress of either a form or single inputloaded:1048576// total upload progress of either a form or single inputtotal:2219064// total bytes to uploadcurrentFile:File{…}// file objectcurrentFileName:"text.txt"// file name of the file currently uploadedcurrentFileProgress:0.47227834703299176// upload progress of that fileoriginalEvent:ProgressEvent{…}// the original XHR onprogress event}The following example implements a Boostrap progress bar for upload
progress of an entire form.<divclass="progress"><divclass="progress-bar"role="progressbar"style="width: 0%;"aria-valuenow="0"aria-valuemin="0"aria-valuemax="100">0%</div></div>(function(){varform=document.getElementsByTagName('form')[0]varprogressBar=document.getElementsByClassName('progress-bar')[0]form.addEventListener('progress',function(event){// event.detail.progress is a value between 0 and 1varpercent=Math.round(event.detail.progress*100)progressBar.setAttribute('style','width:'+percent+'%')progressBar.setAttribute('aria-valuenow',percent)progressBar.innerText=percent+'%'})})()Using S3File in developmentUsing S3File in development can be helpful especially if you want to use
the progress signals described above. Therefore, S3File comes with a AWS
S3 dummy backend. It behaves similar to the real S3 storage backend. It
is automatically enabled, if theDEFAULT_FILE_STORAGEsetting is set
toFileSystemStorage.To prevent users from accidentally using theFileSystemStorageand the
insecure S3 dummy backend in production, there is also an additional
deployment check that will error if you run Django's deployment check
suite:pythonmanage.pycheck--deployWe recommend always running the deployment check suite as part of your
deployment pipeline.Uploading multiple filesDjango does have limited support foruploading multiple
files.
S3File fully supports this feature. The custom middleware makes ensure
that files are accessible viarequest.FILES, even though they have
been uploaded to AWS S3 directly and not to your Django application
server.Using optimized S3Boto3StorageSinceS3Boto3Storagesupports storing data from any other fileobj, it
uses a generalized_savefunction. This leads to the frontend
uploading the file to S3 and then copying it byte-by-byte to perform a
move operation just to rename the uploaded object. For large files this
leads to additional loading times for the user.That's why S3File provides an optimized version of this method atstorages_optimized.S3OptimizedUploadStorage. It uses the more
efficientcopymethod from S3, given that we know that we only copy
from one S3 location to another.froms3file.storages_optimizedimportS3OptimizedUploadStorageclassMyStorage(S3OptimizedUploadStorage):# Subclass and use like any other storagedefault_acl='private'