Package: django-dynamic-storages
Django Dynamic StoragesI'll fill this in with more detail later, but the basic reasoning for this project is that I needed something I could use to extend the already excellentdjango-storagesproject such that the storage field on any givenFileFieldwas a callable I could source a model instance value from. I've also included an abstract class for holding a configuration that stores (securely viadjango-fernet-fields) the access configuration for a given storage backend as well.GoalsI wanted a way to:dynamically/at runtime, assign adjango-storagesbackend to a givenFileFieldin a modeldynamically/at runtime, manage the encryption/decryption of the contents of the aforementionedFileFieldIdeally, I'd like to be able to achieve this following idiomatic patterns fromdjango-storagesin terms of using callable kwargs to achieve this so that the consumer of these components had as much control over details as possible with the least amount of overhead.I took theupload_toimplementation as inspiration for how to implement the above.ComponentsFieldsdynamic_storages.fields.dynamic_storageDynamicStorageFileFieldandDynamicStorageImageFieldare extensions each of the built-in DjangoFileFieldandImageFieldwhich utilizes the storage backends provided bydjango-storages. Rather than setting thestoragekwarg at the field level, we're using a kwarg we namedstorage_instance_callableto manage the callable function workflow for determining the backend to use. This is, primarily, due to Django 3.0 allowing a callable to be passed in for thestoragekwarg, however it does not pass the callable function any context - we simply sidestep that issue by using a different kwarg and then use that to set the storage value.FOr the sake of providing an example, the following is a real-world usage of this process using a callable function which takes the model instance in as an argument. It identifies if a 'team' is attached to the instance and then finds theTeamStorageTarget(a separate model derived fromdynamic_storages.models.AbstractStorageTarget)defget_storage(instance):ifinstanceandgetattr(instance,"team",None):tst=TeamStorageTarget.for_team(instance.team)iftst:returntst.storage_backendreturndefault_storagedynamic_storages.fields.encrypted_contentEncryptedFileFieldandEncryptedImageFieldare extensions each ofDynamicStorageFileFieldandDynamicStorageImageFieldwith the main difference being that these new field definitions encapsulate the process of encrypting and decrypting content for their files at runtime. Two callables are needed to do this, both of which will receive theinstanceobject when called:fernet- the callable assigned to this should return acryptography.fernet.Fernetorcryptography.fernet.MultiFernetfor the handling of encryption operationsget_url- the callable assigned to this should return a URL which is configured/designed to decrypt the content and serve it to the end user as appropriate (i.e. security is baked into your views, not into this field directly). An example of what a view might look like for this is located in'dynamic_storages/views.py'.In a real-world application, your callable forget_urlmight take multiple values from the instance the file belongs to and construct a url to return:deffile_content_url(instance):returnreverse_lazy("api:team-files-contents",args=[instance.file.team.slug,instance.team.id,instance.team.file.name.split(".")[-1]])The view that gets exposed for thisget_urlmight be a custom action within a Django Rest Framework Viewset such as:classFileViewSet(viewsets.ModelViewSet):"""other implementation details not included..."""@action(detail=True,methods=["GET"],name="Get File Contents",url_path="contents.<str:file_ext>")defcontents(self,request,**kwargs):user=request.usertry:ifrequest.user.is_authenticated():team=get_object_or_404(Team,slug=kwargs.get("parent_lookup_team__slug"))file_ext=self.kwargs["file_ext"]ifuser_can_access_team(user,team):f=get_object_or_404(File,team=team,pk=kwargs["pk"])content=f.file.get_decrypted().read()returnHttpResponse(content,content_type=magic.Magic(mime=True).from_buffer(content),)else:log.warn("User{}is not allowed to access resources for team{}".format(user.username,team.name))else:log.warn("User is not authenticated for request to file with{}".format(self.kwargs))exceptExceptionasex:log.warn("Failed to get property for request:{}".format(ex))raiseHttp404dynamic_storages.fields.encrypted_jsonTheEncryptedJSONFieldis meant to be useful in handling the storage of things like storage credentials within a database. It was useful in our testing and we've found that even though it follows an older pattern in the Django community around JSONFields (given that Django 3 has them built in now), this setup serves well for credential storage.  It leverages thedjango-fernet-fieldsEncryptedTextFieldas its storage medium and we're simply marshaling it back and forth from dictionary to string.Other StuffRunning TestsThis project has test-level models... as such, to run the tests built into this project, you'll want to run Django from the./runtests.pyscript which simply adds the test application to theINSTALLED_APPSlist../runtests.pytestTo-DoUpdate the.models.AbstractStorageTargetmodel to include credential checkingCreate async task for handling credential checking periodically forAbstractStorageTarget-based storagelook into further extendingdjango-storagesto handle google drive (since it already supports dropbox)simplify and validate usage of the genericAbstractSecureFileContentsGenericFileContentsViewviewwrite more tests around thedefault_storageprovider (0.4.1)Revisions0.5.3(Mon Nov 23 01:26:23 PM PST 2020) - missed a reference to a variable I'd pulled out/renamed where I was determining mimetype for the file download0.5.2(Mon Nov 23 01:00:08 PM PST 2020) - so I had to take a dependency ondjango-rest-frameworkbut it made the generic views for retrieving encrypted content a lot easier to deal with. New tests and tweaks to the generic views and we're in business!0.5.1(Mon Nov 23 12:!5:14 PM PST 2020) - Renaming Abstract views to 'Generic' and layering them in so that they can be extended/used more easily0.5.0(Mon Nov 23 11:42:33 AM PST 2020) - fixing formatting, ranisortand added some logging to theAbstractStorageTargetprocess which constructs the backend classes.0.4.5(Mon Nov 23 11:24:18 AM PST 2020) - updates to generic/abstract view to pullmime_typefrom a model field if it's specified... adding common logger for all components.0.4.2(Fri Nov 20 07:04:43 PM PST 2020) - wrote more tests around storage providers, object create (for file and img fields) as well as some tidying up around the edges0.4.0(Mon Nov 16 09:50:48 AM PST 2020) - readme updates, documentation fixes, more clarity/examples, added some todo's0.3.8(Mon Nov 16 09:22:41 AM PST 2020) - docstring updates and some changes to the order of operations re: generating a download url for a file via callables0.3.7(Mon Nov 16 09:05:32 AM PST 2020) - Fixing typo0.3.6(Fri Nov 13 10:01:38 AM PST 2020) - consistency issues with get_url call again... mirroring process from the fernet setup0.3.5(Thu Nov 12 03:53:03 PM PST 2020) - fixes to myget_urlstuff - dumb misses0.3.4(Thu Nov 12 11:42:21 AM PST 2020) - minor update to allow for a callable function to be passed in for anEncryptedFileFieldso that you can externally control the view/url that needs to be used to get the contents of this file.