Package: django-scopes
django-scopesMotivationMany of us use Django to build multi-tenant applications where every user only ever
gets access to a small, separated fraction of the data in our application, while
at the same time havingsomeglobal functionality that makes separate databases per
client infeasible. While Django does a great job protecting us from building SQL
injection vulnerabilities and similar errors, Django can't protect us from logic
errors and one of the most dangerous types of security issues for multi-tenant
applications is that we leak data across tenants.It's so easy to forget that one.filtercall and it's hard to catch these errors
in both manual and automated testing, since you usually do not have a lot of clients
in your development setup. Leavingradical, database-dependent ideasaside, there aren't many approaches available in the ecosystem to prevent these mistakes
from happening aside from rigorous code review.We'd like to propose this module as a flexible line of defense. It is meant to have
little impact on your day-to-day work, but act as a safeguard in case you build a
faulty query.InstallationThere's nothing required apart from a simplepip install django-scopesCompatibilityThis library is tested againstPython 3.8-3.10andDjango 3.2-4.0.UsageLet's assume we have a multi-tenant blog application consisting of the three modelsSite,Post, andComment:fromdjango.dbimportmodelsclassSite(models.Model):name=models.CharField(…)classPost(models.Model):site=models.ForeignKey(Site,…)title=models.CharField(…)classComment(models.Model):post=models.ForeignKey(Post,…)text=models.CharField(…)In this case, our modelSiteacts as the tenant for the blog posts and their comments, hence
our application will probably be full of statements likePost.objects.filter(site=current_site),Comment.objects.filter(post__site=current_site),
or more complex when more flexible permission handling is involved. Withdjango-scopes, we
encourage you to still write these queries with your custom permission-based filters, but
we add a custom model manager that has knowledge about posts and comments being part of a
tenant scope:fromdjango_scopesimportScopedManagerclassPost(models.Model):site=models.ForeignKey(Site,…)title=models.CharField(…)objects=ScopedManager(site='site')classComment(models.Model):post=models.ForeignKey(Post,…)text=models.CharField(…)objects=ScopedManager(site='post__site')The keyword argumentsitedefines the name of ourscope dimension, while the string'site'or'post__site'tells us how we can look up the value for this scope dimension
in ORM queries.You could have multi-dimensional scopes by passing multiple keyword arguments toScopedManager, e.g.ScopedManager(site='post__site', user='author')if that is
relevant to your usecase.Now, with this custom manager, all queries are banned at first:>>> Comment.objects.all()
ScopeError: A scope on dimension "site" needs to be active for this query.The only thing that will work isComment.objects.none(), which is useful e.g. for Django
generic view definitions.Activate scopes in contextsYou can now use our context manager to specifically allow queries to a specific blogging site,
e.g.:fromdjango_scopesimportscopewithscope(site=current_site):Comment.objects.all()This willautomaticallyadd a.filter(post__site=current_site)to all of your queries.
Again, we recommend that youstillwrite them explicitly, but it is nice to know to have a
safeguard.Of course, you can still explicitly enter a non-scoped context to access all the objects in your
system:withscope(site=None):Comment.objects.all()This also works correctly nested within a previously defined scope. You can also activate multiple
values at once:withscope(site=[site1,site2]):Comment.objects.all()Sounds cumbersome to put thosewithstatements everywhere? Maybe not at all: You probably
already have a middleware that determines the site (or tenant, in general) for every request
based on URL or logged in user, and you can easily use it there to just automatically wrap
it around all your tenant-specific views.Functions can opt out of this behavior by usingfromdjango_scopesimportscopes_disabledwithscopes_disabled():…# OR@scopes_disabled()deffun(…):…Please note thatdjango-scopesis also active during migrations, so if you are writing a
data migration – or have written one in the past! – you'll have to add appropriate scoping
or use thescopes_disabledcontext.Custom manager classesIf you were already using a custom manager class, you can pass it to aScopedManagerwith the_manager_classkeyword like this:
from django.db import modelsfromdjango.dbimportmodelsclassSiteManager(models.Manager):defget_queryset(self):returnsuper().get_queryset().exclude(name__startswith='test')classSite(models.Model):name=models.CharField(…)objects=ScopedManager(site='site',_manager_class=SiteManager)Scoping the User modelAssume you've got two modelsUserandPost. Using the examples above, you can ensure that users only ever see their own diary posts. But how about leaking other users to the currently logged in user? If you application doesn't have much (or any) interaction between users, you can scope the user model. Please note that you'll need acustom user model. Which base classes your user and manager work off will very between projects.classUser(AbstractUser):objects=ScopedManager(user='pk',_manager_class=UserManager)# (...)Activating the scope comes with a little caveat - you need to use the users primary key, not the whole object:withscope(user=request.user.pk):# do something :)CaveatsLockingWith django-scopes, a seemingly innocent query likeComment.objects.select_for_update().get(pk=3)could cause unexpected locking across your database, since django-scopes will auto-add one or moreJOINstatements to the query, and joined tables willalso be locked.
One possible fix is of course usingscopes_disabled(), around this query.
On most modern databases, there's also a way to specify explicitly which tables you want locked:Comment.objects.select_for_update(of=("self",)).get(pk=3)You can check if your database supports this feature at runtime usingconnection.features.has_select_for_update_of.Admindjango-scopesis not compatible with the django admin out of the box, integration requires a
custom middleware. (If you write one, please open a PR to include it in this package!)TestingWe want to enforce scoping by default to stay safe, which unfortunately
breaks the Django test runner as well as pytest-django. For now, we haven't found
a better solution than to monkeypatch it:fromdjango.testimportutilsfromdjango_scopesimportscopes_disabledutils.setup_databases=scopes_disabled()(utils.setup_databases)You can wrap many of your test and fixtures insidescopes_disabled()as well, but we wouldn't advise to do it with all of them: Especially when writing higher-level functional tests, such as tests using Django's test client or tests testing celery tasks, you should make sure that your application code runs as it does in production. Therefore, writing tests for a project using django-scopes often looks like this:@pytest.mark.django_dbdeftest_a_view(client):withscopes_disabled():u=User.objects.create(...)client.post('/user/{}/delete'.format(u.pk))withscopes_disabled():assertnotUser.objects.filter(pk=u.pk).exists()If you want to disable scoping or activate a certain scope whenever a specific fixture is used, you can do so in py.test like this:@pytest.fixturedefsite():s=Site.objects.create(...)withscope(site=s):yieldsWhen trying to port a project withlotsof fixtures, it can be helpful to roll a small py.test plugin in yourconftest.pyto just globally disable scoping for all fixtures which are not yielding fixtures (like the one above):@pytest.hookimpl(hookwrapper=True)defpytest_fixture_setup(fixturedef,request):ifinspect.isgeneratorfunction(fixturedef.func):yieldelse:withscopes_disabled():yieldModelFormsWhen using model forms, Django will automatically generate choice fields on foreign
keys and many-to-many fields. This won't work here, so we supply helper field
classesSafeModelChoiceFieldandSafeModelMultipleChoiceFieldthat use an
empty queryset instead:fromdjango.formsimportModelFormfromdjango_scopes.formsimportSafeModelChoiceFieldclassPostMethodForm(ModelForm):classMeta:model=Commentfield_classes={'post':SafeModelChoiceField,}django-filterWe noticed thatdjango-filteralso runs some queries when generating filtersets.
Currently, our best workaround is this:fromdjango_scopesimportscopes_disabledwithscopes_disabled():classCommentFilter(FilterSet):…UniquenessOne subtle class of bug that can be introduced by adding django-scopes to your project is if you try to generate unique identifiers in your database with a pattern like this:defgenerate_unique_value():whileTrue:key=_generate_random_key()ifnotModel.objects.filter(key=key).exists():returnkeyIf you want keys to be unique across tenants, make sure to wrap such functions withscopes_disabled()!When using aModelForm(orclass based view) to create or update a model, unexpected IntegrityErrors may occur. ModelForms perform a uniqueness check before actually saving the model. If that check runs in a scoped context, it cannot find conflicting instances, leading to an IntegrityErrors once the actual.save()happens. To combat this, wrap the call inscopes_disabled().classSite(models.Model):name=models.CharField(unique=True,…)# (...)defvalidate_unique(self,*args,**kwargs):withscopes_disabled():super().validate_unique(*args,**kwargs)Further readingIf you'd like to read more about the practical use of django-scopes, there is ablog
postabout its introduction in thepretixproject.Hereis a guide on how to write ashell_scopeddjango-admin command to provide a scoped Django shell.