Package: django-fast-ratelimit
django-fast-ratelimitDjango-fast-ratelimit provides a secure and fast ratelimit facility based on the django caching framework.
It uses a "Fixed window counter"-algorithm based on:https://medium.com/figma-design/an-alternative-approach-to-rate-limiting-f8a06cf7c94cInstallationpipinstalldjango-fast-ratelimitNote: pip >= 19 is requiredNote: version 5 renames package from ratelimit to django_fast_ratelimitusageApply everywhere where wanted in the django app without restrictions:install companion library django-fast-iprestrict >= 0.6.0Decorator:importdjango_fast_ratelimitasratelimit@ratelimit.decorate(key="ip",rate="1/s")defexpensive_func(request):# how many ratelimits request limitingifrequest.ratelimit["request_limit"]>0:# reschedule with end of rate epochreturnrequest_waiting(request.ratelimit["end"])or asyncimportdjango_fast_ratelimitasratelimitimportasyncio@ratelimit.decorate(key="ip",rate="1/s")asyncdefexpensive_func(request):# how many ratelimits request limitingifrequest.ratelimit["request_limit"]>0:# reschedule with end of rate epochawaitasyncio.sleep(request.ratelimit["end"])blocking Decorator (raises RatelimitError):importdjango_fast_ratelimitasratelimit@ratelimit.decorate(key="ip",rate="1/s",block=True,decorate_name="ratelimit",methods=ratelimit.UNSAFE)defexpensive_func(request):# how many ratelimits request limitingifrequest.ratelimit["end"]>0:decorate View (requires group):importdjango_fast_ratelimitasratelimitfromdjango.views.genericimportViewfromdjango.utils.decoratorsimportmethod_decorator@method_decorator(ratelimit.decorate(key="ip",rate="1/s",block=True,methods=ratelimit.SAFE,group="required"),name="dispatch")classFooView(View):...manualimportdjango_fast_ratelimitasratelimitdeffunc(request):ratelimit.get_ratelimit(key="ip",rate="1/s",request=request,group="123",action=ratelimit.Action.INCREASE)# or only for GETratelimit.get_ratelimit(key="ip",rate="1/s",request=request,group="123",methods="GET",action=ratelimit.Action.INCREASE)# also simple calls possible (note: key in bytes format)ratelimit.get_ratelimit(key=b"abc",rate="1/s",group="123",action=ratelimit.Action.INCREASE)# retrieve ratelimitrlimit=ratelimit.get_ratelimit(key="ip",rate="1/s",request=request,group="123")# reset (clears internal counter)counter_before_reset=rlimit.reset()# reset epoch (resets to the start of request/epoch)counter_before_reset=rlimit.reset(request)# decrease counter by arbitary amountrlimit.reset(19)# increase counter by arbitary amountrlimit.reset(-19)# check constraints of rater=ratelimit.parse_rate("1/s")# returns tuple (amount, period)assert(r[1]==1)#  assert period is 1 second# for simple naming use o2g (object to group)ratelimit.get_ratelimit(key=b"abc",rate=r,group=ratelimit.o2g(func),action=ratelimit.Action.INCREASE)manual asyncimportdjango_fast_ratelimitasratelimitasyncdeffunc(request):# retrieve ratelimitrlimit=awaitratelimit.aget_ratelimit(key="ip",rate="1/s",request=request,group="123")# reset (clears internal counter)awaitrlimit.areset()# reset epoch (resets to the start of request/epoch)awaitrlimit.areset(request)# decrease counter by arbitary amountawaitrlimit.areset(19)# increase counter by arbitary amountawaitrlimit.reset(-19)parametersratelimit.get_ratelimit:group: group name, can be callable (fun(request, action))rate: rate limit, multiple modes
Note: if count (first argument) is 0, then it raises the Disabled exception, the second argument must be greater then 0str: default mode , specify rate in form of "1/4s" or "2/s" or "2/m"2 element tuple/list: first argument is amount, second are secondscallable: can return either string or 2 element tuple/list
None (default): raise ValueError if key function doesn't return False to disable further execution, for django-fast-iprestrictmethods: set of checked methods, can be callable (fun(request, group, action)), modes:callable(request, group, action): allow dynamicratelimit.ALL (default): all methods are checked("HEAD", "GET"): list of checked methodsratelimit.invertedset(["HEAD", "GET"]): inverted set of checked methods. Here: every method is checked, except HEAD, GETrequest: ingoing request (optional if key supports it and methods=ratelimit.ALL (default))key: multiple modes possible:bool: True: skip key (should only be used for baking), False: disable cache (like RATELIMIT_ENABLED=False)int: sidestep cache, value will be used for request_limit. 0 is for never blocking, >=1 blocksstr: "path.to.method:argument"str: "inbuildmethod:argument" see methods for valid argumentsstr: "inbuildmethod" method which is ready to use for (request, group, action)tuple,list: ["method"/callable, args...]: method (can be also inbuild or a callable) with arbitary argumentsbytes: static key (supports mode without request)callable: check return of function (fun(request, group, action)), return must be string (converted to bytes), bytes, bool or int (see "key" for effects)empty_to: convert empty keys (b"") to parameter. Must be bytes, bool or int (see "key" for effects) (default: keep b"")cache: specify cache to use, defaults to RATELIMIT_DEFAULT_CACHE setting (default: "default")hash_algo: name of hash algorithm for creating cache_key (defaults to RATELIMIT_KEY_HASH setting (default: "sha256"))
Note: group is seperately hashedhashctx: optimation parameter, read the code and only use if you know what you are doing. It basically circumvents the parameter hashing and only hashes the key. If the key parameter is True even the key is skippedaction {ratelimit.Action}:PEEK: only lookupINCREASE: count up and return resultRESET: return former result and resetRESET_EPOCH: return count after reset of epoch. If neither epoch nor request is given like peek (default: {PEEK})epoch:None: (default): use request as epochint: RESET_EPOCH resets by int. Negative int increasesobject: attach counter to an open object (Note: it cannot be object() directly and neither an object with slots)returns the dataclass Ratelimitor raisesratelimit.Disabledin case of the count in the rate is zeroratelimit.RatelimitFieldscount: how often in the window the ip whatever was callinglimit: limit when it should blockrequest_limit: >=1 should block or reject, 0: should acceptend: when does the block endgroup: group namegroup_key, cache: Optional, when specified reset and areset can be used, internal fieldsFunctions:can_reset: is a reset possible or were bypasses usedreset: function to reset count if cache was used. When given an epoch the same as RESET_EPOCHareset: async version of resetcheck(block=False): raise RatelimitExceeded when block = True and ratelimit is exceededacheck(wait=False, block=False): raise RatelimitExceeded when block = True and ratelimit is exceeded, wait for end of ratelimit duration when wait=Truedecorate_object(obj, name="ratelimit", block=False, replace=False): attach to object obj with name and use old limits too, pass block to checkadecorate_object(obj, name="ratelimit", wait=False, block=False, replace=False): attach to object obj with name and use old limits too, pass block and wait to acheckNote: decorate_object with name=None behaves like check (except return value), the same applies for adecorate_objectarguments:wait: wait until end timestamp when ratelimit was exceeded. Next call should work again, applied before blockblock: raise a RatelimitExceeded exceptionreplace: ignore potential old ratelimit object atttached to object and just replace itwhy only async methods have wait? It doesn't really block (only the userlandthread). In contrast to its sync equivalent it doesn't block the webserver significantlyExample: decorate_objectimportratelimitclassFoo():passr=get_ratelimit(group="foo",rate="1/s",key=b"foo",action=ratelimit.Action.INCREASE,)# manual wayfoo=r.decorate_object(Foo(),name="ratelimit")ifnotfoo.ratelimit.check():raiseratelimit.RatelimitExceeded("custom message",ratelimit=r)else:pass# do cool stuff# simplifiedfoo2=r.decorate_object(Foo(),block=True)# artistic (no point in doing so)r.decorate_object(Foo(),name="ratelimit_is_cool").ratelimit_is_cool.check(block=True)# like check with instance of Foo() as return valuefoo3r.decorate_object(Foo(),name=None,wait=True)# decorate function@r.decorate_object(block=True)deffn():pass# of course also this works@r.decorate_objectdeffn():passratelimit.aget_ratelimit:same asget_ratelimitbut supports async methods and has an optional parameter:wait, which suspends the execution (viaasyncio.sleep) for the time specified in rate (second argument).
This is only possible in async mode, as it would block too much in sync mode.ratelimit.decorate:All of ratelimit.get_ratelimit except request. group is here optional (except for decorations with method_decorator (no access to wrapped function)).
Also supports:block: should hard block with an RatelimitExceeded exception (subclass of PermissionDenied) or only annotate request with ratelimitdecorate_name: under what name the ratelimit is attached to the request. set to None/empty to not decorate request. Uses Ratelimit.decorate_object. Defaults to "ratelimit"wait (only when async possible): suspends execution, triggers force_async when not specifiedforce_async: fail when not called in an async context, can be set to False, to not fail in sync context, or to True to fail always when in sync only contextwhy only async methods have wait? It doesn't really block (only the userlandthread). In contrast to its sync equivalent it doesn't block the webserver significantlyhelpersratelimit.invertedset:inverts a collection, useful for http methodsratelimit.get_RATELIMIT_TRUSTED_PROXY:get theRATELIMIT_TRUSTED_PROXIESparsed as setnote: this function uses a cached subfunction. If you change this setting while testing you may have to call:ratelimit._get_RATELIMIT_TRUSTED_PROXY.cache_clear()ratelimit.get_ip:get client ip from request, usingRATELIMIT_TRUSTED_PROXIESand forwarded headersimportratelimitratelimit.get_ip(request)ratelimit.o2g:auto generate group names for method/function as input, see tests/test_decorators for more infoExample:importratelimitclassO2gView(View):defget(self,request,*args,**kwargs):request.ratelimit2=ratelimit.get_ratelimit(group=ratelimit.o2g(self.get),rate="1/s",key=b"o2gtest",action=ratelimit.Action.INCREASE,)ifrequest.ratelimit2.request_limit>0:returnHttpResponse(status=400)returnHttpResponse()ratelimit.RatelimitExceededRaised when the ratelimit was exceededException, required keyword argument is ratelimit with the ratelimit.
The next arguments are passed to the underlying standard exception class for e.g. customizing the error messageratelimit.DisabledStronger variant of RatelimitExceeded. Used for cases where limit is 0 and there is no way to pass the ratelimit.
It is a shortcut for disabling api.Note: it is weaker than the settingRATELIMIT_ENABLEDNote: it isn't a subclass from RatelimitExceeded because APIs should be able to differ both casesNote: in contrast to RatelimitExceeded it is raised in (a)get_ratelimit and when using decorate, the view function isn't called.ratelimit.protect_sync_onlyfor libraries. In case of async return protected asyncified function otherwise call library directlyrecipesjitter:importratelimitimportasyncioimportsecretsasyncdeffoo()r=awaitratelimit.aget_ratelimit(group="foo",rate="1/s",key=b"foo",action=ratelimit.Action.INCREASE,)# 100ms jitterawaitasyncio.sleep(secrets.randbelow(100)/100)# raise when limit reached, wait until full second jitter is eliminated in raise case as end was created before the jitterawaitr.acheck(wait=True,block=True)methodsSee in methods which methods are available. Here some of them:static: use static key defined by argument, if no argument was specified default to b"static", the argument is automatically converted to bytes
Note: the conversion for non-bytes objects is str(obj).encode("utf8")
Note: it is also possible to specify a bytes key to provide a static argumentip: use ip address as key, argument: [netmask ipv4/]netmask ipv6user: authenticated user primary key or b""user_or_ip: use autenticated user primary key as key. If not autenticated fallback to ip, also with netmask argumentuser_and_ip: same like user_or_ip except that the ip matching also applies for authenticated usersip_exempt_user: same like user_or_ip except that authenticated users are exempted, also with netmask argumentwhen specified with reset actions: reset the ip key with the reset action when a user was founduser_or_ip_exempt:
user with fallback ip.Note:
when permissions, user_ok and staff_ok are not specified exempts for superuser only.
Same behavior for reset like ip_exempt_userWith following parameters (either list or comma seperated):netmask: either ["netmask", ...] or "netmask:.../..."not_use_user_pk: when not exempting, the default uses preferred the authenticated user pk, this param skips this and directly fallback to ip, note: the boolean name is invertednot_use_ip: block if no user was found, together with not_use_user_pk block non privileged accesspermissions: either ["permission", perm1, perm2, ..] or "permission:perm1"user_ok: string, exempts when authenticated userstaff_ok: string, exempts when staffget: generate key from multiple sources, input can be multiple input args or a dict with optionssettingsRATELIMIT_TESTCLIENT_FALLBACK: in case instead of a client ip a testclient is detected map to the fallback. Set to "invalid" to fail. Default ::1RATELIMIT_GROUP_HASH: hash function which is used for the group hash (default: md5)RATELIMIT_KEY_HASH: hash function which is used as default for the key hash, can be overridden with hash_algo (default: md5)RATELIMIT_ENABLEDdisable ratelimit (e.g. for tests) (default: enabled)RATELIMIT_ENABLEdeprecated old name of RATELIMIT_ENABLEDRATELIMIT_KEY_PREFIX: internal prefix for the hash keys (so you don't have to create a new cache). Defaults to "frl:".RATELIMIT_DEFAULT_CACHE: default cache to use, defaults to "default" and can be overridden by cache parameterRATELIMIT_TRUSTED_PROXIES: "all" for allowing all ip addresses to provide forward informations, or an iterable with proxy ips (will be transformed to a set). Note there is a special ip: "unix" for unix sockets. Default: ["unix"]
Used headers are:Forwarded,X-Forwarded-ForUpdate Notes:in version 9.0.0: ip_exempt_superuser and ip_exempt_privileged are replaced by user_or_ip_exemptin version 8.0.0: rate is the 4th argument of a key function, I need it for django-fast-iprestrictin version 7.3.0: rate is now optional (when having an appropiate key (function))in version 7.2.0:RATELIMIT_ENABLEis renamed toRATELIMIT_ENABLED, the old setting is still available, note: in tests where this settings are changed dynamically you may have to import _get_RATELIMIT_ENABLED and clear the cache, in most cases this isn't neccessaryin version 7.0.0 method, group and key functions take an additional parameter: actionin version 6.0.0 some small new restrictions are introduced for key functions as stringin version 5.0.0 the package was renamed to django_fast_ratelimit for having an unique namespace. Reason, we have now a companion library: django-fast-iprestrict
Sorry for the big breaking change.in version 4.0.0 most parameters were made keyword only (helps finding bugs).in version 3.0.0 the name parameter of (a)decorate_object was changed to ratelimitin version 2.0.0 the parameterraise_on_limitwas removed and replaced by check(block=True)in version 1.0.0 the parameterinclude_resetwas removedin version 1.2.0 reset_epoch calls return the counter before reset instead of the count afterTODO:document and test "get" inbuild method