Package: django-request-profiler
This package now requires Python 3.9 and Django 3.2 and above.A very simple request profiler for Django.IntroductionPremature optimization is the root of all evil.There are a lot of very good, and complete, python and django profilers
available. They can give you detailed stack traces and function call timings,
output all the SQL statements that have been run, the templates that have been
rendered, and the state of any / all variables along the way. These tools are
great for optimisation of your application, once you have decided that the
time is right.django-request-profileris not intended to help you optimise, but to help
you decide whether you need to optimise in the first place. It is complimentary.RequirementsSmall enough to run in productionAble to configure profiling at runtimeConfigurable to target specific URLs or usersRecord basic request metadata:Duration (request-response)Request path, remote addr, user-agentResponse status code, content lengthView functionDjango user and session keys (if appropriate)Database query count (if DEBUG=True)It doesn’t need to record all the inner timing information - the goal is to have
a system that can be used to monitor site response times, and to identify
problem areas ahead of time.Technical detailsThe profiler itself runs as Django middleware, and it simply starts a timer when
it first sees the request, and stops the timer when it is finished with the
response. It should be installed as the first middleware inMIDDLEWARE_CLASSESin order to record the maximum duration.It hooks into theprocess_requestmethod to start the timer, theprocess_viewmethod to record the view function name, and theprocess_responsemethod to stop the timer, record all the request
information and store the instance.The profiler is controlled by addingRuleSetinstances which are used to
filter which requests are profiled. There can be many, overlapping,
RuleSets, but if any match, the request is profiled. The RuleSet model
defines two core matching methods:1. uri_regex - in order to profile a subset of the site, you can supply a regex
which is used match the incoming request path. If the url matches, the request
can be profiled.2. user_filter_type - there are three choices here - profile all users, profile
only authenticated users, and profile authenticated users belonging to a given
Group - e.g. create a groups called “profiling” and add anyone you want to
profile.These filter properties are an AND (must pass the uri and user filter), but the
rules as a group are an OR - so if a request passes all the filters in any rule,
then it’s profiled.These filters are pretty blunt, and there are plenty of use cases where you may
want more sophisticated control over the profiling. There are two ways to do
this. The first is a setting,REQUEST_PROFILER_GLOBAL_EXCLUDE_FUNC, which is
a function that takes a request as the single argument, and must return True or
False. If it returns False, the profile is cancelled, irrespective of any rules.
The primary use case for this is to exclude common requests that you are not
interested in, e.g. from search engine bots, or from Admin users etc. The
default for this function is to prevent admin user requests from being profiled.The second control is via thecancel()method on theProfilingRecord,
which is accessible via therequest_profile_completesignal. By hooking
in to this signal you can add additional processing, and optionally cancel
the profiler. A typical use case for this is to log requests that have
exceeded a set request duration threshold. In a high volume environment you
may want to, for instance, only profile a random subset of all requests.fromdjango.dispatchimportreceiverfromrequest_profiler.signalsimportrequest_profile_complete@receiver(request_profiler_complete)defon_request_profile_complete(sender,**kwargs):profiler=kwargs.get('instance')ifprofiler.elapsed>2:# log long-running requests# NB please don't use 'print' for real - use loggingprintu"Long-running request warning:%s"%profilerelse:# calling cancel means that it won't be saved to the dbprofiler.cancel()An additional scenario where you may want to use the signal is to store
the profiler records async - say if you are recording every request for
a short period, and you don’t want to add unnecessary inline database
write operations. In this case you can use thestop()method, which
will prevent the middleware from saving it directly (it will only save
records whereprofiler.is_runningis true, and bothcancelandstopset it to false).fromdjango.dispatchimportreceiverfromrequest_profiler.signalsimportrequest_profile_complete@receiver(request_profiler_complete)defon_request_profile_complete(sender,**kwargs):profiler=kwargs.get('instance')# stop the profiler to prevent it from being saved automaticallyprofiler.stop()assertnotprofiler.is_running# add a job to a queue to perform the save itselfqueue.enqueue(profiler.save)InstallationFor use as the app in Django project, use pip:$pipinstalldjango-request-profiler# For hacking on the project, pull from Git:$gitpullgit@github.com:yunojuno/django-request-profiler.gitTestsThe app installer contains a test suite that can be run using the Django
test runner:$pipinstall-rrequirements.txt$pythonmanage.pytesttest_apprequest_profilerIf you want to test coverage you’ll need to add some dependencies:$pipinstallcoveragedjango-coverage$pythonmanage.pytest_coveragetest_apprequest_profilerThe tests also run usingtox:$pipinstalltox$toxNote: To test with a custom user model, you should override the default User model
by providing a value for the AUTH_USER_MODEL (in testapp/settings) setting that references a custom modelThe tests run onTravison commits to master.UsageOnce installed, add the app and middleware to your project’s settings file.
In order to add the database tables, you should run themigratecommand:$pythonmanage.pymigraterequest_profilerNB the middleware must be thefirstitem inMIDDLEWARE_CLASSES.INSTALLED_APPS=('django.contrib.admin','django.contrib.auth','django.contrib.contenttypes','django.contrib.sessions','django.contrib.messages','django.contrib.staticfiles','request_profiler',)MIDDLEWARE_CLASSES=[# this package's middleware'request_profiler.middleware.ProfilingMiddleware',# default django middleware'django.middleware.common.CommonMiddleware','django.contrib.sessions.middleware.SessionMiddleware','django.contrib.auth.middleware.AuthenticationMiddleware','django.middleware.csrf.CsrfViewMiddleware','django.contrib.messages.middleware.MessageMiddleware',]ConfigurationTo configure the app, open the admin site, and add a new request profiler
‘Rule set’. The default options will result in all non-admin requests being
profiled.LicenceMIT (see LICENCE)