Package: django-apm
django-apmA fully featured APM for django using django.Features:Know each view is getting more requests/errors;Keep track of response times;Registers errors and notify you/your team on slack/discord with the exception, traceback and logs;Keep track of requests headers/query and payloads (on errors), making it easy to reproduce them in a separate environment.Notify your team using celery if desired;Notifications example:
On slack:On discord:InstallationInstall the package using your favorite packaging tool: pip / poetry / pdm, etc.Adddjapm.apm.apps.ApmConfigtoINSTALLED_APPSin your django settings. Make sure to add it beforedjango.contrib.admin.Otherwise, the dashboard link will not show up in Django admin.
If you're going to use theapm_api_view: Don't forget to addrest_frameworkto yourINSTALLED_APPS.
It also requires thedjango.contrib.sitesapp installed. It must come before thedjango.contrib.adminfor the dashboard link show up.Adding the middlewaresAdding the ApmMetricsMiddleware middleware (optional).
If you want to track the requests timing, add thedjapm.apm.middlewares.ApmMetricsMiddlewaretoMIDDLEWAREin your django settings. Note that this will store into your databaseallrequests that use the decoratorapm_api_view(This decorator is explained later), no cleanup is done. Is up to you to clean the registers periodically if desired. It's better to keep this middleware theclosest to the topof yourMIDDLEWAREas possible , this way theellapsedtime is closer to the reality.Adding the ErrorTraceMiddleware (optional).
If you want to track errors and receive notifications, add thedjapm.apm.middlewares.ErrorTraceMiddlewaretoMIDDLEWAREin your django settings. We recommend that you keep this middlewareclosest to the bottomof yourMIDDLEWAREas possible. This middleware will notify any exception raised on your view to all integrations added (the setup is explained later).Including the URLsdjapmcomes with a dashboard that's accessible through the django-admin. It only allows superusers to view it. To enable the dashboard include the following in your rooturls.pyfile:fromdjango.urlsimportincludeurlpatterns=[# Your other patterns...path("apm/",include("djapm.apm.urls")),]Now go to the admin, you should be able to see the dashboard link.The dashboard shows the data collected from the middlewares. It useschart.jsto render the graphics. The data is get from the database via API. Example:Upgrading your viewsdjango-apm comes with 3 decorators:apm_api_view,apm_viewandapm_admin_view. Also, it comes with two ClassBasedViews:ApmViewandApmAPIView. Also there's a ModelAdmin for tracking POST requests:ApmModelAdmin. Each of these adds some attributes to therequest, they are:id(str): A string UUID4;logger(logging.Logger): A logger that you can use to log to the sdout and keep track of all logs emitted._log_handler(djapm.apm.log.ApmStreamHandler): We use this handler so we can capture all logs emitted by your view;Do not use this directly._json(Any): An alias torest_framework.Request.datawe store this because it's only available at the view, and if not stored, we wouldn't be able to track the payload in the middleware.If you enabled theApmMetricsMiddleware, this request and response will be saved to the database. If you enabled theErrorTraceMiddlewareany errors uncaught raised by your view will be notified to you. This does not includes therest_framework.serializers.ValidationError, sincerest_frameworkitself already handles this exception.So, let's get started with each decorator:apm_api_view:
This decorator is an extension of therest*framework.decorators.api_viewdecorator, so you don't need to wrap your view with therest_frameworkdecorator.Example api view:fromdjapm.apm.decoratorsimportapm_api_viewfromdjapm.apm.typesimportApmRequest@apm_api_view(["GET"])defyour_view(request:ApmRequest,**kwargs):logger=request.loggerreq_id=request.id...What's changed from therest_frameworkfunction-based view:Therequestparameter is now of the typeApmRequest, it inherits fromrest_framework.request.Requestso you still have access to all it's attributes.
You receive aloggerand aidthat you can use. Theloggerwas created just before your view using a default, if you need to use a different logger you can pass the logger name in the decorator.apm_view:
Add this decorator to all your django regular function-based view's that you want to keep track of.Example django view:fromdjapm.apm.decoratorsimportapm_viewfromdjapm.apm.typesimportPatchedHttpRequest@apm_view()defyour_django_regular_view(request:PatchedHttpRequest,**kwargs):...apm_admin_view:
Add this decorator to all your django regular admin view's that you want to keep track of.Example django admin view:fromdjango.contribimportadminfromdjapm.apm.decoratorsimportapm_admin_viewfromdjapm.apm.typesimportPatchedHttpRequestclassYourAdmin(admin.ModelAdmin):@apm_admin_view()defyour_custom_admin_view(self,request:PatchedHttpRequest,**kwargs):...# don't forget to override the `get_urls` the same way as when you did with your custom view.The class-based-view (CBV) version:If you want to keep track of your CBVs. Inherit theApmViewfor your regular django views, or theApmAPIViewfor your DRF's API views. This way we can add the required attributes before your view gets called.Example django CBV view:fromdjango.views.genericimportCreateView# or any other viewfromdjapm.apm.viewsimportApmViewfromdjapm.apm.typesimportPatchedHttpRequestclassYourCreateView(CreateView,ApmView):...defpost(self,request:PatchedHttpRequest,**kwargs):...Example DRF CBV view:fromdjapm.apm.genericsimportApmCreateAPIView# or any other viewfromdjapm.apm.typesimportApmRequestclassYourAPICreateView(ApmCreateAPIView):...defcreate(self,request:ApmRequest,*args,**kwargs):...The Django Admin POST requests:
If you want to keep track of your admin model Creations, inherit fromApmModelAdmininstead of the regularadmin.ModelAdmin. Example:fromdjango.contribimportadminfromdjapm.apm.adminimportApmModelAdminfromyourapp.modelsimportAModel@admin.register(AModel)classYourModelAdmin(ApmModelAdmin):...Running the migrations:Since we rely on the database, you must migrate your database:python manage.py migrate. This will create these models:ApiRequest- Keep tracks of requests, including headers, query parameters;ApiResponse- Keep tracks of responses, including status_code, ellapsed time.ErrorTrace- Keep tracks of errors on your views.RequestLog- Keep track of all logs that were logged from therequest.logger;Integration- Keep track of all your integrations;NotificationReceiver- Keep track of all notifications receivers of an integration.Create an IntegrationNow go to the admin, you should now see the models registered there.
Go to integrations and create at least one integration. Current supported integrations:Slack;
Uses the Slack App key to send messages through to it's API. Here's aguideon how to create your app and get the key. Scopes required:channels:read,chat:write. After you get this configured. Install the slack app into the workspace that should send messages. Later, allow/invite the bot to a conversation/channel. When creating theIntegrationon the admin, the slack integration supports both thenameandidof a channel conversation.
Thenameof the channel is the visible name that appear on slack, note that using the name will require that we fetch theidon the slack api.
Theidof the channel can be retrieved on Slack. Click with the mouse's right button and click onCopy->Copy Link, theidis the last part of the URL.
The same applies for a user.Discord;
Uses a Discord Bot account to send messages through it's API. Here's aguideon how to create your Bot account. Required scopes:guilds,SEND_MESSAGES.
When creating theIntegrationon the admin, the discord integration supports both thenameandidof a channel conversation.
Thenameof the channel is the visible name that appear on discord, note that using the name will require that we fetch theidon the discord api. So we need to: Get your Botguilds, for eachguildget it'schannelsand lookup for a channel with the exact name that was provided. The message is sent to the exact first channel found on any guild, so use a specific name.
Theidof the channel can be retrieved on Discord. Click with the mouse's right button and click onCopy Link, theidis the last part of the URL.
The same applies for a user.That's it! You're setup to keep track of your Requests. Also, take a look into the source codeexamplefolder, that contains a django-project that uses thedjapmpackage.Optional settingsdjango-apmhas a set of optional configurations, that can be set on your django settings module. Checkout them below:APM_DEFAULT_LOGGER_NAME: The default logger name that will be used if None is supplied to theapm_view_decorator. Defaults to:apm_api_view;APM_REQUEST_SAVE_HEADERS: Boolean that when set toTruewill save the request headers. Defaults toTrue;APM_REQUEST_SAVE_QUERY_PARAMETERS: Boolean that when set toTruewill save the request query parameters as json. Defaults toTrue;APM_REQUEST_SAVE_QUERY_STRING: Boolean that when set toTruewill save the request raw query string. Defaults toTrue;APM_NOTIFY_USING_CELERY: Boolean that when set toTruewill dispatch a celery task when notificating. Since the process of notificating Integration can take a long time, we suggest you to set this toTrue. Defaults toFalse.APM_NOTIFY_ON_DEBUG_TRUE: Boolean that when set toTruewill notify errors even whenDEBUG=True. Defaults toFalse.APM_USE_DATABASE: String representing a key to theDATABASESdjango setting. Defaults to the django-default "default". Useful for changing which database apm will use to store it's data.Storage considerationsSincedjapmuses the database to register, get metrics, this may lead to a lot of storage being used.djapmdoesn't do cleanups, if you use celery-beat it may be useful to have a scheduled task to do the cleanup. Most of the data displayed on the dashboard is from the last 7 days.You also may find useful to use a separate database for this metrics, errors. For that use theAPM_USE_DATABASEsetting.Ellapsed time considerationsThe ellapsed time displayed/registered is not precise from what your clients may be having. Since we only start keeping track of the time when the request first enters theApmMetricsMiddleware.Project FutureInclude support for ViewSets.