Package: django-admin-cursor-paginator
django-admin-cursor-paginatorDrop-in replacement for django admin default pagination that works fast with huge tables.PrerequisitesYou are using django (>2.0) adminSome of your models/tables contains so much data that list page lagsYou don't want to make separate CRM page for them and just need fast and simple solutionYou're ok that you won't see total objects or pages count and will navigate only by +-1 pageIndex must be created for the ordering field, otherwise you won't get any significant speedup
(default integer pks work perfectly out-of-box)Ordering field type can be almost anything that can be "encoded" viastr()and then be used inqueryset.filter()NOTEIf ordering field is not unique then pagination can mess a bit in edge cases.
But for the most often case with datetime fields with milliseconds likecreated_atit won't be a problem.
It can be solved in future by adding support for multiple ordering fields.InstallationAll you need is to install it viapiplike that:pipinstalldjango-admin-cursor-paginator...and extendINSTALLED_APPS(it is needed only for templates):INSTALLED_APPS=['django.contrib.admin',# ...'admin_cursor_paginator',# ...]NOTEThis app overrides{% block pagination %}in defaultchange_list.htmlto insert custom pagination template tag, so if you use patchedchange_list.htmltoo,
don't forget to copy pagination block or (seems better) to inherit this app tpl like{% extends 'admin_cursor_paginator/change_list.html' %}.Also, there's a knownissue with grapellithat is fixed the same way.Usage exampleSo this app will certainly help you if you're using admin to see some (usually) last inserted entries
and may be also want to search for particular objects.But anyways it's just a regular django admin page.fromadmin_cursor_paginatorimportCursorPaginatorAdminclassYourModelAdmin(CursorPaginatorAdmin):cursor_ordering_field='-created_at'# default is '-pk'classAnotherModelAdmin(CursorPaginatorAdmin):cursor_ordering_field='parent__sort'# joins are supported as wellThe problemDefault django admin list view makes several queries to show data.
The first one is something likeselect count(*) from app_modelto calculate pages range
andselect * from app_model limit 10 offset 1000to fetch current page data.
Both of them start to produce performance problems as your table grows.For example, with >1M entries in postgresql tablecount(*)took a few seconds (in our case) because of seq scan.
Limit+offset also performs badly on far pages.The solutionThe solution is to use cursor pagination instead of default one (you can find many standalone packages for it).
It doesn't needcount(*)query at all and performs fast by using filter instead of limit+offset.Some useful links:https://cra.mr/2011/03/08/building-cursors-for-the-disqus-api/https://www.django-rest-framework.org/api-guide/pagination/#cursorpaginationhttps://hakibenita.com/optimizing-the-django-admin-paginatorTest projectIntestprojectfolder you can (surprisingly) find test project that uses this app
for one model so that you can see live example and check how it works if you want.All you need is to install requirements fromrequirements.txtin some env,
run./manage.py runserver, login as admin:admin to django admin and openProductlist.Project includes sqlite3 file with some test data and preinstalleddebug_toolbarto observe db queries.You can play with ordering field or data amount or may be test it in your target db.Example of how pagination will look like: