Package: djangoql
Advanced search language for Django, with auto-completion. Supports logical
operators, parenthesis, table joins, and works with any Django model. Tested on
Python 2.7, 3.5 - 3.11, Django 1.8 - 4.2. The auto-completion feature has been
tested in Chrome, Firefox, Safari, IE9+.See a video:DjangoQL demoDjangoQL is used by:Is your project using DjangoQL? Please submit a PR and let us know!ContentsInstallationAdd it to your Django adminUsing DjangoQL with the standard Django admin searchLanguage referenceDjangoQL SchemaCustom search fieldsCan I use it outside of Django admin?Using completion widget outside of Django adminInstallation$pipinstalldjangoqlAdd'djangoql'toINSTALLED_APPSin yoursettings.py:INSTALLED_APPS=[...'djangoql',...]Add it to your Django adminAddingDjangoQLSearchMixinto your model admin will replace the standard
Django search functionality with DjangoQL search. Example:fromdjango.contribimportadminfromdjangoql.adminimportDjangoQLSearchMixinfrom.modelsimportBook@admin.register(Book)classBookAdmin(DjangoQLSearchMixin,admin.ModelAdmin):passUsing DjangoQL with the standard Django admin searchDjangoQL will recognize if you have definedsearch_fieldsin your ModelAdmin
class, and doing so will allow you to choose between an advanced search with
DjangoQL and a standard Django search (as specified by search fields). Example:@admin.register(Book)classBookAdmin(DjangoQLSearchMixin,admin.ModelAdmin):search_fields=('title','author__name')For the example above, a checkbox that controls search mode will appear near
the search input. If the checkbox is on, then DjanqoQL search is used. There is
also an option that controls if that checkbox is enabled by default -djangoql_completion_enabled_by_default(set toTrueby default):@admin.register(Book)classBookAdmin(DjangoQLSearchMixin,admin.ModelAdmin):search_fields=('title','author__name')djangoql_completion_enabled_by_default=FalseIf you don’t want two search modes, simply removesearch_fieldsfrom your
ModelAdmin class.Language referenceDjangoQL is shipped with comprehensive Syntax Help, which can be found in Django
admin (see the Syntax Help link in auto-completion popup). Here’s a quick
summary:DjangoQL’s syntax resembles Python’s, with some minor
differences. Basically you just reference model fields as you would
in Python code, then apply comparison and logical operators and
parenthesis. DjangoQL is case-sensitive.model fields: exactly as they are defined in Python code. Access
nested properties via., for exampleauthor.last_name;strings must be double-quoted. Single quotes are not supported.
To escape a double quote use\";boolean and null values:True,False,None. Please note
that they can be combined only with equality operators, so you can
writepublished = False or date_published = None, butpublished > Falsewill cause an error;logical operators:and,or;comparison operators:=,!=,<,<=,>,>=- work as you expect;string-specific comparison operators:startswith,not startswith,endswith,not endswith- work as you expect. Test whether or not a
string contains a substring:~and!~(translated into__icontainsunder the hood).
Example:name endswith "peace" or author.last_name ~ "tolstoy";date-specific comparison operators, compare by date part:~and!~.
Example:date_published ~"2021-11"- find books published in Nov, 2021;test a value vs. list:in,not in. Example:pk in (2, 3).DjangoQL SchemaSchema defines limitations - what you can do with a DjangoQL query.
If you don’t specify any schema, DjangoQL will provide a default
schema for you. This will walk recursively through all model fields and
relations and include everything it finds in the schema, so
users would be able to search through everything. Sometimes
this is not what you want, either due to DB performance or security
concerns. If you’d like to limit search models or fields, you should
define a schema. Here’s an example:classUserQLSchema(DjangoQLSchema):exclude=(Book,)suggest_options={Group:['name'],}defget_fields(self,model):ifmodel==Group:return['name']returnsuper(UserQLSchema,self).get_fields(model)@admin.register(User)classCustomUserAdmin(DjangoQLSearchMixin,UserAdmin):djangoql_schema=UserQLSchemaIn the example above we created a schema that does 3 things:excludes the Book model from search viaexcludeoption. Instead ofexcludeyou may also useinclude, which limits a search to
listed models only;limits available search fields for Group model to only thenamefield
, in the.get_fields()method;enables completion options for Group names viasuggest_options.An important note aboutsuggest_options: it looks for thechoicesmodel
field parameter first, and if it’s not specified - it will synchronously pull
all values for given model fields, so you should avoid large querysets there.
If you’d like to define custom suggestion options, see below.Custom search fieldsDeeper search customization can be achieved with custom search fields. Custom
search fields can be used to search by annotations, define custom suggestion
options, or define fully custom search logic. Indjangoql.schema, DjangoQL
defines the following base field classes that you may
subclass to define your own behavior:IntFieldFloatFieldStrFieldBoolFieldDateFieldDateTimeFieldRelationFieldHere are examples for common use cases:Search by queryset annotations:fromdjangoql.schemaimportDjangoQLSchema,IntFieldclassUserQLSchema(DjangoQLSchema):defget_fields(self,model):fields=super(UserQLSchema,self).get_fields(model)ifmodel==User:fields+=[IntField(name='groups_count')]returnfields@admin.register(User)classCustomUserAdmin(DjangoQLSearchMixin,UserAdmin):djangoql_schema=UserQLSchemadefget_queryset(self,request):qs=super(CustomUserAdmin,self).get_queryset(request)returnqs.annotate(groups_count=Count('groups'))Let’s take a closer look at what’s happening in the example above. First, we
addgroups_countannotation to the queryset that is used by Django admin
in theCustomUserAdmin.get_queryset()method. It would contain the number
of groups a user belongs to. As our queryset now pulls this column, we can
filter by it. It just needs to be included in the schema. InUserQLSchema.get_fields()we define a custom integer search field for theUsermodel. Its name should match the name of the column in our queryset.Custom suggestion optionsfromdjangoql.schemaimportDjangoQLSchema,StrFieldclassGroupNameField(StrField):model=Groupname='name'suggest_options=Truedefget_options(self,search):returnsuper(GroupNameField,self)\.get_options(search)\.annotate(users_count=Count('user'))\.order_by('-users_count')classUserQLSchema(DjangoQLSchema):defget_fields(self,model):ifmodel==Group:return['id',GroupNameField()]returnsuper(UserQLSchema,self).get_fields(model)@admin.register(User)classCustomUserAdmin(DjangoQLSearchMixin,UserAdmin):djangoql_schema=UserQLSchemaIn this example we’ve defined a custom GroupNameField that sorts suggestions
for group names by popularity (no. of users in a group) instead of default
alphabetical sorting.Custom search lookupDjangoQL base fields provide two basic methods that you can override to
substitute either search column, search value, or both -.get_lookup_name()and.get_lookup_value(value):classUserDateJoinedYear(IntField):name='date_joined_year'defget_lookup_name(self):return'date_joined__year'classUserQLSchema(DjangoQLSchema):defget_fields(self,model):fields=super(UserQLSchema,self).get_fields(model)ifmodel==User:fields+=[UserDateJoinedYear()]returnfields@admin.register(User)classCustomUserAdmin(DjangoQLSearchMixin,UserAdmin):djangoql_schema=UserQLSchemaIn this example we’ve defined the customdate_joined_yearsearch field for
users, and used the built-in Django__yearfilter option in.get_lookup_name()to filter by date year only. Similarly you can use.get_lookup_value(value)hook to modify a search value before it’s used in
the filter.Fully custom search lookup.get_lookup_name()and.get_lookup_value(value)hooks cover many
simple use cases, but sometimes they’re not enough and you want a fully custom
search logic. In such cases you can override main.get_lookup()method of
a field. Example below demonstrates Useragesearch:fromdjangoql.schemaimportDjangoQLSchema,IntFieldclassUserAgeField(IntField):"""
    Search by given number of full years
    """model=Username='age'defget_lookup_name(self):"""
        We'll be doing comparisons vs. this model field
        """return'date_joined'defget_lookup(self,path,operator,value):"""
        The lookup should support with all operators compatible with IntField
        """ifoperator=='in':result=Noneforyearinvalue:condition=self.get_lookup(path,'=',year)result=conditionifresultisNoneelseresult|conditionreturnresultelifoperator=='not in':result=Noneforyearinvalue:condition=self.get_lookup(path,'!=',year)result=conditionifresultisNoneelseresult&conditionreturnresultvalue=self.get_lookup_value(value)search_field='__'.join(path+[self.get_lookup_name()])year_start=self.years_ago(value+1)year_end=self.years_ago(value)ifoperator=='=':return(Q(**{'%s__gt'%search_field:year_start})&Q(**{'%s__lte'%search_field:year_end}))elifoperator=='!=':return(Q(**{'%s__lte'%search_field:year_start})|Q(**{'%s__gt'%search_field:year_end}))elifoperator=='>':returnQ(**{'%s__lt'%search_field:year_start})elifoperator=='>=':returnQ(**{'%s__lte'%search_field:year_end})elifoperator=='<':returnQ(**{'%s__gt'%search_field:year_end})elifoperator=='<=':returnQ(**{'%s__gte'%search_field:year_start})defyears_ago(self,n):timestamp=now()try:returntimestamp.replace(year=timestamp.year-n)exceptValueError:# February 29returntimestamp.replace(month=2,day=28,year=timestamp.year-n)classUserQLSchema(DjangoQLSchema):defget_fields(self,model):fields=super(UserQLSchema,self).get_fields(model)ifmodel==User:fields+=[UserAgeField()]returnfields@admin.register(User)classCustomUserAdmin(DjangoQLSearchMixin,UserAdmin):djangoql_schema=UserQLSchemaCan I use it outside of Django admin?Sure. You can add DjangoQL search functionality to any Django model usingDjangoQLQuerySet:fromdjango.dbimportmodelsfromdjangoql.querysetimportDjangoQLQuerySetclassBook(models.Model):name=models.CharField(max_length=255)author=models.ForeignKey('auth.User')objects=DjangoQLQuerySet.as_manager()With the example above you can perform a search like this:qs=Book.objects.djangoql('name ~ "war" and author.last_name = "Tolstoy"')It returns a normal queryset, so you can extend it and reuse if
necessary. The following code works fine:print(qs.count())Alternatively you can add DjangoQL search to any existing queryset,
even if it’s not an instance of DjangoQLQuerySet:fromdjango.contrib.auth.modelsimportUserfromdjangoql.querysetimportapply_searchqs=User.objects.all()qs=apply_search(qs,'groups = None')print(qs.exists())Schemas can be specified either as a queryset option, or passed
to.djangoql()queryset method directly:classBookQuerySet(DjangoQLQuerySet):djangoql_schema=BookSchemaclassBook(models.Model):...objects=BookQuerySet.as_manager()# Now, Book.objects.djangoql() will use BookSchema by default:Book.objects.djangoql('name ~ "Peace")  # uses BookSchema# Overriding default queryset schema with AnotherSchema:Book.objects.djangoql('name ~ "Peace", schema=AnotherSchema)You can also provide schema as an option forapply_search()qs=User.objects.all()qs=apply_search(qs,'groups = None',schema=CustomSchema)Using completion widget outside of Django adminThe completion widget is not tightly coupled to Django admin, so you can easily
use it outside of the admin if you want. The widget isavailable on npmas a
standalone package.
See the source code and the docs in thedjangoql-completionrepo on GitHub.The completion widget is also bundled with thedjangoqlPython package on PyPI. If
you’re not using Webpack or another JavaScript bundler, you can use the
pre-built version that ships with the Python package. Here is an example:Template code,completion_demo.html:{% load static %}<!DOCTYPE html><htmllang="en"><head><metacharset="UTF-8"><title>DjangoQL completion demo</title><linkrel="stylesheet"type="text/css"href="{% static 'djangoql/css/completion.css' %}"/><scriptsrc="{% static 'djangoql/js/completion.js' %}"></script></head><body><formaction=""method="get"><pstyle="color: red">{{ error }}</p><textareaname="q"cols="40"rows="1"autofocus>{{ q }}</textarea></form><ul>{% for item in search_results %}<li>{{ item }}</li>{% endfor %}</ul><script>DjangoQL.DOMReady(function(){newDjangoQL({// either JS object with a result of DjangoQLSchema(MyModel).as_dict(),// or an URL from which this information could be loaded asynchronouslyintrospections:{{introspections|safe}},// css selector for query input or HTMLElement object.// It should be a textareaselector:'textarea[name=q]',// optional, you can provide URL for Syntax Help link here.// If not specified, Syntax Help link will be hidden.syntaxHelp:null,// optional, enable textarea auto-resize feature. If enabled,// textarea will automatically grow its height when entered text// doesn't fit, and shrink back when text is removed. The purpose// of this is to see full search query without scrolling, could be// helpful for really long queries.autoResize:true});});</script></body></html>And in yourviews.py:importjsonfromdjango.contrib.auth.modelsimportGroup,Userfromdjango.shortcutsimportrender_to_responsefromdjango.views.decorators.httpimportrequire_GETfromdjangoql.exceptionsimportDjangoQLErrorfromdjangoql.querysetimportapply_searchfromdjangoql.schemaimportDjangoQLSchemafromdjangoql.serializersimportDjangoQLSchemaSerializerclassUserQLSchema(DjangoQLSchema):include=(User,Group)suggest_options={Group:['name'],}@require_GETdefcompletion_demo(request):q=request.GET.get('q','')error=''query=User.objects.all().order_by('username')ifq:try:query=apply_search(query,q,schema=UserQLSchema)exceptDjangoQLErrorase:query=query.none()error=str(e)# You may want to use SuggestionsAPISerializer and an additional API# endpoint (see in djangoql.views) for asynchronous suggestions loadingintrospections=DjangoQLSchemaSerializer().serialize(UserQLSchema(query.model),)returnrender_to_response('completion_demo.html',{'q':q,'error':error,'search_results':query,'introspections':json.dumps(introspections),})LicenseMIT