Package: django-elastic-appsearch
Integrate your Django Project with Elastic App Search with ease.DocumentationThe full documentation is athttps://django_elastic_appsearch.readthedocs.io. Read our step-by-step guide on integrating App Search with your existing Django project over atMedium.DependenciesPython >= 3.6Django >= 2.2elastic-enterprise-searchserpyUsageInstallingInstall Django Elastic App Search:pip install django_elastic_appsearchAdd it to yourINSTALLED_APPS:INSTALLED_APPS=(...'django_elastic_appsearch',...)Add the Elastic App Search URL and Key to your settings module:APPSEARCH_HOST='localhost:3002'APPSEARCH_API_KEY='some_appsearch_api_token'Configuring app search indexable modelsSingle engineConfigure the Django models you want to index to Elastic App Search. To index to one engine you can do this by inheriting from theAppSearchModel, and then setting some meta options.AppsearchMeta.appsearch_engine_name- Defines which engine in your app search instance your model will be indexed to.AppsearchMeta.appsearch_serialiser_class- Defines how your model object will be serialised when sent to your elastic app search instance. The serialiser and fields used here derives fromSerpy, and you can use any of the serpy features like method fields.Example:fromdjango_elastic_appsearch.ormimportAppSearchModelfromdjango_elastic_appsearchimportserialisersclassCarSerialiser(serialisers.AppSearchSerialiser):full_name=serialisers.MethodField()make=serialisers.StrField()model=serialisers.StrField()manufactured_year=serialisers.Field()defget_full_name(self,instance):return'{}{}'.format(make,model)classCar(AppSearchModel):classAppsearchMeta:appsearch_engine_name='cars'appsearch_serialiser_class=CarSerialisermake=models.CharField(max_length=100)model=models.CharField(max_length=100)manufactured_year=models.CharField(max_length=4)Multi engineConfigure the Django models you want to index to Elastic App Search. To index to multiple engines you can do this by inheriting from theAppSearchMultiEngineModel,
and then setting a meta option.AppsearchMeta.appsearch_serialiser_engine_pairs- A list of tuples of serialisers then engines that define which engine in your app search instance your model will
be indexed to and how your model object will be serialised when sent to your elastic app search instance. The serialiser and fields used here derives fromSerpy, and you can use any of the serpy features like method fields.Example:fromdjango_elastic_appsearch.ormimportAppSearchModelfromdjango_elastic_appsearchimportserialisersclassCarSerialiser(serialisers.AppSearchSerialiser):full_name=serialisers.MethodField()make=serialisers.StrField()model=serialisers.StrField()manufactured_year=serialisers.Field()defget_full_name(self,instance):return'{}{}'.format(make,model)classTruck(AppSearchMultiEngineModel):"""A truck."""classAppsearchMeta:appsearch_serialiser_engine_pairs=[(CarSerialiser,"trucks")]make=models.TextField()model=models.TextField()year_manufactured=models.DateTimeField()Using model and queryset methods to index and delete documentsThen you can callindex_to_appsearchanddelete_from_appsearchfrom your model objects.Send the car with id 25 to app search.frommymodelsimportCarcar=Car.objects.get(id=25)car.index_to_appsearch()Delete the car with id 21 from app search.frommymodelsimportCarcar=Car.objects.get(id=21)car.delete_from_appsearch()Calling these on anAppSearchModelwill return a single response object, and calling them on anAppSearchMultiEngineModelwill return a list of response objects.You can also callindex_to_appsearchanddelete_from_appsearchon QuerySets ofAppSearchModelSend all cars where the make is ‘Toyota’ to app search.cars=Car.objects.filter(make='Toyota')cars.index_to_appsearch()Delete all cars where the make is ‘Saab’ from app searchcars=Car.objects.filter(make='Saab')cars.delete_from_appsearch()index_to_appsearchmethods on the QuerySet and your model also supports an optionalupdate_onlyparameter which takes in a boolean value. Ifupdate_onlyis set toTrue, the operation on the app search instance will be carried out as aPATCHoperation. This will be useful if your Django application is only doing partial updates to the documents.This will also mean that your serialisers can contain a subset of the fields for a document. This will be useful when two or more Django models or applications are using the same app search engine to update different sets of fields on a single document type.Example below (Continued from the aboveCarexample):fromdjango.dbimportmodelsfromdjango_elastic_appsearch.ormimportAppSearchModelfromdjango_elastic_appsearchimportserialisersclassCarVINNumberSerialiser(serialisers.AppSearchSerialiser):vin_number=serialisers.StrField()classCarVINNumber(AppSearchModel):classAppsearchMeta:appsearch_engine_name='cars'appsearch_serialiser_class=CarVINNumberSerialisercar=models.OneToOneField(Car,on_delete=models.CASCADE,primary_key=True)vin_number=models.CharField(max_length=100)defget_appsearch_document_id(self):return'Car_{}'.format(self.car.id)frommymodelsimportCarVINNumbercar_vin=CarVINNumber.objects.filter('car__id'=25).first()car_vin.vin_number='1M8GDM9A_KP042788'car_vin.save()car_vin.refresh_from_db()car_vin.index_to_appsearch(update_only=True)You’ll notice that we’ve set theappsearch_engine_nametocarsso that the VIN number updates will go through to the same engine. You’ll also notice that we’ve overridden theget_appsearch_document_idmethod to make sure that VIN number updates do go through the same related car document.The above example will update the car document with id 25 with the new VIN number and leave the data for the rest of the fields intact.Important note:PATCHoperations on Elastic App Search cannot create new schema fields if you submit schema fields currently unknown to your engine. So always make sure you’re submitting values for existing schema fields on your engine.Use with your own custom queryset managersIf you want to specify custom managers which also has this functionality, you can inherit fromdjango_elastic_appsearch.orm.AppSearchQuerySetfromdjango_elastic_appsearch.ormimportAppSearchModel,AppSearchQuerySetclassMyCustomQuerySetManager(AppSearchQuerySet):defmy_custom_queryset_feature(self):# Do Something coolpassclassMyCustomModel(AppSearchModel):field_1=models.CharField(max_length=100)# Set the custom managerobjects=MyCustomQuerySetManager.as_manager()Use a custom document id for appsearchBy default, the unique document ID which identifies your model objects in app search is set to<model_name>_<object_id>. If we take the car example above, aCarobject with an id of543will have the document IDCar_543in app search.You can customise this value by overriding theget_appsearch_document_idmethod on your model class.Eg. You can do the following to make sure that the document ID on appsearch is exactly the same as the ID on your model object.classCar(AppSearchModel):classAppsearchMeta:appsearch_engine_name='cars'appsearch_serialiser_class=CarSerialisermake=models.CharField(max_length=100)model=models.CharField(max_length=100)manufactured_year=models.CharField(max_length=4)defget_appsearch_document_id(self):returnself.idSettingsThis package provides various Django settings entries you can use to configure your connection to the Elastic App Search instance you’re using.APPSEARCH_HOSTRequired: YesDefault: No default valueThis is arequiredsetting to tell your Django application which Elastic App Search instance to connect with.APPSEARCH_HOST='localhost:3002'APPSEARCH_API_KEYRequired: YesDefault: No default valueThis is arequiredsetting to tell your Django application the private key to use to talk to your Elastic App Search instance.APPSEARCH_API_KEY='private-key'APPSEARCH_USE_HTTPSRequired: NoDefault:TrueThis is anoptionalsetting to configure whether to use HTTPS or not when your Django application communicates with your Elastic App Search instances. It defaults toTrueif it’s not set. This might be useful when you’re running your Django project against a local Elastic App Search instance. It’s insecure to have this asFalsein a production environment, so make sure to change toTruein your production version.APPSEARCH_USE_HTTPS=FalseAPPSEARCH_CHUNK_SIZERequired: NoDefault:100This is anoptionalsetting to configure the chunk size when doing queryset indexing/deleting. Elastic App Search supports upto a 100 documents in one index/destroy request. With this setting, you can change it to your liking. It defaults to the maximum of100when this is not set. This might be useful when you want to reduce the size of a request to your Elastic App Search instance when your documents have a lot of fields/data.APPSEARCH_CHUNK_SIZE=50APPSEARCH_INDEXING_ENABLEDRequired: NoDefault:TrueThis is anoptionalsetting to configure if you want to disable indexing to your Elastic App Search instance. This is useful when you want to disable indexing without changing any code. When it’s set toFalse, any code where you useindex_to_appsearch()ordelete_from_appsearch()will not do anything. It’s set toTrueby default when it’s not set.APPSEARCH_INDEXING_ENABLED=TrueExample with all settings entriesAPPSEARCH_HOST='localhost:3002'APPSEARCH_API_KEY='private-key'APPSEARCH_USE_HTTPS=FalseAPPSEARCH_CHUNK_SIZE=50APPSEARCH_INDEXING_ENABLED=TrueWriting TestsThis package provides a test case mixin calledMockedAppSearchTestCasewhich makes it easier for you to write test cases againstAppSearchModel’s andAppSearchMultiEngineModel’s without actually having to run an Elastic App Search instance during tests.All you have to do is inherit the mixin, and all the calls to Elastic App Search will be mocked. Example below.fromdjango.testimportTestCasefromdjango_elastic_appsearch.testimportMockedAppSearchTestCasefrommyapp.test.factoriesimportCarFactoryclassBookTestCase(MockedAppSearchTestCase,TestCase):deftest_indexing_book(self):car=CarFactory()car.save()car.index_to_appsearch()self.assertAppSearchModelIndexCallCount(1)You will have access to the following methods to check call counts to different mocked app search methods.self.assertAppSearchQuerySetIndexCallCount— Check the number of times index_to_appsearch was called on a appsearch model querysets.self.assertAppSearchQuerySetDeleteCallCount— Check the number of times delete_from_appsearch was called on an appsearch model querysets.self.assertAppSearchModelIndexCallCount— Check the number of times index_to_appsearch was called on an appsearch model objects.self.assertAppSearchModelDeleteCallCount— Check the number of times delete_from_appsearch was called on an appsearch model objects.If you are using a subclass ofAppSearchQuerySetthat overrides methods without calling the super class version you can use thequeryset_classkey word argument to thesetUpfunction to mock it. Example below.fromdjango.testimportTestCasefromdjango_elastic_appsearch.testimportMockedAppSearchTestCaseclassBusTestCase(MockedAppSearchTestCase,TestCase):"""Test the `MockedAppSearchTestCase`."""defsetUp(self,*args,**kwargs):"""Load test data."""kwargs['queryset_class']='example.querysets.CustomQuerySet.'super().setUp(*args,**kwargs)Using the elastic app search python clientWe use the officialelastic app search python clientunder the hood to communicate with the app search instance. So if needed, you can access the app search instance directly and use the functionality of the official elastic app searchclient. Example below.fromdjango_elastic_appsearch.clientsimportget_api_v1_enterprise_search_clientclient=get_api_v1_enterprise_search_client()client.search('cars','Toyota Corolla',{})ContributingContributors are welcome!Prior to opening a pull request, please create an issue to discuss the change/feature you’ve written/thinking of writing if it doesn’t already exist.Please write simple code and concise documentation, when appropriate.Please write test cases to cover the code you’ve written, where possible.Read theContributingsection of our documentation for more information around contributing to this project.Running TestsDoes the code actually work?$ pipenv install --dev
$ pipenv shell
(django_elastic_appsearch) $ toxCreditsTools used in rendering this package:Cookiecuttercookiecutter-djangopackageHistory1.1.5 (2021-05-11)Transfer ownership to InfoxchangeUpdate documentationDependency upgrades1.1.4 (2021-05-05)Updated documentationDependency upgrades1.1.3 (2021-04-12)Add Django 3.2 to the test matrixDependency upgradesMinor fixes1.1.2 (2021-02-11)Security patch1.1.1 (2021-02-04)Dependency upgradesSecurity patchImprove returned responses1.1.0 (2021-01-27)Dependecy upgradesAdd ability to index a model object into multiple app search engines1.0.2 (2020-12-29)Dependency upgrades1.0.1 (2020-12-15)Dependency upgrades1.0.0 (2020-11-26)Python 3.9 supportUpdate the project status to stable0.7.6 (2020-11-26)Dependency upgradesUpdate documentation0.7.5 (2020-10-28)Security patch0.7.4 (2020-10-05)Add support for testing overridden queryset methodsUpdate documentation0.7.3 (2020-08-25)Remove support for Django 2.0 and Django 2.1Add support for Django 3.1Update documentation0.7.2 (2020-08-25)Dependency upgrades0.7.1 (2020-07-31)Dependency upgrades0.7.0 (2020-07-30)Implement ability to do partial updates to documentsDependency upgrades0.6.11 (2020-06-22)Fix failing dependency check with pipenvDependency upgrades0.6.9 (2020-05-15)Dependency upgrades0.6.8 (2020-03-31)Dependency upgradesSecurity patches0.6.7 (2020-02-25)Dependency upgrades0.6.6 (2020-02-01)Dependency upgrades0.6.3 (2020-01-03)Move from Travis CI to Github ActionsDocumentation updates0.6.2 (2020-01-02)Dependency upgradesDocumentation improvementsAdd linting for CISetup automatic PyPI releases0.6.1 (2019-12-24)Dependency upgrades0.6.0 (2019-12-04)Remove support for Python 3.5Add support for Python 3.8Add support for Django 3Dependency upgradesBump development status to Beta0.5.6 (2019-12-03)Dependency upgrades0.5.5 (2019-11-14)Dependency upgrades0.5.4 (2019-10-02)Dependency upgrades0.5.3 (2019-08-28)Improve documentationRefactor settings nameAPPSEARCH_URL->APPSEARCH_HOST0.5.1 (2019-08-26)Improve test coverageImprove documentationAdd serpy as an official dependencyBump dependency versionsAdd code of conduct0.4.2 (2019-08-16)Switch to the new official Elastic App Search python clientDocumentation improvements0.2.3 (2019-08-02)Use Pipenv for dependency managementConfigure Dependabot for automatic dependency upgradesRemove support for Python 3.4Documentation improvements0.2.2 (2019-07-29)Bug fixesDocumentation improvements0.1.0 (2019-07-26)First release on PyPI.