Package: odoo-addon-connector-search-engine
Base module for connecting Odoo with external search engines. This addon is
intended to be used as a base for other addons that implement specific search
engines. It’s designed to be easily extensible and modular.Table of contentsInstallationUsageOverviewIndexing lifecycleKnown issues / RoadmapChangelog16.0.0.1.7 (2023-12-15)16.0.0.1.4 (2023-11-29)16.0.0.1.2 (2023-11-28)16.0.0.1.1 (2023-10-13)16.0.0.1.0 (2023-10-13)12.0.x.y.z (YYYY-MM-DD)Bug TrackerCreditsAuthorsContributorsMaintainersInstallationThis addon uses the native json python package provided by python. When
a json for a record is recomputed, the new value is compared to the original
one to see if an export to the search engine index is needed.  This is
done by comparing the md5 of the two json strings. This process when done on
a large number of records can be slow when the json is large and complex. To speed
up this process you can install the orjson package.pipinstallorjsonUsageOverviewA search engine is a system designed to store information in a way that makes
it easy to find through search and analytics queries. The main difference
between a search engine and a database is that a search engine is optimized
for search and analytics queries, while a database is optimized for
transactional and relational queries.This addons is designed around 4 main concepts:The search engine backendis used to define into Odoo the kind
of search engine that will be used to index the data. It’s main responsibility
is to provide an instance ofodoo.addons.search_engine.tools.adapter.SearchEngineAdapterthat will be used to communicate with the search engine.The search engine indexis used to define into Odoo the index where
the data will be indexed. An index is always linked to a search engine backend.
The index provides methods to use to manage the lifecycle of the data put into
the index for the records of a given model. To do so, it uses:The SearchEngineAdapterprovided by the backend to communicate with the
search engine.A ModelSerializerthat is used to transform an odoo record into
a dictionary that can be indexed into the search engine.A JsonValidatorthat is used to validate the data that is to be
indexed into the search engine.The RecordSerializer and IndexDataValidator are defined on the index itself.
The current addon provides a default implementation only for the IndexDataValidator.
You can find into the github repositorysearch-engineAn implementation of the RecordSerializer based
on the jsonifier addonconnector_search_engine_jsonifier.The search engine indexable recordis a mixin that is used to define
the records that can be indexed into a search engine index. The mixin
provides methods:To add a record to an index.To remove a record from an index.To mark the record into an index (the search engine bindings) as to be
recomputed (This method should be called when modifications are made on
the record that could impact the data that are indexed into the search
engine. It will instruct the index that the record must be recomputed and
re-indexed).It also ensures that when the record is unlinked, it is removed from the indexes
it was indexed into.The search engine bindingis a model that represents the link between
an index and an indexable odoo record. It give you access to the data
that are indexed into the search engine for the record. It’s also used to
manage the lifecycle of the data into the search engine. When a binding is
created, it’s marked as to be computed. Once the data are computed, the
binding is marked as to be indexed. Once the data are indexed, the binding
is marked as indexed. If the linked record is unlinked, the binding is
marked as to be removed. Once the data are removed from the search engine,
the binding is deleted.Indexing lifecycleThe indexing lifecycle is based on the following steps:When a record is added to an index, a binding is created and marked as to be
computed.A cron job scheduled every 5 minutes will look for bindings that are to be
computed and for each of them will schedule a job to re compute the json data.When the json data is computed, the binding is marked as to be exported if the
json is valid and is different from the one that has been computed last time.A cron job scheduled every 5 minutes will ensure the syncing with the search
engine. It will:look for bindings that are to be exported and for each of them will schedule
a job to export the json data into the search engine. Once exported, the
binding is marked as ‘done’.look for bindings that are to be removed and for each of them will schedule
a job to remove the data from the search engine. Once removed, the binding
is deleted.To keep in sync the data from your model instance and the data that are indexed
into the search engine, you should call the method_se_mark_to_updateon the
mode instance when you make modifications that could impact the data that are
indexed into the search engine.When the method_se_mark_to_updateis called, the binding is marked as to be
computed.From there, the same process as described above will be used to recompute the
data and reindex them into the search engine.When a model instance is unlinked, the binding is marked as to be removed. From
there if will be processed by the job syncing the data with the search engine.NoteIn previous versions of this addon, there was no method to mark a record as
to be recomputed. As a consequence, all the records were re-computed every day
to ensure that the data in the search engine were up to date. This was a
performance issue and consumed a lot of resources. If despite this, you want
to recompute all the records every day, you can activate the cron jonSearch engine: recompute all indexand deactivate the one namedearch engine: Generate job for recompute binding to recompute per index.Known issues / RoadmapImplement generic trigger for binding
based on ir.export linked to the index
(the aim is to set the binding to be updated
if we modify a field configured in the exporter)Changelog16.0.0.1.7 (2023-12-15)BugfixesEnsure that the record’s model is compatible with the index’s model before
adding a new record to the index.Before this change, the index would silently ignore records that were not
compatible with the index’s model. This could lead to unexpected behavior and
errors when the record was later used to be serialized to JSON and exported to
a search engine. (#177)Lower memory consumption by disabling prefetch for the field ‘data’ on the binding model.The field ‘data’ is a json field that is not used in the view or common management
operations of the binding model. This json field can be very large. By disabling
the prefetch, we avoid to overload the database and Odoo with useless data. (#179)16.0.0.1.4 (2023-11-29)BugfixesFix error when calling the methodsexport_recordordelete_recordfrom
these.bindingmodel when called on a recordset with items from differentse.backend.Theexportanddeletemethods involves the use of aBackend Adapterto
communicate with the target search engine. We then need to process the bindings
by backend to call the correct adapter and ensure at same time a batch process
of the requested operation for all the records linked to the same backend. (#173)16.0.0.1.2 (2023-11-28)BugfixesAdd missing description on the “se.binding.state.updater” model. As well as
ensuring consistency in the model definition, this change removes a
warning message from the server logs at registry load time.Prevent warning message in server logs when running tests. (#172)16.0.0.1.1 (2023-10-13)BugfixesFixes cache issue with these_binding_idsfield on thes.indexable.recordmodel. When a binding is created or updated or deleted, the cache for these_binding_idsfield for referenced records is now invalidated. That way,
the next time the field is accessed after such an operation, the value is
recomputed to reflect the change. (#163)16.0.0.1.0 (2023-10-13)FeaturesA new actionUpdate stateis now available onSearch Engine Recordobjects.
This action allows you to update the state of selected records on the tree view.Add a smart button to quickly access to the bound records from theSearch Engine BackendandSearch Engine Recordviews. (#162)BugfixesFix Search Engine Binding form view. The fields data and error are now
properly displayed and fit the width of the form.Makes the Odoo’s admin user a member of theSearch Engine Connector Managergroup. (#162)12.0.x.y.z (YYYY-MM-DD)TODOBug TrackerBugs are tracked onGitHub Issues.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us to smash it by providing a detailed and welcomedfeedback.Do not contact contributors directly about support or help with technical issues.CreditsAuthorsAkretionACSONE SA/NVCamptocampContributorsSébastien BEAU <sebastien.beau@akretion.com>Laurent Mignon <laurent.mignon@acsone.eu>Simone Orsi <simone.orsi@camptocamp.com>Raphaël Reverdy <raphael.reverdy@akretion.com>MaintainersThis module is maintained by the OCA.OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.This module is part of theOCA/search-engineproject on GitHub.You are welcome to contribute. To learn how please visithttps://odoo-community.org/page/Contribute.