Package: zopyx.typesense
zopyx.typesenseWhat is zopyx.typesensezopyx.typesensein an add-on for Plone 6 that provides an integration with
the search engineTypesense. The functionality is
similar withcollective.solr.The reasons for using Typesense arevery easy to install (single binary or via Docker)multi-field text indexesauto-generated (but highly customizable) search UIoptional facted search for refining fulltext queriesminimal invasive into Plone (does not replace the ZCatalog)very fast searchextensible and customizablescalable (through a Typesense cluster)federated search over multiple collections (aka Plone sites)open-sourceon-premise or Typesense cloud (commercial offering)Clickherefor an intro video.zopyx.typesense is nota replacement of the ZCatalog/portal_catalog (document changes are pushed to
Typesense as an independent entity completely decoupled from Plone's
portal_catalog)a drop-in replacement of the Plone search.zopyx.typesensecomes with its own
configurable auto-generated search UIRequirementsPlone 6 (tested)Plone 5.2 (untested, supposed to work for Dexterity-only sites, under Python 3)ArchitectureTypesense runs as dedicated service either on-premise or somewhere in the cloud.![Architecture](https://github.com/zopyx/zopyx.typesense/raw/master/docs/Typesense.pngIndexingzopyx.typesensepushes content changes (new documents, updated documents, deleted documents) through
Plone lifecycle hooks into Typesense.SearchingThe search interaction between browser and Typesense happens directlywithoutPlone being in-between.  When you open the@@typesense-searchview, the
search configuration (as defined through the search templatesearch.ptand an
associatedapp.jsfile) is loaded from Plone for auto-generating the search
UI (search input, faceted search) within the browser. The search query is
directly send to Typesense and the results are directly shown within the
browser as soon as they are available.  Plone isnotinvolved in the search.
No additional filtering (e.g. regarding access control) happens through Plone.InstallationAddzopyx.typesenseto your buildout, re-run buildout and install it within Plone.For Typesense installation, please check the installation docs of Typesense
(either for installation through Docker or through the standalone binary).There is no public release at this time. You need to installzopyx.typesenseusingmr.developeras source checkout from Github.ConfigurationTheTypesense settingswithin the Plone controlpanel:Name of Typesense collection- must be a unique name for the document pool
of your Plone siteAPI Key- the administrative API key (as configured in Typesense)Search API Key- the search API key (as configured in Typesense)URL of Typesense node X- the URL(s) of the Typesense node or Typesense
clusterCollection schema- the schema of the Typesense collecton (see Typesense
docs)TheTypesense administrationwithin the Plone controlpanel:Example setupFor a typical Plone dev setup which Plone running on localhost, you want to run
Typesense (for dev purpose) on the same machine using Docker:docker run -p 8108:8108 -v /tmp/data:/data typesense/typesense:0.22.1 --data-dir /data --api-key=some_api_key --search-only-api-key=some_search_api_key --enable-corsThis command will store the Typesense data files under/tmp/dataand open a
connection on port8108.  You need to specify two api keys: one for admin
access and one for search api.After installingzopyx.typesensein Plone, you would specify both API keys in
the Typesense control panel within Plone along withhttp://localhost:8108as
URL for Typesense node 1.Search UIThe search UI is auto-generated from a minimal HTML template which defines the
basic layout together with the widgets to be usedsee
here.The integration of the template with the Typesense UI generator is impleted
through some lines of Javascript code insrc/zopyx/typesense/browser/static/app.js.Views@@typesense-searchrenders the main (auto-generated) search form. This view can be applied on
any folder level. This view applied on a folder implies a filter by subpath (only indexed content
with the given folder will be searched).@@typesense-settingsrenders the Plone control panel of Typesense@@typesense-settingsrenders the Plone control panel of Typesense@@typesense-adminrenders the administration control panel for Typesense@@typesense-indexed-contentapplied on any arbitrary content object will
display all data indexed for the given content objectIndexing integrationzopyx.typesenseintegrates into Plone using the lifecycle event for content
being added, updated, deleted (plus workflow changes). By default,zopyx.typesenseindexes all metadata of all content objects (basically Dublin
Core) plus the textual content of a content objects (title, description and the
content of allRichTextfields).  Similar to Plone'sSearchableTextindex,
all textual content is being indexed within Typesense in thetextfield.It is possible to override the indexing behavior per-content-type using an
indexing adapter implementingITypesenseIndexDataProvider. Seehere for an
example.Indexing binary content and office formatsThe indexer for theFilecontent-type has optional support for indexing
binary content or office formats using ApacheTIKA.
The server URL of the (optional) Tika server must be configured through theTypesensecontrol panel. The most simple approach for running Tika is by
using Docker:docker run -d -p 9998:9998 apache/tikaConfigure Tika within the Typesense control panel ashttp://localhost:9998.
Ensure that the configuration regarding host, IP address and port numbers fit
your security requirements (usually, you don't want expose an internal service
on public IP addresses, but localhost only).Bulk reindexingThe Typesense admin control panel contains a buttonReindex allwhich willcreate a new collection whose name is defined as <collection_name>-submit all Dexterity content objects for reindexingalias the new collection with the official name of the configured collectionThis means that an existing Typesense collection remains online until the
complete reindexing process has finished. The Typesense collection with the
fresh index content replaces then the old collection in an atomic operation.
There should be no impact on the availability of the search functionality
during reindexing.Custom search viewsThe default Typesense search view@@typesense-searchshould work for most
common usecases of a Plone site with the default content types. If you need
something different, you have two options:customizesearch.ptandapp.jsaccording to your own need (e.g. usingz3c.jbot)create your search view(s) with your own search template and the related JS
configurationTransactions and eventual consistencyAll indexing/unindexing operations happen asyncronously to Plone and outside
Plone's transaction system.  So, content changes might be available in
Typesense with a short delay.Cavecats and known issueszopyx.typesensedoes not integrate (by-design) with Plone's security and
access model.  The main purpose ofzopyx.typesenseis to act as a search
engine for public sites.  So it is recommended at this time to index only
public content.ResourcesTypesense websiteTypesense APIInstantSearch.jsAuthorAndreas Jung |info@zopyx.com|www.zopyx.comPaid service forzopyx.typesenseis available on request.