Package: wildcard.hps
wildcard.hpsCastleCMS and Plone integration withOpenSearchThis product was forked fromcollective.elasticsearchin order to provide integration with OpenSearch instead of ElasticSearch. OpenSearch itself is
a fork of ElasticSearch and compatible with, at least, the ES 7.10.x series of releases (at least
at opensearch-py 1.1.0). Compatibility may diverge in the future, and while the collective.elasticsearch
package will likely try to maintain compatibility with ElasticSearch, wildcard.hps is intended
to maintain compatibility with OpenSearch.QuickstartFirst, start up an instance (for official guides, see theopensearch project documentation)$ docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:latest
$ curl -XGET https://localhost:9200 -u 'admin:admin' -kSecond, setup Plone/CastleCMS:addwildcard.hpsto theeggssection of your buildoutrun buildoutrestart your instance, using relevant Environment Variables to connect to your opensearch instanceinstall the 'Wildcard HPS' productunder the 'Wildcard HPS' control panel, click 'Convert Catalog' then 'Rebuild Catalog'Configuration Settings are passed as environment variables. See the "Configuration" section
below for more details.OverviewThis package aims to index all fields theportal_catalogindexes
and allows you to delete theTitle,DescriptionandSearchableTextindexes which can provide significant improvement to performance and RAM usage.OpenSearch queries are ONLY used when Title, Description and SearchableText
text are in the query. Otherwise, Plone's default catalog will be used.
This is because Plone's default catalog is faster on normal queries than using
OpenSearch.ConfigurationConfiguration for OpenSearch connections, and custom index naming, is done through
Environment Variables. This allows per-instance customization without the need to
modify site data, and allows for many deployments to use the same cluster(s) withoutneedingto do per-site customized index names.Available Environrment Variable Options:HPS_ZOPE_CONF_PATHpath to a zope.conf to get a Zope app instanceNOTE: this is only needed for thereindex_hpsscript that gets installed.
Seewildcard/hps/scripts/reindex.py.HPS_OVERRIDE_LOGGINGif present, will tell thereindex_hpsscript to override the root logging
configuration, and print logging to console at INFO level.HPS_FORCE_ENABLEdefault: noaccepted values (all other values are equivalent to False): Yes, True, 1, Onwill force the "enabled" lookup to be TrueHPS_INSTANCE_INDEX_PREFIXdefault: Nonea string value prepended to index names used by the Plone instances this addon is installed intoHPS_INCLUDE_TRASHED_BY_DEFAULTdefault: noaccepted values (all other values are equivalent to False): Yes, True, 1, Onwill default searchResults to include trashed entries (which are not included by default)HPS_FOCE_EXTERNAL_INDEXESdefault: Nonea list of object properties that will be included in the externally index object (IE
the indexed object in opensearch)OPENSEARCH_HOSTSdefault:https://admin:admin@localhost:9200a list of RFC-1738 formated urls. multiple urls can be specified by putting a space between urls.NOTE: for now, the opensearch-py (1.1.0) does not respect the HTTP auth info that is formatted
as part of the URL, instead useOPENSEARCH_HTTP_USERNAMEandOPENSEARCH_HTTP_PASSWORDto pass
the same HTTP auth to each request to any node listed as a host.OPENSEARCH_HTTP_USERNAMEdefault: Nonea username to use in all connections to any node in theOPENSEARCH_HOSTSlistOPENSEARCH_HTTP_PASSWORDdefault: Nonea password to use in all connections to any node in theOPENSEARCH_HOSTSlistOPENSEARCH_TIMEOUTdefault connection timeoutOPENSEARCH_RETRY_ON_TIMEOUTdefault: Offaccepted values (all other values are equivalent to False): Yes, True, 1, Onretry connection to different node when connection failsOPENSEARCH_SNIFF_ON_STARTdefault: Falseaccepted values (all other values are equivalent to False): Yes, True, 1, Onrefresh nodes before doing anythingOPENSEARCH_SNIFF_ON_CONNECTION_FAILdefault: Falseaccepted values (all other values are equivalent to False): Yes, True, 1, Onrefresh nodes after a node fails to respondOPENSEARCH_SNIFFER_TIMEOUTdefault: Nonerefresh node list on this time (in seconds) intervalOPENSEARCH_SNIFF_TIMEOUTdefault: 0.1timeout of sniff requestOPENSEARCH_USE_SSLdefault: Falseaccepted values (all other values are equivalent to False): Yes, True, 1, Onconnections to OpenSearch will use SSLOPENSEARCH_VERIFY_CERTSdefault: Trueaccepted values (all other values are equivalent to False): Yes, True, 1, Onverify SSL certificates when using SSL connections to OpenSearchOPENSEARCH_SSL_SHOW_WARNdefault: Trueaccepted values (all other values are equivalent to False): Yes, True, 1, Onwhen verifying SSL certificates is disabled, then a warning will be shown by defaultOPENSEARCH_CERTS_PATHdefault: Nonea path to a directory containing CA Certificates used in SSL verificationOPENSEARCH_CLIENT_CERT_PATHdefault: Nonea path to a PEM formated SSL client certificate for SSL client authOPENSEARCH_CLIENT_CERT_KEY--default: Nonea path to a PEM formated SSL client key for SSL client authCompatibilityOnly tested with Plone 5 with Dexterity types.Only compatible with versions of OpenSearch (and ElasticSearch) compatible
with theopensearch-pylibrary.For ElasticSearch integration, seecollective.elasticsearch.StateSupport for all index column types is done EXCEPT for the DateRecurringIndex
index column type. If you are doing a full text search along with a query that
contains a DateRecurringIndex column, it will not work.Celery supportThis package comes with Celery support where all indexing operations will be pushed
into celery to be run asynchronously.Please see instructions for collective.celery to see how this works.Running testsFirst, start an instance of OpenSearch.Second,$ virtualenv ./env
$ ./env/bin/pip install -r requirements.txt
$ ./env/bin/buildout -c buildout.cfg
$ ./bin/testChangelog1.4.4 (2023-10-11)missing '.items()'1.4.3 (2023-10-11)handle unicode for index data derived from IAdditionalIndexDataProvider adapters1.4.2 (2023-05-15)abstract unicode handling code for hook when getting index data, and handle
tuples, lists, and dict values1.4.1 (2023-05-11)handle unicode error and fix bug in hook when getting index data1.4.0 (2022-11-04)allow a custom prefix to be defined for fetching connection settings from the
environment (default to the previous hard-coded 'OPENSEARCH_' value)1.3.0 (2022-08-17)add HPS_FORCE_EXTERNAL_INDEXESupdate default set returned when external indexes setting is not configured yet1.2.1 (2022-06-23)fix some view name's in the control panel templates1.2.0 (2022-05-25)add HPS_INCLUDE_TRASHED_BY_DEFAULT env for disabling a filter on searchResults
from WildcardHPSCatalog (see readme entry for HPS_INCLUDE_TRASHED_BY_DEFAULT)1.1.1 (2022-05-12)add property on wildcard.hps.opensearch.WildcardHPSCatalog for the instance prefix1.1.0 (2022-05-12)initial fork from:https://github.com/collective/collective.elasticsearch/commit/d21bf7b9311a9fc923283eeff11c42f4145180b4this fork aims to primarily maintain compatibility with the OpenSearch project, which
itself has forked from ElasticSearch 7.10.