Package: repoze.pgtextindex
repoze.pgtextindexis an indexing plugin forrepoze.catalogthat provides a text search engine based on the powerful text indexing
capabilities of PostgreSQL 8.4 and above. It is designed to take the
place of any text search index based onzope.index. Installation
typically requires few or no changes to code that already usesrepoze.catalog.The advantages ofrepoze.pgtextindexoverzope.index.textinclude:Performance. For large datasets,repoze.pgtextindexcan be
orders of magnitude faster thanzope.index, mainly becauserepoze.pgtextindexdoes not have the overhead of unpickling
objects thatzope.indexhas.Lower RAM consumption. Users ofzope.indexwork around the
unpickling overhead by keeping large caches of unpickled objects
in RAM. Even worse, each thread keeps its own copy of the object
cache. PostgreSQL, on the other hand, does not need to maintain
complex structures in RAM. The PostgreSQL process size tends to
be constant and reasonable.Maintenance. The text indexing features of PostgreSQL are well
documented and receive a great deal of active maintenance, whilezope.indexhas not received much developer attention for
years.repoze.pgtextindexdoes not cause PostgreSQL to be involved in
every catalog query and update. Only operations that use or change the
text index hit PostgreSQL.Usagerepoze.pgtextindexis used just like any other index inrepoze.catalog:from repoze.pgtextindex import PGTextIndex

index = PGTextIndex(
    discriminator,
    dsn,
    table='pgtextindex',
    ts_config='english',
    drop_and_create=False,
    maxlen=1048575)The arguments to the constructor are as follows:discriminatorTherepoze.catalogdiscriminator for this index.  For more
information on discriminators see therepoze.catalog documentation.
This argument is required.dsnThe connection string for connecting to PostgreSQL.  This argument is
required.tableThe table to use for the index.  The default is ‘pgtextindex’.ts_configThe PostgreSQL text search configuration to use for the index.  The
default is ‘english’ which is the default built in configuration which
ships with PostgreSQL.  For more information on text search
configuration, see thePostgreSQL full text search documentation.drop_and_createIfTruethe table and index used will dropped (if it exists) and
(re)created.  The default isFalse.maxlenThe maximum number of characters to index per document.  The default is
1048575 (2**20 - 1), which is the maximum allowed by the to_tsvector
function.  Reduce this to improve query speed, since the
ts_rank_cd function retrieves and decompresses entire TOAST tuples
when querying.1.4 (2015-06-20)WeightedQueries can now be used as query result caches, making it
possible to search the catalog many times while hitting the text
index only once.When a query generates a large number of results, pgtextindex now disables
the expensive text ranking for that query.  The max_ranked attribute
controls the threshold for disabling ranking.  The default max_ranked
value is 6000.Improved speed by using BTrees instead of Buckets and by using
cursor.fetchall() instead of iter(cursor).1.3 (2014-09-03)Changed the ‘marker’ column to an array and changed the ‘marker’ attribute of
‘IWeightedQuery’ to accept either a single marker string or a sequence of
marker strings. Since the database schema has changed,
‘PGTextIndex.upgrade()’ will need to be run on any indexes created with an
older version of the code. (LP #1353483)1.2 (2012-10-09)Improved query speed by about 10% by duplicating the query parameter
rather than joining with the query.Added the maxlen option to allow a configurable document size limit.1.1 (2012-09-06)Handle concurrent index updates cleanly.1.0 (2012-09-01)Retry on IntegrityError to avoid meaningless errors.Added metrics using the perfmetrics package.0.5 (2012-04-27)Switched to read committed isolation and removed explicit locking.
The explicit locking was reducing write performance and may have been
interfering with autovacuum.  This change raises the probability
of temporary inconsistency, but since this package did not provide
ACID compliance anyway, developers already need to be prepared for
temporary inconsistency.0.4 (2011-11-18)Truncate text to 1MB per document in order to stay under (silly) limit
imposed by PostgreSQL.0.3 (2011-06-30)Fixed PostgreSQL ProgrammingError when query string contains a backslash
character.  (LP #798725)Added ability to mark content with arbitrary markers which can be used as
discriminators at query time.  (LP #792334)Support searches for words containing an apostrophe.  (LP #801265)0.2 (2011-06-15)Reworked the scoring method: added a per-document score coefficient.
The score coefficient can boost the score of documents known to be
trustworthy.Added the IWeightedText interface.  The discriminator function can
return an IWeightedText instance to control the weights and
coefficient.Added the IWeightedQuery interface.  Text index queries can
pass an IWeightedQuery instance to control the weight values.Allow persistent objects to be indexed, since the usual objection
(accidental ZODB references) does not apply.Do not drop and create the table by default, making PGTextIndex
easier to use outside ZODB.Added the ‘get_contextual_summaries’ and ‘get_contextual_summary’
methods to the index.Compatability with repoze.catalog 0.8.0.0.1 (2011-01-20)Initial release.