Vandelay authority existing queue search repair

When searching for an existing authority queue, search for an authority
queue, not a bib queue.  Apart from simply not working, the query will
fail with an "invalid input value for enum vandelay.bib_queue_queue_type"
and result in a client-side error.

https://bugs.launchpad.net/evergreen/+bug/989391

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Optimize full text search with a CTE

The tsquery used directly in a JOIN was resulting in bad
plan with lots of nested loops and operations. Using a CTE
to define the query improves the plan.

Also, instead of ARRAY_AGG(DISTINCT foo), just return the
whole thing in an array. Not great for dupes, but we can
handle those inside search.query_parser_fts if we really
need to.

Signed-off-by: Dan Scott <dscott@laurentian.ca>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
QP support for nested WITH construction

Nested query plans build their own tsquery chunks which have to be
individually JOINed to the final query.  To do this, each tsquery SELECT
gets its own entry in the top-level WITH, complete with alias for
joining.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
Add libnet-https-any-perl as precise deb requirement in Makefile.install.

Business::OnlinePayment::PayPal requires it, and it does not always want
to install from CPAN. It says it require either Net-SSLeay or
Crypt-SSLeay but the CPAN package never wants to install unless they
both are installed. The deb package for Ubuntu seems to do the right
thing.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
QueryParser: comma-delimit multiple WITH clauses

Bill Erickson's fix for my breakage of QueryParser.pm for the speedup
via the WITH common table expression usage inadvertently left out
the commas required to delimit multiple WITH clauses, resulting in SQL
query errors.

This fixes that oversight. Note that the fts-replacement.pl script came
in quite handy for testing this; recommended.

Signed-off-by: Dan Scott <dan@coffeecode.net>
Signed-off-by: Bill Erickson <berick@esilibrary.com>
abstract query representations from QueryParser

When you perform a search, you now also get back an abstract
representation of your search query.  fts-replacement.pl is a test
script that shows you things that QueryParser does, including the new
abstract query feature, and it also reconstitutes a string from the
abstract query to demonstrate the correctness of that query.

[Also]
Make last-wins true for facets, filters and modifiers -- arg!; Parameterized values win in the staged-search QP wrapper

Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Signed-off-by: Dan Scott <dan@coffeecode.net>
Fix typo in TEST_SETUP for QueryParser.pm

"author|coporate" doesn't exist, probably doesn't matter much in
principle but is worthwhile getting rid of the distraction.

Signed-off-by: Dan Scott <dscott@laurentian.ca>
Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Makes a warning go away when running fts-replacement.pl, but may have...

...consequences.

Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Signed-off-by: Dan Scott <dan@coffeecode.net>
Use spaces instead of tabs in QueryParser.pm

Copying swathes of SQL from logs into psql would often trigger the
autocompletion that psql offers, garbling a query awfully. Define the
local $spc var as two spaces and use that instead, leaving open the
option for reverting to tabs or anything else people prefer.

Also remove trailing whitespace, as this whole thing is about space.

Signed-off-by: Dan Scott <dan@coffeecode.net>
Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
QueryParser test script

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
Query Parser nested dynamic filters

Dynamic filters (from metabib.record_attr) are now parsed along with
search strings and facets as Query Parser descends the parse tree.  The
WHERE clause for the query as a whole is now a combination of the main
WHERE and the ON clause for JOINing to metabib.record_attr.  This gives
us the ability to create complex boolean nested queries using
SVF/record_attr filters, unlike before where nested filters were either
ignored or always treated as global filters.

For example:

concerto && ( ( item_type(a) subject|topic[Music] ) || item_form(a) )

This query now successfully limits the "concerto" search to results that
have either (item_type(a) AND subject|topic[Music]) OR item_form(a).

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
Query Parser merged copy / non-dynamic filters

When more than one copy-level filter (e.g. locations()) is used in a
query-parser query, all but the first are ignored.  The goal of this
work is to compress multiple copy-level filters into a single filter
that represents the full collection.  Wherever possible, boolean
structures will be honored, to the extent possible for post-search
copy-level filters.

Examples:

concerto locations(1,2) locations(3,4)

==> concerto locations(1,2,3,4)

( concerto locations(3,4,5) ) || (piano locations(3,5,7) )

==> ( concerto || piano ) locations(3,4,5,6,7)

( concerto locations(3,4,5) ) && ( piano locations(3,5,7) )

==> concerto piano locations(3,5)

Note, in the last 2 examples, the final query does not exactly match the
original.  This is because copy-level filters are applied after the
initial search and cannot be executed as part of the  nested query.  The
best we can do is create a representation of the final copy-level
filter, based on the nesting, and apply it to the final result of the
nested search.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
Search Filter Groups and Generic Queries : DB / IDL

Adds 3 new tables.  The first is a general purpose container for
query-parser queries.  These can be used by other entities for storing
saved searches, saved search filters, etc.  Each contains a query string
and a label.

The two other tables are for creating org unit saved search filter
groups.  A filter group is a named set of search queries, treated as
filters, collected to create a high-level search filter.  Filter group
entries use search_query's under the covers, so the content of the
filter can be any valid query string, including other filters or even
other filter groups.

For example, you could create an "Audience" filter group with entries
like, "Children" => "audience(a, b) locations(3,4,5)" to filter on copy
location in addition to the MARC audience data.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
Teach QueryParser about search filter callbacks, and if a callback returns a value, parse it at the current recursion level

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
Give the PG QP driver a subquery callback for the new saved_query() filter

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Bill Erickson <berick@esilibrary.com>
New QueryParser filter_group_entry filter

filter_group_entry() takes one or more filter_group_entry IDs as
parameters and applies the included sub-searches as filters on the
current search.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
tpac : search_HINT cache fixes

When searching for multiple values like a list of IDs, generate the
cache key from the sorted JSON form of the search value instead of the
bare ARRAY ref.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
TPAC: Search Filter Groups

Adds support for a new CGI param:

fg:<filter_group_code>=filter_group_entry

Each filter group entry will be mapped to a filter_group_entry()
filter and appended to the current search query.

This includes a new filter selector builder at
parts/filter_group_selector.tt2.

Usage, assuming a filter_group with code "audience":

<span>[% ctx.filter_groups.audience.label %]<span>
<span>
[%
    INCLUDE 'opac/parts/filter_group_selector.tt2'
    filter_group=audience
    none_ok=1
%]
<span>

The set of filter groups available to the TPAC are those defined at the
search org unit and its ancestors.  If multiple filters have the same
code name, the filter owned closest to the context org unit is used.
This allows for global filters with local overrides.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
stamping upgrade script for saved queries

Signed-off-by: Mike Rylander <mrylander@gmail.com>
TPAC: recover the ability to override hold placement failures

There is a certain class of hold failure events (e.g. HOLD_EXISTS,
HOLD_ITEM_CHECKED_OUT) that can be overridden by patrons when they have
the correct override permission.  This change recovers that ability,
which was recently removed with 99e8fc893a.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>