Bug 9239: Add QueryParser module to Koha

Although the OpenILS::QueryParser class is going to be moved into a
separate CPAN distribution, this hasn't been done yet, and for ease of
testing it makes sense to include the class in Koha for the moment.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239: Introduce QueryParser driver for PQF

Since the most expressive query language supported by Zebra is PQF, this
patch adds a PQF driver for QueryParser which will translate QueryParser
queries into standard PQF (guided by mappings which have been written to
match Koha's existing Zebra configuration) which can then be sent to
Zebra. This driver, Koha::QueryParser::Driver::PQF(::*) extends the
OpenILS::QueryParser(::*) class(es), so as to preserve maximum
interoperability between the various users of the QueryParser driver.

Initially, search syntax is as follows:
* AND operator: &&
* OR operator: ||
* GROUPING operators: ( )

Fields can mostly be searched using the ccl prefixes they have now. The
exception is the various date limits which are searched with a syntax
like this: pubdate(2008)

For sorting, you can simply add #title-sort-az (etc.) to your query.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Test Passed successfully after installing missing dep for Test::Deep

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239: Add syspref to enable QueryParser

In light of the experimental nature of the QueryParser code at this
juncture, it behooves Koha to have the option of disabling QP, at least
for the purpose of comparing results between QP-enabled and QP-disabled
searches. This patch adds a UseQueryParser syspref to do just that. In
cases where the user enables QP without having QP installed, however,
there will be no dramatic failure, and Koha will simply fall back to
using the non-QP search code.

In order to reduce the overhead added by the introduction of QueryParser,
this patch adds a C4::Context->queryparser accessor to a static
QueryParser object which is initialized the first time it is required
and then reset thereafter.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Fixed merge conflict in sysprefs.sql and updatedatabase.sql

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixed merge conflict in sysprefs.sql und updatedatabase.pl
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239: Allow the use of QueryParser for all queries

With the inclusion of this patch, all searches will (try) to use
QueryParser for handling queries for both the bibliographic and authority
databases if UseQueryParser is enabled. If QueryParser is unavailable,
UseQueryParser is disabled, or the search uses CCL indexes, the old
search code will be used.

To test:
1) Apply patch.
2) Run the unit test with `prove t/QueryParser.t`
3) Enable the UseQueryParser syspref.
4) Try searches that should return results in the following places:
   * OPAC (simple search)
   * OPAC (advanced search)
   * OPAC (authorities)
   * Staff client (header search)
   * Staff client (advanced search)
   * Staff client (cataloging search)
   * Staff client (authorities)
   * Staff client (importing a batch using a match point)
   * Staff client (searching for an item for adding to a label)
   * Staff client (acquisitions)
   * Staff client (searching for a record to create a serial)
   * ANYWHERE ELSE I HAVE FORGOTTEN
5) Disable the UseQueryParser syspref. Repeat at least some of the
   searches you did above.
6) If all searches worked, sign off.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Searching still works as expected for variuos places.
QueryParser syspref seemed to be enabled by default

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 follow-up: POD and perlcritic fixes

There were several perlcritic violations for returning explicit undef,
and splitting up the various driver classes resulted in POD compliance
errors, where the POD was starting with head2 rather than head1. This
patch fixes those errors (but has no functional impact).

Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 follow-up: update koha-conf.xml

The previous patches were missing the koha-conf.xml updates. This patch
updates koha-conf.xml and makes the changes neccessary to include the
QueryParser configuration file in the packages.

To test:
1) Run Makefile and check generated koha-conf.xml to confirm that the
   line <queryparser_config>...</queryparser_config> is there with an
   absolute path.
2) That was it.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested update successfully - new line now shows up and
query parser is used.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: fix highlighting and merge error

QP searches with && broke search highlighting on the OPAC details page.
This patch corrects encoding of the query_desc parameter that is passed
to the details page.

My last attempt at rebasing also transposed the variable for index
names with the variable for operators, meaning that the dropdown in
the basic search did not work.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixes some problems raised during QA successfully.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA-follow-up: quiet warnings and fix problem with limits

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: escape CGI input

Koha was not previously escaping CGI input, which caused problems for
highlighting and is a security issue.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Thx for fixing this.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: the last QA follow-up was missing a require

This patch also corrects the definition of the an= index, which was
missing exactness.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: don't fall back to kw if field is specified

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: remove stray debug code

Remove a line of debug code from EG, provide better error handling
when presented with weird data in the authority linker, and correct
queryparser configuration to reflect the correct configuration for
Zebra.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: better error handling

* Fix a long-standing bug in the linker that could crash the linker when
run against odd data.
* Sanitize input to SimpleSearch.
* Correctly handle CCL indexes with QP.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: escape quotes in searches

This patch escapes quotes in searches for those rare instances where
there is a literal quote inside a search term.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: amend error handling

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239 QA follow-up: fix overeager quote escaping

The quote escaping added two follow-ups ago was a little too eager,
and escaped perfectly valid quotes in some instances. This patch moves
the escaping deeper into the loop so that no needed quotes will be
escaped.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Test plan included here, as this is the last patch in the series:

All tests pass.

OPAC
Simple search
- Keyword searches using boolean operators || and && - OK
- Pubdate searches using pubdate() - OK
- Using different search options: keyword, title, etc. - OK

Facets
- limit to available from a query parser simple search - OK
- Other facets - OK

Advanced search
- Date range search - OK
- Itemtype limits - OK
- Language limit - OK
- Combining various serach options with and without 'more' options
  and using boolean operators - OK

Authorities
- Searching for partial and complete names like "M", "Benedictus" works. - OK
- Biblio count isc correct - OK
- Links to linked biblios work. - OK.

INTRANET
Staged MARC import
- Import records using various matching points - OK

Serials
- Search for a serial in subscription form - OK

Acquisitions
- Search for existing record from basket - OK

Authorities
- Running the linker script with QP on works. - OK
- Searching for authorities works. - OK
- Biblio count isc correct - OK
- Links to linked biblios work. - OK.

NOTE: As agreed with Jared in chat, query parser should be
off by default for the next release.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Bug 9239: Disable QP by default
Bug 9239: Increment version number

Database update for bug 9239 has been given version 3.11.00.029
Merge branch 'bug_9239' into 3.12-master
Bug 4906: Allow fine calculation to exclude grace period

This patch adds the system preference FinesIncludeGracePeriod which
allows the library the choice to include the grace period when calculating
a fine ( Koha's current behavior ) or to ignore the grace period for
the purposes of fines calculation. It is set by default to preserve the
current behavior.

Signed-off-by: Srikanth Dhondi <srikanth@catalyst.net.nz>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Rebased March 8, 2013. Included renaming pref.
Works as advertised. Renamed the bug title and opened a new report for the
original bug.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>