Bug 11803: use $dbh consistently in _koha_modify_item

This is just some code cleanup, no behavior change expected.
Also replacing errstr with err in testing the results. (See DBI.)

Test plan:
Modify an item and save it.

Followed test plan. No problems found.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11746 - Add edit item links to record details page

In order to edit a specific item right now, one must select the edit
pulldown, then choose edit items, the click the edit link for the
particular item the librarian wishes to edit. It would be much more
convenient to have an edit link for each specific item in the items
table on the record details page.

Test Plan:
1) Apply this patch
2) View the record details for a record with items
3) Click the edit link for a particular item
4) Note that you are taken immediately to the item editor
   for this particular item

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Havilah Lyon <havilah@aflibrary.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11746 [QA Followup] display edit item link only if user has permission to edit items

Signed-off-by: Havilah Lyon <havilah@aflibrary.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 10751: standardize validation of title in staff purchase suggestions form

The form for adding a new purchase suggestion in the staff client
indicates that the title field is required but does nothing to enforce
this rule. This can be handled client-side with HTML5 validation
attributes and Koha's built-in validation plugin. This patch implements
this.

To test, apply the patch and go to Acquisitions ->
Suggestions -> New purchase suggestion. Try submitting the form without
entering a title. Doing so should trigger a validation warning.

Submission of the form with valid data should work correctly. Editing an
existing suggestion should also work correctly.

The patch displays a message 'Required' near the title field.
After submitting a empty title it changes to 'This field is required'
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
The title field is required since commit
d565fb02e9fca2ac9c506e4e7d44fd8b0d66f06a
This one makes it more consistent with formatting. It makes sense.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11707: fix editing additional attributes for patron whose category code includes a space

When editing additional attributes of a patron of a category with a
code with space (ie "CAT 3"), the Javascript is broken and Additional
attributes are not displayed.

This patch corrects by adding simple quotes around category code in JS
code.

Test plan :
- Create an patron additional attribute
- Create a new patron category with a space in code. ie "CAT 3"
- Create a new patron of this category : /cgi-bin/koha/members/memberentry.pl?op=add&categorycode=CAT 3
=> Without patch the JS is broken (Syntax error, unrecognized expression:
   [data-category_code=CAT 3]) and additional attributes are not
   displayed.
=> With patch the JS is not broken and additional attributes are
   displayed and can be edited.

Signed-off-by: Marc Véron <veron@veron.ch>

In current Koha version it is not possible to create category codes
with blanks.

To test, I changed an existing category code directly in the database.
Then I followed the Test plan.
Without patch, attribute types were not displayed.
With patch, they were displayed as expected.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
I recommend not using codes with spaces and fix existing ones.
Koha doesn't allow you to add category codes with spaces, so the only way
to do it is via SQL.
This patch fixes a problem and passes all tests.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11768: Show patron cardnumber in OPAC personal details

In previous versions of Koha, the patron cardnumber was displayed in the
personal details section of the OPAC for logged in users. A patron needs
to be able to access his or her cardnumber in some cases to access
online resources a library makes available that requires his or her
cardnumber. This is especially important for patron self registration
where a patron may not have a physical library card. This field should
be re-added to the patron personal details page.

Test Plan:
1) Apply this patch
2) Log into the OPAC
3) Browse to "your personal details"
4) Verify the patron cardnumber is visible

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Restores cardnumber on personal details tab in OPAC for
prog and bootstrap theme. Passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11819 - Don't show catalog edit menu if user has no edit permissions

Currently if the logged in user lacks any cataloging permissions the
bibliographic detail page (catalogue/detail.pl) still shows the Edit
button, but with an empty dropdown menu. This patch corrects the toolbar
include so that the button will not appear at all.

To test, view the biblio detail page as a user with various combinations
of the following permissions:

- edit_items
- edit_catalogue
- items_batchmod
- items_batchdel

The edit button should appear with the correct set of links when the
user has any combination of the above permissions. If user has none the
button should not appear.

Followed test plan, patch behaves as expected
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as expected, passes all tests and QA script.
Template only change.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11809: don't display 'Clear date' on OPAC patron details form if date of birth is required

On the OPAC's patron details form, if the date of birth field is
required we shouldn't offer the user a "clear date." This patch removes
the "clear date" link if the field is required.

To test, configure the BorrowerMandatoryField system preference to
require dateofbirth. Enable the OPACPatronDetails system preference and
log in to the OPAC. View the "personal details" form and confirm that
there is no "clear date" link next to the required date of birth field.

Remove dateofbirth from the BorrowerMandatoryField system preference and
re-test. The "clear date" link should appear.

Test in both prog and Bootstrap themes.

I tested with prog and bootstrap themes, adding/removing field 'dateofbirth'
in sys pref PatronSelfRegistrationBorrowerMandatoryField.
Patch behaves as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as expected, passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11717 - Use new DataTables include in member templates

Bug 10649 introduced a new include file for adding DataTables-related
JavaScript assets. This patch adds use of this include file to all
member pages which use DataTables.

Apply the patch and test the following pages to confirm that table
sorting works correctly:

- Patron details (members/moremember.pl)
  - The checkouts, relatives' checkouts, and holds tables have been
    modified to exclude articles when sorting of titles.

- Notices (members/notices.pl)
- Circulation history (members/readingrec.pl)
- Statistics (members/statistics.pl)
- Update child to adult patron (members/update-child.pl) - This is the
  page which appears in a pop-up when you update a child patron to an
  adult (from the "More" menu in the toolbar when viewing patron
  details).

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11710 - Use new DataTables include in cataloging Z39.50 search template

This patch modifies the template for the cataloging Z39.50 search
template to use the new DataTables include. This simplifies the
inclusion of assets and updates to the latest DataTables version.

To test, go to cataloging and choose "New from Z39.50." Perform a search
and confirm that table sorting still works correctly.

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11716 - Use new DataTables include in label manage template

This patch modifies the template for the label manage pages to use
the new DataTables include. This simplifies the inclusion of assets and
updates to the latest DataTables version.

To test, view the following pages and confirm that table sorting is
working correctly:

Manage layouts (/cgi-bin/koha/labels/label-manage.pl?label_element=layout)
Manage templates (/cgi-bin/koha/labels/label-manage.pl?label_element=template)
Manage profiles (/cgi-bin/koha/labels/label-manage.pl?label_element=profile)
Manage batches (/cgi-bin/koha/labels/label-manage.pl?label_element=batch)

Signed-off-by: wajasu <matted-34813@mypacks.net>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11736 - Use new DataTables include in Koha news templates

Bug 10649 introduced a new include file for adding DataTables-related
JavaScript assets. This patch adds use of this include file to the Koha
news page.

To test you should have existing news items with varying creation and
expiration dates. Apply the patch and confirm that table sorting works
correctly for all settings of the dateformat system preference.

C4::NewsChannels.pm has been modified so that it now passes an
unformatted date to the template, where the KohaDates plugin is used to
apply the correct formatting. Sorting is based on the unformatted date.

Also corrected: Capitalization errors.

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, no problems found.
Also passes tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11805 - Use validation plugin when creating new basket in Acquisitions

The page for adding a new basket in Acquisitions includes some custom
form validation JavaScript which can be removed in favor of HTML5
validation attributes and Koha's built-in validation plugin. This patch
does so.

To test, apply the patch and go to Acquisitions -> Choose a vendor ->
New basket. Try submitting the form without entering a basket name. This
should trigger a validation warning.

Submission of the form with valid data should work correctly. Editing an
existing basket should also work correctly.

Patch works as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as expected, passes all tests and QA script.
Tried adding a new basket with/without basket name and editing
an existing basket editing/emptying the basket name.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11829: fix JavaScript error on staff client biblio details pages if template variables are undefined

The JavaScript included in cat-toolbar.inc expects several variables to
be passed to the template for use in calculations. Several scripts
do not pass the required variables to the template, leading to a JS
error.

This patch re-adds (See Bug 9823) hold count calculation to the required
scripts and copies the acquisitions-related count calculation from
detail.pl into related bibliographic detail scripts so that the
commonly-included cat-toolbar.inc can receive correct values.

To test, locate a bibliographic record which has no holds and is not
used in a current or deleted order, but which has a local cover image
attached. Check the following views to confirm there is no JavaScript
error:

- Normal
- MARC
- Labeled MARC
- ISBD
- Items
- Images (from the Images tab on the "Normal" detail page)

Next, locate a bibliographic record with no items attached which is used
in an Acquisitions order. On each of the pages listed above, choose
"Delete record" from the Edit menu. This should trigger a warning that
the record is used in an order. Cancel the deletion.

Finally, delete the order which references the bibliographic record. On
each of the pages listed above, choose "Delete record" again. This
should trigger a warning that the record is used in a deleted order.
Cancel the deletion.

I went through the test plan, everything works OK.
It resolves Bug 11831 as well.

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
- Printing from MARC and ISBD view works again
- No more Javascript errors found
- Deleting a record with attached order gives the correct warning,
  both for ordered and cancelled.

Passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11798: fix problems with column sorting and row visibility in invoices table

1) Sorting on "Billing date" (functionality introduced by bug 10794)
doesn't work properly any longer after bug 10401 was pushed (another
side-effect being that "Vendor" column is currently not sortable
at all).
2) Only "top ten" invoices are always displayed in the table, and it's
not possible to change that without missing top/bottom DataTables
pagination bars.

This patch fixes sorting for "Billing date" &  "Vendor" columns
in invoices table and restores the ability to see all table
rows/search results (not just "top ten").

Test plan (you'll need to have 11+ invoices with various billing
dates entered in the system):
1) apply patch,
2) view the Acquisitions invoices page (acqui/invoices.pl), do some
search (= as broad search as possible) and confirm that "Billing
date" and "Vendor" columns now do sort correctly,
3) in situations when there are more than 10 search results, they
should all be visible in the invoices table.

NOTE: Generating invoices is a painfully convoluted mess.
      All my "Billing date" entries were blank, so was not
      able to confirm Billing date issues. However, Vendor
      sorting issues, seeing more than 10, and in my bizarre
      initial case the non-visible checkboxes were all fixed.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11096: add regression tests for indexing/retrieving large MARC records

This adds direct tests of indexing and retrieving
bib records that are larger than the ISO2709 limit
of 99,999 octets.

To test: verify that prove -v t/db_dependent/Search.t passes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11096: Wrap each test set configuration in Search.t into a subtest

This makes it easier to read. Also fixes the test count, and is rebased
against 11499 which is needed to correctly test the patches for this bug
(i.e. indexing mode needs to be correctly set in koha-conf.xml)

To test:

[1] Run prove -v t/db_dependent/Search.t

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11096: support the retrieval of large MARCXML records

This patch makes Koha <-> Zebra use MARCXML for the serialization when
using DOM, and USMARC for GRS-1.

* The following functions are modified to set the Zebra record syntax
according to the current sysprefs and configuration:

- C4::Context->Zconn
- C4::Context-_new_Zconn

* A new function 'new_record_from_zebra' is introduced, which checks the
context we are in, and creates the MARC::Record object using the right
constructor.

The following packages get touched to make use of the new function:
- C4::Search
- C4::AuthoritiesMarc

and the same happens to the UI scripts that make use of them (both in
the OPAC and STAFF interfaces).

* Calls to the unsafe ZOOM::Record->render()[1] method are removed.

Due to this last change the code for building facets was rewritten. And
for performance on the facets creation I pushed higher version
dependencies for MARC::File::XML and MARC::Record (we rely on
MARC::Field->as_string).

* Calls to MARC::Record->new_from_xml and MARC::Record->new_from_usmarc
are wrapped with eval for catching problems [2].

* As of bug 3087, UNIMARC uses the 'unimarc' record syntax. this case is
  correctly handled.
* As of bug 7818 misc/migration_tools/rebuild_zebra.pl behaves like:

- bib_index_mode (defaults to 'grs1' if not specified)
- auth_index_mode (defaults to 'dom')

here we do exactly the same.

To test:
 - prove t/db_dependent/Search.t should pass.
 - Searching should remain functional.
 - Indexing and searching for a big record should work (that's what the
   unit tests do).
 - Test an index scan search (on the staff interface):
    Search > More options > Check "Scan indexes".
 - Enable 'itemBarcodeFallbackSearch' and try to circulate any word, it
   shouldn't break.
 - Searching for a biblio in a new subscription shouldn't break.
 - Running bulkmarcimport.pl shouldn't break.
 - And so on... for the rest of the .pl files.

[1] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#render()
[2] a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11078: Add locking to rebuild_zebra

This patch adds locking to rebuild_zebra.pl to ensure that simultaneous
changes are prevented (as one is likely to overwrite the other).
Incremental updates in daemon mode will skipped if the lock is busy
and they will be picked up on the next pass.  Non-daemon mode
invocations will also exit immediately if they cannot get the lock
unless the new flag -wait-for-lock is specified, in which case they
will wait until the get the lock and then proceed.

Supporting changes made to Makefile.PL and templates for the new
locking directory (paralleling the other zebra lock directories).
We stash the zebra_lockdir in koha-conf.xml so rebuild_zebra.pl
can find it.

To address earlier QA concerns we:
1. added code to check if flock is available and ignore locking if
it's missing (from M. de Rooy)

2. changed default for adhoc invocations to abort if they cannot
obtain the lock.  Added option -wait-for-lock if the user prefers
to wait until the lock is free, and then continue processing.

3. added missing entry to t/db_dependent/zebra_config.pl

4. added a fallback locking directory of /tmp

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Doug merged the original patch with the QA changes.
Just for the record, noting here that the original patch was tested
extensively too by Martin Renvoize.
I have added a followup for some exceptional cases.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 11078: QA Follow-up for missing file permissions on lockfile

The original patch creates a lockfile in the ZEBRA_LOCKDIR.
It can fall back to /var/lock or even /tmp.
If the create fails, it dies. This can be considered as very
exceptional.

This followup adjusts the fallback location in /var/lock or /tmp
slightly.  It appends the database name to the folder in order to
prevent interfering between multiple Koha instances. Creation of the
lockfile has been moved to a subroutine extending directory and file
creation testing.

In the very unlikely case that we cannot create the lockfile (after
three separate tries), this follow-up allows you to continue instead
of die.  This is just as we did before we had file locking here. Every
time skipping a reindex could cause more harm than continuing and
having the race condition once in a while.

Test plan:
Test adding and removing lockdir from your koha-conf.xml. Check fallback.
Note that fallback in /var/lock or /tmp must contain database name.
Remove the lockdir config line and remove permissions from fallback. In
this case the reindex should continue but with a warning.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested with daemon and one-off invocation simultaneously.
Tested new wait parameter.
Tried all variations of lock directory (changing permissions etc.)

Signed-off-by: Galen Charlton <gmc@esilibrary.com>