Bug 11352: fix unexpected data loss issues with batch patron deletion/anonymization

Batch patron deletion/anonymization tool has some issues:

1) If 'dateformat' setting in I18N/L10N system preferences is set
to anything other then 'iso' (eg. 'metric'), bulk deletion when using
'expiration date is before' criterion is not working properly. Date
entered in this field will be efectivelly ignored (or possibly
sometimes wrongly interpreted as different date, in other format)
on the final patron deletion stage. This may result in deleting
(or moving to trash) more borrower records then intended.
2) Bulk/batch patron deletion should skip borrowers with
nonzero account balance (ones with oustanding fines or credits)
3) This tool shouldn't offer to choose as deletion criterion
those patron categories which have category_type set to 'S'
(= staff patron categories)

This patch fixes above mentioned problems. It also adds an option
to "test run" patron batch deletion, and makes this option
the default choice in "warning" stage.

Test plan:
- prepare test database with some patron records (at least 2,
the more the better) set up in such a way that they will be vulnerable
to issues 1 & 2
- confirm issues 1,2
- restore test database
- apply patch
- ensure issues 1 & 2 are no longer present - first by using new "test
run" option: for #1, record counts in "warning" stage and "final" stage
should be now the same; for #2, observe that patron records with nonzero
balance are now excluded from deletion
- redo the tests, this time choosing "delete permanently" and "move
to trash" instead of "test run"
- test #3 by changing "Category type" to "S" in some
test patron categories - after that, those categories should no
longer be choosable as deletion criteria.

Signed-off-by: Magnus Enger <digitalutvikling@gmail.com>
Tested with dateformat = dd/mm/yyyy. I tested with two expired patrons,
one with fines and one without. Before the patch a lot of unexpected
patrons were deleted along with the expected ones. After
applying the patch only the expired patron was deleted, not the
one with fines. The test run and the "real" run reported correct numbers.

The patch also makes sure no patron categories with category_type = S
are suggested for batch deletion.

Note: The ergonomics of the "Batch delete/anonymize" tool is hardly
optimal, but this patch fixes a real, data-loosing bug, so let's
deal with the ergonomics later.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 69216d2217519cd6d8ad97710236251abdd89ae9)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 0853bfd32c840aedf13a5aa3327905c8c6b2067e)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Bug 11329: Check for MARC record existence in opac-showmarc

Instead of just running as_formatted, check if GetMarcBiblio returned
a reference. If you e.g. did not pass an id, return 404 instead of 500.
Consistent with opac-export.pl

Test plan:
[1] Run opac-showmarc.pl with valid biblionumber in id parameter.
[2] Remove id parameter from URL. You should get a 404 now.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 80bcbd798991eaa57bcc7b5ac8906b276fbed5bf)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 5d4a6bb712fe2d4b6179cc15eaf29cda2676b547)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Bug 11329: Check for MARC record existence in catalogue/showmarc

Intranet counterpart patch. Same check as in opac/opac-showmarc.

Test plan:
[1] Run showmarc.pl with valid biblionumber in id parameter.
[2] Remove id parameter from URL. You should get a 404 now.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Passes koha-qa.pl, works as advertised.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit d92dc91f778f5db3360f3070d0bb5d44a0d75024)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit ce7cd56c0b60031d6140a61e0f135db17deeba31)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Fixing typo in updatedatabase
Bug 11522 self-registration formating

1. in prog theme, enable opac self-registration and inspect clear date with firebug
   note that there are empty paragraph tags
2. add date of birth as a requirment in self-registration
3. apply patch
4. refresh page and notice that empty paragraph tags have been deleted,
   so that required lines up with clear date
5. repeat the above steps for bootstrap theme

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 11476 - Opac self registration title pull down offers 2 empty options

If you have patron titles enabled, the OPAC patron details form lists 2
empty spaces instead of 1. An empty option is passed by the script, so
the template does not need to include one.

To test you must have at least one patron title set in BorrowersTitles
system preference and OPACPatronDetails must be set to "allow."

1. Apply the patch
2. Log in to the OPAC
3. Click the "personal details" tab.
4. Click the "salutation" dropdown. There should be only one blank line
   preceding the existing titles.

Test in both prog and bootstrap.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described for existing and creating new accounts.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 11554 Capitalization fix in patron account on fines tab

Works as described.

To test: Create patron account with outstanding fines. Open "Fines" tab.
Confirm that capitalization is correct.

Sponsored-by: Hochschule für Gesundheit (hsg), Germany
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
String patch.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 10691: 5xx not properly linked by authid in authority search result list [Staff]

This patch changes the URL and data used to show the 'see also' links
on the Staff's authority search results page.

Bonus points: makes some strings translatable.

To test:
- On your dev setup (master) create some authority records (i created personal name authorities).
- Pick one of them and link 400$a to another one, do the same with 500$a
- Add some other 400$a and 500$a entries with plain text (i.e. no linking)
- Make sure zebra is running and changes got indexed.
- In the staff interface search for the authority that is linked to the others.
- Check the 'see also:' link points to an authority search
- Apply the patch
- Reload/re-do the search
- Check the 'see also:' link points to the authority id for linked authorities,
  and to an authority search result in the case of plain text entries.
- Check that the authority search from the cataloguing interface still works as usual.

Regards
To+

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Work as described. No errors

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Bug 10691: 5xx not properly linked by authid in authority search result list

This patch changes the URL and data used to show the 'see also' links
on the OPAC's authority search results page.

Bonus points: makes some strings translatable.

To test:
- On your dev setup (master) create some authority records (i created personal name authorities).
- Pick one of them and link 400$a to another one, do the same with 500$a
- Add some other 400$a and 500$a without linking (i.e. plain text)
- Make sure zebra is running and changes got indexed.
- In the OPAC search for the authority that is linked to the others.
- Check the 'see also:' link points to an authority search
- Apply the patch
- Reload/re-do the search
- Check the 'see also:' link points to the authority id in the case of linked authorities,
  and to an authority search in the case of plain text names.

Regards
To+

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Work as described. No errors

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Bug 11363 - Label printer profile units are not translatable

Changed the unit names form an import to a switch case so that it is in
the .tt so that it can be translated

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 11138 - Add auto_unsuspend_reserves to crontab.example

To test:

Set up and run the cronjobs from crontab.example with a hold set to unsuspend today.
The hold should be unsuspended.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 11138 - auto_unsuspend_holds cronjob not in default cron jobs.

to test:
Install the packages with this patch, observe the /etc/cron.daily/koha-common file - auto_unsuspend_holds should now be in the default list.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 11513 - Warnings in Patron categories

Just going to the patron categories page triggered errors.
Running through all the plain options also triggered other
warnings. This fix silences them.

Discovered tabs I had not corrected by running qa test tool.

Some errors which I could not trigger were also fixed, such
as line 248 shown by Merllissia Manueli.

TEST PLAN
---------
 1) Log in to staff client
 2) Click 'Administration'
 3) Click 'Patron categories'
 4) Click '+ New category'
 5) Enter a dummy category and click 'Save'
 6) Click 'Edit' for the dummy category.
 7) Change a value and click 'Save'
 8) Click 'Delete' for the dummy category.
 9) Confirm to delete.
10) Review error log, several new warnings
11) Apply patch
12) Run the koha qa test tool.
13) Click 'Home'
14) Click 'Administration'
15) Click 'Patron categories'
16) Click '+ New category'
17) Enter a dummy category and click 'Save'
18) Click 'Edit' for the dummy category.
19) Change a value and click 'Save'
20) Click 'Delete' for the dummy category.
21) Confirm to delete.
22) Review error log, no new warnings

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Followed test plan, saw no errors in the log after applying the patch.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested various dates and enrollment periods with different settings of
dateformat pref. Works as advertised. No warnings.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 11513: (follow-up) format enrollment end date consistently

The previous patch added use of the KohaDates TT plugin, so this
patch makes sure that it gets used to format the display of all
occurrences of the enrollment end date.

To test:

[1] Create a patron category with a fixed end date.
[2] Bring up the list of all categories and verify that the date
    is displayed based on the dateformat value.
[3] Delete the category, and verify that the confirmation dialog
    formats the date correctly.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 6b6377316eb49d94476e9442728389cc4c9297ff)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 1448ee66b6dff5d9fd4caa0789fe49d83906f0c8)
Bug 11290: fix display of patron home page for certain translations

To test
1) In a fresh Koha, generate translation into XX
2) Select XX for the staff interface (i.e. enable it, and choose it)
3) Go to the Patrons page
4) Everything looks fine
5) In the admin page, create a patron category with any non-ASCII
   character.
6) Go back to the Patrons page, everything is broken
7) Apply patch
8) regenerate translation
9) Go back to the Patrons page, everything looks fine
10) Switch to english it still looks fine

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Tried with ru-RU and fixes the problem.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Problem is also visible on untranslated templates changing
one of the sample category descriptions to include diacritics.

[RM note: I wasn't able to reproduce this on an unstranslated
 template -- I tried using Arabic, Cyrllic, and Spanish characters]

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 3e1b781adc253722209e46385fb3248ae8d668f9)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 923657526c2d7ac6d39ce5fd998c8249f43e3f51)
Bug 11608: Remove the word 'library' from funds list

The list of funds on the main acq page shows the library's name
followed by the word 'library'. I don't think this word is necessary
as most libraries have the word 'library' in their names. Even
if they don't they probably just want their library name to show
without extra words.

To test:

* Visit the acquisitions module before applying
* See the library's name + 'library' above the funds list
* Apply patch
* Look again at funds list and 'library' should be gone

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: remove a space

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit b865e1a024331fbbb76570f84afc9302a7340f7d)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 740bd016a035d4e8b2fcfc385c10dbbe979ec227)
Bug 11608: Delete unused commented lines in template

On the way, these lines are commented and can be deleted safely.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Removes an HTML comment with unused code.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit fb8e159daad8556832d45c9581e32b76aee02e1c)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 6f9e61df9a9fec128fda5c264d6a99d2451e839d)
Bug 11493: Revert "Bug 6869 Batch item modification fails whit accented characters"

This reverts commit c9905750e53e2e1615842ac696d5eeb0a8ac07fc.

Test plan:
1/ Open a bib record with more than 1 item ,
2/ Select all items
3/ Chose Modify selected items
4/ put you "ق ك و" in notes
5/ Verify the items have been updated correctly
6/ put you "éàç" in notes
7/ Verify the items have been updated correctly
8/ put you "ق ك و éàç" in notes
9/ Verify the items have been updated correctly

Patch behaves as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Confirmed bug on master and that the patch fixes it.
Passes all tests and the QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit e55d89be20acb58deb45a30c113bf0f07170fd33)

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 1bf1dacc7b2ec64b9b17d9b3947209918b3ba02f)
Bug 10611: Use mysql_auto_reconnect instead of ping

DBD::Mysql provides a mysql_auto_reconnect flag. Using it avoids
the time required to do a $dbh->ping().

Benchmarks:

use Modern::Perl;
use C4::Context;
for ( 1 .. 1000 ) {
    $dbh = C4::Context->dbh;
}

* without this patch on a local DB:
perl t.pl  0,49s user 0,02s system 98% cpu 0,525 total
* without this patch on a remote DB:
perl t.pl  0,52s user 0,05s system 1% cpu 37,358 total
* with this patch on a local DB:
perl t.pl  0,46s user 0,04s system 99% cpu 0,509 total
* with this patch on a remote DB:
perl t.pl  0,49s user 0,02s system 56% cpu 0,892 total

Testing the auto reconnect:
use Modern::Perl;
use C4::Context;
my $ping = $dbh->ping;
say $ping;
$dbh->disconnect;
$ping = $dbh->ping;
say $ping;

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Real improvement. No koha-qa errors

prove t/db_dependent/Circulation_issuingrules.t produces no error
prove t/db_dependent/Context.t produces no error

Test
1) dumped Koha DB, load it on a non-local server
2) run sample script whit and without patch, local and remote

use Modern::Perl;
use C4::Context;
for ( 1 .. 100000 ) {
    my $dbh = C4::Context->dbh;
}

Main difference I note is with remote server
a) without patch
real    0m16.357s
user    0m2.592s
sys     0m2.132s

b) with patch
real    0m0.259s
user    0m0.240s
sys     0m0.012s

I think this could be good for DBs placed on
remote servers

Bug 10611: add a "new" parameter to C4::Context->dbh

When dbh->disconnect is called and the mysql_auto_reconnect flag is set,
the dbh is not recreated: the old one is used.

Adding a new flag, we can now force the C4::Context->dbh method to
return a new dbh.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Bug 10611: Followup: remove useless calls to dbh->disconnect

These 3 calls to disconnect are done at the end of the script, they are
useless.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 4ae3665ad8afb5d283d989c4622a91690e1c4f8a)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Conflicts:
	C4/Context.pm

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 82f5b527d8cb79e8d1309b42faa461bcea12924d)
Bug 10611: (follow-up) cache DBMS driver code in context object

Rather than use a package-level variable, use an object-level
one.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 7d39b3c72ea2422b09e3411bef974d87aecdabf9)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Conflicts:
	C4/Context.pm

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 1de3d85574568b8eef769dc5d16af879b920b65f)