Bug 12550 - Add ability to delay loading of patron's checkouts in circulation and patron details

Now that a patron's checkouts are loaded asynchronously, we can further
improve Koha's performance by not loading the checkouts table when it is
not needed. For example, if a librarian is checking out 5 items to a
patron, we really don't need to load the table during the scanning of
items 1 to 4, just for item 5. Another example would be browsing to the
patron details table in order to change a patron's password.

Test Plan:
1) Apply this patch
2) Browse to circulation.pl for a patron
3) Note the table is not loaded automatically
4) Click the "Show checkouts" button
5) Note the checkouts table loads
6) Check the "Always show checkouts immediately" checkbox
7) Reload the page, the checkouts should now load immediately
8) Repeat steps 3-7 for the patron details page ( moremember.pl )

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: remove trailing spaces.
Note: I am not sure the checkbox is at the best place, but I don't have
a better suggestion.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 12550 [QA Followup] - Make checking the checkbox load the table of checkouts

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 13020 - Checkouts table default ordering is incorrect

The checkouts table is no longer keeping "today's checkouts" in the
order they were scanned.

Test Plan:
1) Create 3 records "Test 1", "Test 2" and "Test 3" each with one item
2) Check out the items to a patron in the order "Test 2", "Test 1", "Test 3".
3) Note the order is incorrect.
4) Apply this patch
5) Refresh the page
6) Note the order is now correct

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Note that the bug already exists before bug 12550. I checked on a 3.14.x
branch (without the ajax circ stuff), and the order is the same as with
this patch.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 13035: Overdue notices send notices for due date in future.

Overdue notices, script uses method days_between which returns always
positive value of days, no mater if due_date is higher or lower from
date_to_run. This causes overdue notices to be send for real overdue and
for checkouts with due date in future, which have same days_between as
in notice triggers.

To reproduce:
1. Set up overdue notice triggers for eg. 2 day.
2. Checkout items for 2 borrowers first gets item with due date 2 days
in past and second gets 2 days in future.
3. run ./misc/cronjobs/overdue_notices.pl -n -t
4. watch that notices was generated for both of them.

To test:
1. Repeat steps 1,2
2. Apply patch
3. run ./misc/cronjobs/overdue_notices.pl -n -t
4. should be 1 notice for borrower with overdue item.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script, just adds an additional test.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 8687 - Overdues: Add utf-8 support to HTML and additional fields to CSV output

This patch contains small improvements made in the overdue_notices.pl script.

- UTF-8 support in the HTML output
- Add the phone, cardnumber, branchname and letternumber columns
  to the csv output

Testing:

I) Create an issue overdue (intranet):
0) Enter in a patron account, add some characters like
   ユーザー別サイト in his address
1) Check if in Patron category administration the overdue column
   is YES for his code
2) In Tools -> Notice triggers define first delay = 7, letter
   Overdue notice. Nothing for second ni third
3) Check if Tools -> Notices and slips, code overdue is available
4) checkout a book with a retard more than 7 days.
5) Check if Adminstration -> Defining circulation and fine rules
   for all libraries has fine rules for his patron category,
   item type book.

II) Before applying the patch
0) Run misc/cronjobs/overdue_notices.pl -n -html .
1) Open the file notices_"date".html in a brower
2) Validate the characters are not correct

III) After applying the patch
0) Run misc/cronjobs/overdue_notices.pl -n -html .
1) Open the file notices_"date".html in a brower
2) Validate the characters are correct
3) Run misc/cronjobs/overdue_notices.pl -n -csv temp.csv
4) Validate phone, cardnumber, branchname and letternumber
   columns and values

Sponsored-by: CCSR

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script.
Works as described, no problems found.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Added the Sponsored-by line.
Bug 12914 - Wrong message 'Patron(..) is blocked for 2014-09-30 day(s).

The message in circulation.tt assumed to get days but date is given. Updated comments and message depending on expiration date or no expiration of restriction.

The message shows up on top of Bug 643 Allow override of 'debarred' status if a patron has a restriction.

Replaced date_format with date template (see comment #6)

Signed-off-by: Nick Clemens <nick@quecheelibrary.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 12068 - label-create-pdf.pl Add support for RTL language

On top of Bug 8375

If you print a label using arabic/hebrew script,
letters are printed in logical direction, from left
to right, giving a mangled result

This patch will try to fix those cases adding a new
perl dependency, Text::Bidi, and using the automagic
feature if it's log2vis() function to rearrange chars
based on detected text 'direction'

To test:
1. Install Text::Bidi package
(apt-get install libtext-bidi-perl)

2. Try a batch, using Helvetica, with a mix of
ltr and rtl (arabic/hebrew) titles, chars are good,
but direction is bad

NOTE: I suggest changing the mapping for 'HO' font
on koha-conf.xml, from DejaVuSans-Oblique.ttf to
DejaVuSans.ttf to view 'title' chars

3. Apply the patch

4. Try again, now the result is good

Formerly a followup of Bug 8375, look sample pics
on that Bug.
Rebased following changes on Bug 8375

Note: Arabic titles will not be displayed, because
current code selects Oblique variant (unless you
change mapping as suggested on 2. )
Hebrew looks good.

Rebased and move use of new dependency to Labels.pm

Rebased on master

Signed-off-by: Karam Qubsi <karamqubsi@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested with the help of Bernardo. With the patch
the characters of RTL strings appear in the correct
order in the generated PDF files.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 10960 - Holds Queue Report display loc code not description

The holds queue report shows codes for item shelving location and for
library transfer location. This patch adds template plugin handling of
both these fields to display their full description.

To test, make sure you have one or more items in your holds queue which
have a shelving location. Rebuild your holds queue if necessary and view
it. Items which have a shelving location should show the description
instead of the code. The "send to" column should show the library name
instead of code.

Signed-off-by: Chris <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, making use of the TT plugins.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 13114: Prevent Shibboleth Patches from spamming logs

- The shibboleth patch introduced an undefined message into the error
  logs, when shiboleth is disabled.

Testplan

1. Ensure shibboleth is disabled.
2. Refresh any opac page
3. See 'Use of uninitialized value $ENV{"REMOTE_USER"} in string ne at
/home/koha/kohaclone/C4/Auth.pm line 711.' popup in the opac-error.log
4. Apply patch
5. Refresh opac page
6. Error should no longer appear

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 13115 - Holidays.t will fail some unit tests on Sundays

Due to the fact that Sunday is a repeating holiday in the example data
which is used for part but not all of Holidays.t, the last test will
fail on Sundays. The test is to see if today is *not* a special holiday
for MPL, but since it's already a repeating holiday, is_holiday will
return true rather than false.

Test Plan:
1) Wait until Sunday
2) prove t/db_dependent/Holidays.t
3) Note the failure
4) Apply this patch
5) Repeat step 2
6) Note there is no longer a failure

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Used my time machine... :)

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 12068: (rm followup) remove useless newline introduced on merging

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 13085: bugfix to allow call of export.pl in commandline without id_list_file parameter

Test plan
1/ run ./tools/export.pl
2/ notice an error like cannot open 0: No such file or directory at ./tools/export.pl line 209.
3/ apply patch
4/ run again
5/ no errors

Signed-off-by: Chris <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, adding an additional check.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11812 - Add missing "required" indicator to fields which are required

Form fields in the staff client which are required should be configured
to be so by doing several things:

- Add a class "required" to the field's <label>
- Add a class "required" to the form field
- Add 'required="required"' to the form field
- Apppend a <span class="required">Required</span> after the form field.

Several places in the templates are missing the <span>. This patch adds
them.

To test, apply the patch and view the following pages to confirm that
the "Required" text appears:

- Acquisitions -> Add an order to a basket from a new (empty) record.
  Title, quantity, and fund should indicate they are required.

- Administration -> Authority types ->  New authority type. The
  authority type and description fields should indicate they are
  required.

- Administration -> Authority types -> MARC structure -> New tag. The
  tag field should indicate it is required.

- Patron types and categories -> New category. Category code,
  description, and category type should indicate that they are required.
  FIXME: Enrollment period is required but the user must choose one. I'm
  not sure how to handle that clearly.

- Tools -> CSV profiles. Profile name, profile type, and profile MARC
  fields should indicate they are required on both the new and edit
  forms.

- Administration -> Manage MARC modification templates. Under "Create a
  new template" the name field should indicate that it is required.

- Tools -> Batch patron modification -> Submit a batch for editing. Any
  fields which are required according to your BorrowerMandatoryField
  system preference should indicate that they are required.

Patch behaves as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

QA Edits: Re-worded the "required" not on the batch patron edit form,
added a missing word to the help text on that page. On the csv-profiles
page I removed an unnecessary "javascript:" protocol from the markup.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works and passes QA script.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 9859: fix nsb_clean side effect

Signed-off-by: Mathieu Saby <mathieu.saby@univ-rennes2.fr>
This sub was causing 2 bugs :
- tools/exports.pl --clean was removing Â
- authority search plugin used in cataloging was removing Â in suggested authorities displayed dynamicly (using ajax)
After applying the patch,
- NSB/NSE are still removed by nsb_clean
- tools/exports.pl --clean is no more removing Â
- authority search plugins is no more removing Â

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 9859: Follow-up - Adding a simple unit test for nsb_clean

This tests uses the example given on the perl file on the
bug report.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 12884: Get rid of redefined subroutine warnings in dateaccessioned.pl

Instead of removing all warnings, we should remove warnings about the
redefined subroutines.

Test plan:
- link the dateaccessioned plugin with one of your biblio field
- edit an item
- verify they are no warnings in the Koha log file.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 12005 : Creating a new zebra connection for each time we need one

Zebra is not designed to have persistent connections, under cgi this
didn't matter the scripts would get a new connection each time, but
under plack we try to use dead connections

This patch changes it so plack works the same way that cgi did.

To test:
Apply this patch
Do some searches
Check everything still works

Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 13104 - Changing date or time format on a production server will create duplicate fines!

The system preference TimeFormat decides if times are formatted in 24
hour or 12 hour format. The issue at hand is that this changes the time
description for fines. Let us assume the time format is currently 24
hour, and there are overdue items with accruing fines. When the format
is changed to 12 hour, and fines.pl is run, it will begin looking for
fines with a 12 hour format. Since there are no, it will create a new
fine and you will now have two fines for the same item!

There are solutions which require database manipulation:
1) Update all existing fines from 24 hour to 12 hour format
2) Delete all existing accruing fines so they are all regenerated the
following night

The same situation applies to DateFormat.

The accounts rewrite ( Bug 6427 ) will resolve this issue, as it does
not look for existing fines based on the description.

I believe the best course of action until bug 6427 is pushed to master
is to add a warning to the system preference to not alter it on a
production server with existing fines.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Adds some documentation to warn about the problem.
Tested the new addition shows up in po files.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 9368 [ALTERNATE] - specific behavior of yr and acqdate indexes

This patch fixes the "Publication date" and "Acquisition date"
searches when using non-QueryParser and QueryParser searches.

It adds structure attributes to the template, which is consistent
with how phrase searching is currently handled.

It removes unnecessary code from Search.pm, adds some necessary
code which is consistent with existing code, and adds a lot of
explanatory comments.

_TEST PLAN_

Before applying:
0) Turn off QueryParser
1) Try a "Publication date" or "Acquisition date" search from the
staff client advanced search.
2) Note that even though the description on the result page makes
it seem like you're doing an index-specific search, you're actually
doing a keyword search. You can verify this by checking the 008
from positions 7 to 10 for "Publication date" or "Date accessioned"
on items for "Acquisition date".

3) Turn on QueryParser
4) Try doing the searches from Step 1.
5) A "Publication date" search should probably produce zero results

After applying patch:
6) Keep QueryParser on
7) Try doing the searches from Step 1.
8) Notice that you're actually getting results consistent with
your search (ie the 008/7-10 shows the date you searched for,
and there is a "Date accessioned" in items which matches your
search)

9) Turn off QueryParser
10) Note that your results are exactly the same as step 8
(N.B. this is because QueryParser is falling back to non-QP mode
instead of producing a bad query.)

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11926: Render community koha statistic usages

Goals:
- Collecting Koha usage statistics
- Rendering stats on a community website
- Having a big bicture of how koha is used

3 parts in the project:
- this patch in koha
- hea-ws which collects data
- hea-app which renders data

Installation:
1/ Fill systempreferences:
UsageStatsLastUpdateTime UsageStatsID UsageStatsShare UsageStatsLibraryName
2/ Setup a cron in your crontab (ex: at 3:00 every first of the month):
0 3 1 * * export KOHA_CONF=/home/koha/etc/koha-conf.xml; export PERL5LIB=/home/koha/src; perl /home/koha/src/C4/UsageStats.pm

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>