Bug 13064 - Indexing problem with ICU on control characters

The ICU configuration files contains a rule to remove control characters :
  <transform rule="[:Control:] Any-Remove"/>
This rule is before tokenization.

The problem is that "[:Control:]" regex contains line feed, carriage return and tab. See http://www.regular-expressions.info/posixbrackets.html.
So when several lines are indexed, last word of line is joined with first line of next line. Thoses words are then not searchable.

For example :
  First line
  Second line
This will become "First lineSecond line", tokenized as "First", "lineSecond" and "line".

Test plan :
- Use ICU in Zebra configuration
- Choose an indexed field, like 300$a
- Create a new record
- Enter several lines in choosen field, like :
  First line
  Second line
- Index this record
=> Without patch the search on "Second" does not return the record
=> With patch the search on "Second" returns the record
- Same tests with tab and carriage return instead of line feed

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11413: Change the field number logic

This patch series is a bugfix for the Marc modification templates tool.

Bug description:
If you want to do an action (delete/update/move/...) on a multivalued
field and if a condition is defined on the same field, it is highly
probable the resulted record will not be what you expect.

For example:
Deleting All (or the first) fields 650 if 245$a="Bad title" works with
the current code.
BUT if you want to delete All (or the first) fields 650 with a condition
on 650$9=42, and if at least one field matches the condition :
- if you have selected all, all fields 650 will be deleted, even the
  ones who do not match the condition.
- if you have selected first, the first 650 field will be deleted, even
  if it does not match the condition.
The expected behavior is to delete the fields matching the
condition (and not all the 650 fields).

What this patch does:
This patch introduces 2 changes in the logic of Koha::SimpleMARC.
The first change is a change of the prototypes for the 2 routines
field_exists and field_equals. Now they return the "field number" of the
matching fields.
The second change is the type of the "n" parameter for all routines
using it in Koha::SimpleMARC. Before this patch, the "n" parameter was a
boolean in most cases. If 0, the action was done on all fields, if 1
on the first one only. Now it is possible to specify the "field numbers"
(so the array of field numbers which is returned by field_exists or
field_equals) for all routines which had the n parameter.

Test plan for the patch series:
Note: This test plan describes a specific example, feel free to create
your own one.
0/ Define a marc modification template with the following action:
  Delete field 245 if 245$9 = 42
1/ choose and export a record with several 245 fields.
For ex:
  245
    $a The art of computer programming
    $c Donald E. Knuth.
    $9 41
  245
    $a Bad title
    $c Bad author
    $9 42
2/ import it using the Stage MARC for import tool.
3/ verify the imported record does not contain any 245 field.
4/ apply all the patches from this bug report
5/ do again steps 2 and 3
6/ verify the imported record contains only one 245 field, the one with
245$9=41
7/ verify the unit tests passed:
  prove t/SimpleMARC.t
  prove t/db_dependent/MarcModificationTemplates.t

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11413: UT to show up the issue

These UT reflect this change:
- deletion of the field 245 if 245$a='Bad title'
- move of the 650 field to 651 if 650$9=499

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11413: Reflect the changes to the interface

Test plan:
- add/edit an action on the marc modification templates tools
- choose an action and define a condition
- define the source field as same as the condition field
- verify the All/1st dropdown list is changed to Every/1st

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11413: Fix return for ModifyRecordWithTemplate

Make sure the ModifyRecordWithTemplate routine returns undef.

This patch also removes a warning if GetModificationTemplates is called
without parameter.

Verify
  prove t/db_dependent/MarcModificationTemplates.t
returns green.

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11413: Fix field_numbers

This fix is a global fix for the MarcModificationTemplate feature.
Some unit tests were missing and some behaviors were wrong.
For instance, if you tried to update a non existent field, the script
crashed.

The following line was completely stupid:
    if $from_field ne $to_subfield

The field_number equals 1 if the user wants to update the first field
and 0 for all fields.

The field_numbers (note the s) variable contains the field numbers to
update. This array is filled if a condition exists (field exists or
field equals).

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 10473 - Max length should be 2 digit for adding multiple copies in add items page

As per the discussion, a prompt on a hard coded soft-limit is
far more acceptable as a solution.

TEST PLAN
---------
 0) Back up your DB. -- because a backup is always good!
 1) Log in to staff client
 2) Navigate to any biblio details
    (e.g. cgi-bin/koha/catalogue/detail.pl?biblionumber=#####)
 3) Click the 'Edit' dropdown button.
 4) Click 'Edit items'.
 5) Click 'Add multiple items'
 6) Enter a crazy high number (e.g. 999)
 7) Click 'Add'
    -- Koha just adds it! YIKES!
 8) Apply patch
 9) Repeat steps 5-7
10) Click 'Cancel'
    -- Koha does not add the items.
11) Repeat steps 5-7
12) Click 'Ok'
    -- Koha does add the items.
13) run koha QA test tools
14) Restore your DB.

Signed-off-by: Nick Clemens <nick@quecheelibrary.org>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 10473: Get rif of the placeholder

This placeholder is not really relevant here.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 10473: (RM followup) small wording change

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>