BibFormat: fix snippets for phrase queries

* Displays correctly snippets for phrase queries.
* Corrects highlighting of only whole words.
  (closes #1201)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
BibDocFile: support new magic library

* There are two flavours of the magic library (for file
  format guessing): the original bindings coming with the
  CLI file utilities, and the new python-magic bindings
  wrapping libmagic using ctypes. Support for the new
  library is here introduced (while still allowing for
  the old one).  (closes #1207)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-1.0' into maint-1.1

* maint-1.0:
  BibDocFile: support new magic library
global: test suite nosification

* Makes unit and regression test cases more friendly with respect to
  nose defaults.  That is, decorates functions that are not test cases
  as such, otherwise nose's defaults would consider them when running
  the test suite.  The nosification is useful for plugging test case
  output to Jenkins.

* Fixes all the codebase except WebSubmitFileConverterTestGenerator
  dynamically generated test cases in websubmit_regression_tests.py.

* Note that it is now possible to run unit tests like:
  `nosetests /opt/invenio/lib/python/invenio/*_unit_tests.py`
  and regression tests like:
  `nosetests /opt/invenio/lib/python/invenio/*_regression.py`.
  This gives equivalent results to `inveniocfg --run-unit-tests` and
  `inveniocfg --run-regression-tests`, respectively, modulo dynamically
  generated WebSumit tests mentioned above.

* Also, activates WebComment test cases that were accidentally
  commented out in 231caa6b80a41ad9a6b37f5bf7ddb142190742ed.
htmlutils: fix single quote escaping

* Fixes function escape_html() to translate single quotes into
  correct HTML entity; `&#39;` -> `'` instead of `&#34;` -> `"`.
htmlutils: improve js string escaping

* Moves/renames jsonutils.wash_for_js() function to
  htmlutils.escape_javascript_string() as this function would mostly
  be used to escape javascript in HTML context.
  Updates bfe_bookmark accordingly.

* No longer adds double-quotes around the returned string in
  order to be more flexible regarding the context in which the
  string is inserted.

* More complete escaping of Javascript chars when 'json' is not
  installed.

* Adds parameters to control the level of escaping, to be adjusted
  according to the context (HTML vs JS output, XHTML vs HTML,
  content-type, use of CDATA delimiters).

* Adds docstrings and unit tests.

* Merge note: fixed indentation in `htmlutils.py`.

* Merge note: cherry-picked from
  86f6acbca4fe48a4af19940a6e6ab4344f5d0682.

Tested-by: Tibor Simko <tibor.simko@cern.ch>
WebSession: login_method changes

* Is more gentle when login_methods are altered on an existing
  installation (e.g. by falling back to the default login method
  if the settings of a user specify a no longer existing login_method).

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-1.0' into maint-1.1

* maint-1.0:
  WebSession: login_method changes
global: test suite nosification

* Nosifies test suite for the maint-1.0 branch too.  The tests can be run via
  `sudo -u www-data nosetests /opt/invenio/lib/python/invenio/*_tests.py`.
Merge branch 'maint-1.0' into maint-1.1

* maint-1.0:
  global: test suite nosification
BibAuthorID: improvements, fixes and optimizations

* Now it's possible to disambiguate surname clusters without
  precomputing clusters for every surname, makes testing and tweaking
  much faster
* Disable citation analysis for INSPIRE disambiguation, too big memory
  footprint for such a small increase in accuracy
* Tweaking: facilities to collect and analyze statistics about final
  results in disambiguation.  Allow easy plotting of results features
  against thresholds and so on
* Added lots of assertions and tests to allow early detection of
  insidious problems during disambiguation
* Names comparison now translates unicode to ascii, thus ignoring
  accents. This improves disambiguation and search.
* Outdated claims are now correctly handled by the interface, and
  present message boxes to the user in order to make clear what is
  happening
* Greatly improved debug prints to ease debugging
* Comparison functions now behave properly and offer always correct
  results in correct range
* Comparison function fixed bugs in handling of INSPIREid
* Redistribution of weight in comparison function to achieve more
  significative results
* Disambiguation computations redefined in order to limit operation
  overflow/underflow problems
* Paper claiming code now handles safely all known stressful situations
  dealing with outdated data
* Admin paper claiming interface offers the possibility to manage
  external ids for persons.  Kind and features of external IDs are
  configurable through config file.
* UI style fixes
* UI fixed various bugs (mostly javascript related)
* UI improved extendability and reusability in templates through
  verbiage dictionaries
* UI Disabled option to 'forget decision' on rejected papers which made
  no sense
* Added a prototype for export of personID information in XML
  format. Usable but unfinished, will probably be moved to bibexport.

Co-authored-by: Nedko Nedkov <nedko.stefanov.nedkov@cern.ch>
BibAuthorID: improvements and fixes

* Fixes names capitalization
* Backend improvements and fixes
BibAuthorID: UI and backend improvements

* Backend:
  * optional disambiguation logging to file per process
  * optional dump of disambiguation final result for deep analysis
  * grapher to plot dump results
  * disambiguation algorithm optimizations and fixes
  * fixes in names comparison functions
  * rabbit recomputes external ids for persons after updates

* UI:
  * button in data tab to force recomputation of external ids, either
    add missing or force_rewrite
  * show personid number in data tab
  * bigger external id textbox
  * after updating something in data tab, the page remembers the right tab
  * added nofollow attribute on links

Co-authored-by: Nedko Nedkov <nedko.stefanov.nedkov@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>
BibUpload: FMT regression test case fix

* Fixes FMT regression test case that was failing randomly
  due to incomplete SQL comparison query.  (closes #1152)

Co-authored-by: Samuele Kaplun <samuele.kaplun@cern.ch>
BibAuthorID: fix in name comparisons

* Fixes name comparisons.  (closes #1313 #1314)

Tested-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-1.0' into maint-1.1

* maint-1.0:
  BibUpload: FMT regression test case fix

Conflicts:
	modules/bibupload/lib/bibupload_regression_tests.py
BibFormat: setUp/tearDown in unit tests

* BibFormat unit tests were amending bibformat_engine internal
  configuration variable to set up testing path. The implementation
  was lacking however proper tearDown causing all sorts of side
  effects to tests happening afterwards, in particular affecting
  get_record()/print_record() calls for empty record.
  This is now fixed.  (closes #1319)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
BibFormat: fixes unit tests

* FormatTest failed to append CFG_TMPDIR to sys.path and resulted
  in tests failing under some conditions. (closes #1320)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
BibFormat: checks for bibformat bin

* Before calling call_old_bibformat we check that bibformat
  binary exists. We do not install it anymore so it only exists
  as legacy.

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
WebSearch: advanced search notification if no hits

* If an Advanced Search query yields no hits due to the boolean
  operation, inform the user and also print the total search
  results numbers for each individual search pattern, quite like
  in the Simple Search query.

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>