BibFormat: fix snippets for phrase queries

* Displays correctly snippets for phrase queries.
* Corrects highlighting of only whole words.
  (closes #1201)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-1.1'

* maint-1.1:
  BibFormat: fix snippets for phrase queries
solrutils: add documentation

* Adds Solr documentation to the BibIndex and BibRank Admin Guides
solrutils: better invalid character handling

* Removes invalid characters (closes #1197)

* Consolidates removal of control characters
  and invalid characters

* Adds unit tests for character removal
solrutils: removal of unused code

* Search results are directly retrieved by opening a Solr URL
solrutils: better collection filter generator

* Empty filter gets accepted by Solr
solrutils: fix for ranking result display

* Passes jrec and rg for id output format
BibRank: detect external word similarity ranker

* Detects if Invenio uses external word similarity ranker
solrutils: add search and ranking tests

* Adds unit and regression tests for search and ranking

* Amends existing BibRank and WebSearch unit tests to only test
  native word similarity ranker

* Merge note: amended kwalitee in the following files:
    modules/miscutil/lib/solrutils_unit_tests.py
    modules/miscutil/lib/solrutils_regression_tests.py

Tested-by: Tibor Simko <tibor.simko@cern.ch>
BibDocFile: support new magic library

* There are two flavours of the magic library (for file
  format guessing): the original bindings coming with the
  CLI file utilities, and the new python-magic bindings
  wrapping libmagic using ctypes. Support for the new
  library is here introduced (while still allowing for
  the old one).  (closes #1207)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-1.0' into maint-1.1

* maint-1.0:
  BibDocFile: support new magic library
Merge branch 'maint-1.1'

* maint-1.1:
  BibDocFile: support new magic library

Conflicts:
	modules/bibdocfile/lib/bibdocfile.py
global: test suite nosification

* Makes unit and regression test cases more friendly with respect to
  nose defaults.  That is, decorates functions that are not test cases
  as such, otherwise nose's defaults would consider them when running
  the test suite.  The nosification is useful for plugging test case
  output to Jenkins.

* Fixes all the codebase except WebSubmitFileConverterTestGenerator
  dynamically generated test cases in websubmit_regression_tests.py.

* Note that it is now possible to run unit tests like:
  `nosetests /opt/invenio/lib/python/invenio/*_unit_tests.py`
  and regression tests like:
  `nosetests /opt/invenio/lib/python/invenio/*_regression.py`.
  This gives equivalent results to `inveniocfg --run-unit-tests` and
  `inveniocfg --run-regression-tests`, respectively, modulo dynamically
  generated WebSumit tests mentioned above.

* Also, activates WebComment test cases that were accidentally
  commented out in 231caa6b80a41ad9a6b37f5bf7ddb142190742ed.
Merge branch 'maint-1.1'

* maint-1.1:
  global: test suite nosification

* Merge note: adds all tests from `refextract_unit_tests.py`,
  amends other files like `solrutils_unit_tests.py`,
  `weblinkback_regression_tests.py`, and
  `websearch_regression_tests.py`.

Conflicts:
	modules/miscutil/lib/testutils.py
	modules/refextract/lib/refextract_authextract_unit_tests.py
solrutils: clean unit and regression test suite

* Moves Atlantis test cases to regression test suite.
  (closes #1284)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
htmlutils: fix single quote escaping

* Fixes function escape_html() to translate single quotes into
  correct HTML entity; `&#39;` -> `'` instead of `&#34;` -> `"`.
Merge branch 'maint-1.1'

* maint-1.1:
  htmlutils: fix single quote escaping
htmlutils: improve js string escaping

* Moves/renames jsonutils.wash_for_js() function to
  htmlutils.escape_javascript_string() as this function would mostly
  be used to escape javascript in HTML context.
  Updates bfe_bookmark accordingly.

* No longer adds double-quotes around the returned string in
  order to be more flexible regarding the context in which the
  string is inserted.

* More complete escaping of Javascript chars when 'json' is not
  installed.

* Adds parameters to control the level of escaping, to be adjusted
  according to the context (HTML vs JS output, XHTML vs HTML,
  content-type, use of CDATA delimiters).

* Adds docstrings and unit tests.

* Merge note: fixed indentation in `htmlutils.py`.

Tested-by: Tibor Simko <tibor.simko@cern.ch>
mathpreview: js-based math preview panel

* JQuery plugin to display math rendering (via MathJax) preview
  next to input fields or textareas.

* Plugged for submissions, and enabled via CFG_WEBSUBMIT_USE_MATHJAX.
  (closes #1221)

* Enriches Submission Guide in consequence.

* Updates MathJax configuration to enable '$' escaping, with
  tex2jax->processEscapes=true, as recommended in MathJax doc
  when using '$' as inline delimiters

Tested-by: Tibor Simko <tibor.simko@cern.ch>
htmlutils: improve js string escaping

* Moves/renames jsonutils.wash_for_js() function to
  htmlutils.escape_javascript_string() as this function would mostly
  be used to escape javascript in HTML context.
  Updates bfe_bookmark accordingly.

* No longer adds double-quotes around the returned string in
  order to be more flexible regarding the context in which the
  string is inserted.

* More complete escaping of Javascript chars when 'json' is not
  installed.

* Adds parameters to control the level of escaping, to be adjusted
  according to the context (HTML vs JS output, XHTML vs HTML,
  content-type, use of CDATA delimiters).

* Adds docstrings and unit tests.

* Merge note: fixed indentation in `htmlutils.py`.

* Merge note: cherry-picked from
  86f6acbca4fe48a4af19940a6e6ab4344f5d0682.

Tested-by: Tibor Simko <tibor.simko@cern.ch>