batchuploader: import fix

* Fixes invalid import statement.  (addresses #1779)

Reported-by: Petr Broz <petr.broz@gmail.com>
Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
global: fix legacy static files includes

* Addresses issue with broken style and javascript on several legacy
  pages.  (addresses #1777)

Reported-by: Petr Broz <petr.broz@gmail.com>
Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
base: PEP8 and PEP257 clean-up in factory

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
base: fix for wsgi PATH_INFO handling

* Addresses issue with url encoded arguments in Flask routes. E.g. for
  route `/collection/<name>` the view argument `name` was not unescaped
  when accessed by `/collection/Books%20%26%20Reports` on Apache and
  this led to error during lookup in database.  (closes #1823)

Reported-by: Petr Broz <petr.broz@gmail.com>
Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
search: fix browse pagination links

* Addresses issue with browsing results using next/prev links without
  field name.  (closes #1824)

Co-authored-by: Petr Broz <petr.broz@gmail.com>
Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
installation: mercurial addition

* Adds mercurial to the list of dependencies for installation
  in INSTALL guide after 787b62e8.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
deposit: assets import clean-up

* Changes js and css import to use url_for.  (closes #1817)

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Co-authored-by: Jiri Kuncar <jiri.kuncar@cern.ch>
Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
assets: Flask-Assets update to 0.10.dev

Signed-off-by: Yoan Blanc <yoan.blanc@cern.ch>
assets: requirejs and uglifyjs

* Enables two more filters.

* Enables modular JavaScript applications with AMD structuring.

Signed-off-by: Yoan Blanc <yoan.blanc@cern.ch>
jsonalchemy: hotfix for optional fields

* Fixes an unexpected behaviour which happen when a field in the
  conversion sheet is not present in the marcxml (e.g. email).  When
  marcxml is transformed to dict everything is fine, but when it is
  converted back to marcxml, it will get into an infinite loop because
  the field is not present in the dict. Hence it will try to eval the
  field and it will end in an infinite import loop.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jiri Kuncar <jiri.kuncar@cern.ch>
jsonalchemy: unit tests for module import fix

* Adds a field to fields.cfg, which is a name of module.

* Adds a test that will try to convert a marcxml to json
  and then from json to marcxml. In this test email is
  a field which is not present in the marcxml, and then
  bibfield will try to import it like a module.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jiri Kuncar <jiri.kuncar@cern.ch>
base: requirejs bundle ordering fix

Signed-off-by: Yoan Blanc <yoan.blanc@cern.ch>
base: demosite cleanup

* Prevents the demosite command to load the testsuite init script.

Signed-off-by: Yoan Blanc <yoan.blanc@cern.ch>
formatter: type check of recid in BibFormatObject

* Checks type of recid when creating a BibFormatObject. If the recid
  is not passed as integer, it will cast it and alert the admin.

* Cherry-picked from d66c87f54413f8c871749e86a0f41c50d2d645d5
  inspirehep/ops/prod.

Signed-off-by: Javier Martin Montull <javier.martin.montull@cern.ch>
Reviewed-by: Jiri Kuncar <jiri.kuncar@cern.ch>
formatter: int or long type cast of recid

* Converts recid to int when it is taken from the marcxml.

* Cherry-picked from c614bcc7fe22531b11c68ac29aca73bc6fb73169
  inspirehep/ops/prod.

Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
Acked-by: Jiri Kuncar <jiri.kuncar@cern.ch>
workflows: abstraction layer for workers

* Adds an abstract class to be used like a base for
  worker wrapper.

* Implements a worker wrapper for abstraction for
  celery worker.

* Starts the implementation of a worker wrapper for
  redis worker.

* Fixes PEP8 and sphinx documentation specification.

* Fixes mistake about iteritems with six.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: more unit tests

* Adds more unit tests to the workflows module, including
  new test workflows. Coverage should now be about 50%.

* Modifies API internals for unit tests conformation.

* Adds one file for tests which are using asynchronous
  worker to compute.

* Corrects unit tests.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: global improvement

* Updates workflow to comply with the modification in
  tasks libraries.

* Fixes an error with the new ObjectVersion type with
  python3.

* Fixes sphinx mistakes.

* Removes useless regex.

* Fixes PEP8 mistakes.

* Adds more documentation to some functions.

* Removes useless and obsolete code, factorize some parts.

* Changes the start workflow function to give the
  possibility to start from a new clean BibWorkflowObject.

* Adds the refresh frequency as a parameter for workflows
  synchronization.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
oaiharvest: PEP8 and errors category

* Fixes PEP8 errors in a strict way.

* Allows log of errors of loglevel=40 in oaiharvest.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: overhaul update

* Adds more documentation.

* Fixes some sphinx errors in documentation.

* Changes imports to relives one when possible.

* Adds a new way to handle errors, to allow easy reading.

* Removes obfuscations of exceptions.

* Clarifies exceptions.

* Updates the way exception unicodes are handling.

* Fixes the behavior of stop_on_error.

* Adds Python 3 compatibility where possible.

* Adds a method to clean extra_data in a Workflow Engine object.

* Fixes problems with Python 2.6 loggers.

* Adds reliable way to handle types for BibWorkflowObject.

* Adds a method to get widget message easily.

* Adds an abstraction method for jsonalchemy conversion in
  workflow tasks.

* Adds more tests.

* Adds start test procedure.

* Fixes bad sorting instructions in Holding Pen.

* Improves the way of restarting workflows.

* Simplifies celery worker.

* Reorganizes tree of the module.

* Tasks cleaning and optimizations.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>