pidutils: add pid normalize feature

* Adds support for normalizing a persistent identifier.
  E.g. http://dx.doi.org/10.1234/foo is normalized to
  10.1234/foo.

Signed-off-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
Reviewed-by: Jiri Kuncar <jiri.kuncar@cern.ch>
sherpa_romeo: error handling improvement

* Increases cache timeout in order to create a temporary db with
  journals and publishers.

* Handles parse errors in ElementTree, caused usually by bad formatted
  data returned from the service.

Reviewed-by: Jiri Kuncar <jiri.kuncar@cern.ch>
BibWorkflow: workflow model changes

* Renames WfeObject to BibWorkflowObject and changes the way
  these are created - now with strict dictionary format for API
  functions.

* Fixes display of modal windows for workflow and objects details.

* Moves JavaScript to separate files.

* Adds MIME type evaluation function.

* Changes data structure in a few places (e.g. new URI field).

* Fixes tests for new data structure.

* Adds database logging for workflow engine.

* Improves Celery worker plugin.

* Fixes import of WfeObject in WebDeposit. Changed to
  BibWorkflowObject.

Tested-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
BibWorkflow: add Holding Pen interface

* Adds a new Holding Pen interface that is going to replace the
  existing holding-pen functionality in OAIHarvest in
  the future. This new web-interface displays incoming objects
  (currently BibWorkflowObjects) for human oversight before being
  entered into the system, by Dinos Kousidis.

* Adds a new widget-support for Holding Pen in order to have custom
  UI Widgets related to workflow tasks that needs to be resolved by
  a human operator, by Dinos Kousidis.

* Adds and changes standard tasks provided by BibWorkflow to
  accomodate these changes, by Wojciech Ziolek.

* Adds new data-type property to BibWorkflowObject model, by
  Wojciech Ziolek.

* Adds object database logging, by Guillaume Lastecoueres.

Co-authored-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
Co-authored-by: Wojciech Ziolek <wojciech.ziolek@cern.ch>

Tested-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
BibWorkflow: API changes

* Changes BibWorkflow API functions from run() to start() and
  changes all sample tasks and workflows to accomodate this
  change, by Jan Aage Lavik.

* Renames workflow and object logging models and removes legacy
  or unused logging models.

* Several PEP8 and cosmetic updates.

Co-authored-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
BibWorkflow: `Workflow` API prototype

* Updates Workflow model function to get extra_data.
BibWorkflow: API updates

* The way run a workflow in a task queue has changed from a
  task_queue parameter into seperate "delayed" functions, such
  as start_delayed() or continue_oai_delayed().

* Adds custom kwargs to be passed to all workflow API functions
  that will be put into the workflow engine's extra_data column.
  This allows one to access this data from the workflow tasks.

* Documentation improved and corrected overall. Especially in
  bibworkflow_api.

* Updated worker plugins and unit tests to reflect these new
  changes.

* Renames several functions from camelCase to underscore_case.

* Renames Holding Pen template files from bibholdingpen_* to
  bibworkflow_* as well as other restructu.

* Adds a custom exception in bibworkflow_worker_engine instead
  of raising ValueError.

* Changes worker API names.

* Removes references to external_save as it is no longer used.

* Various fixes to the Holding Pen, improving PEP8 etc.

* Fixes many kwalitee issues.

* Adds Redis backend functionality, by Wojciech Ziolek.

Co-authored-by: Wojciech Ziolek <wojciech.ziolek@cern.ch>
BibWorkflow: updates of Holding Pen

* Updates the Holding Pen interface implentation with
  new structure.

* Several changes to BibWorkflow models.

Co-authored-by: Dinos Kousidis <konstantinos.kousidis@cern.ch>
Co-authored-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
BibWorkflow: workflows loading/naming

* Adds a plugin_builder in workflows loading so that the name of
  the workflow can be configured also inside the file and not only
  at the filename.

* Renames unit tests to regression tests.

Signed-off-by: Konstantinos Ntemagkos <konstantinos.ntemagkos@cern.ch>
BibWorkflow: PEP8 compatibility fixes

Signed-off-by: Konstantinos Ntemagkos <konstantinos.ntemagkos@cern.ch>
BibWorkflow: model relationship addition

* Adds SQLAlchemy relationship for already existing foreign key between
  BibWorkflowObject and Workflow.

Signed-off-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
WebDeposit: refactoring of data processing and ui

* Refactors field and form processing

* Adds view of past depositions.

* Adds tests further regression tests.

* Adds preingest and file functions.

* Adds perfill form from cache feature.

* Adds Hogan JavaScript templates for easier customization of
  JavaScript templating.

* Moves HTML for plupload widget into a template for easier
  customization.

* Dynamically generates subclasses of all WTForm fields and makes them
  available for WebDeposit.

* Removes unneeded integer field, since all WTForm fields are
  available now.

* Fixes issue where everytime a workflow is loaded multiple workflow objects are
  created.

* Fixes an issue where groups were not appearing according to the
  order defined in the form, due to the use of a dictionary instead of
  a list.

Co-authored-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
WebDeposit: class-based design refactoring

* Refactors WebDeposit to a class-based design to simplify data handling and
  allow easier customization.

* Adds storage abstraction layer for WebDeposit based on PyFilesystem.

* Adds file upload improvements including sorting of files.

* Adds webdeposit specific signals.

* Adds support for using field enclosures such as FieldList and FormField.

Co-authored-by: Konstantinos Ntemagkos <konstantinos.ntemagkos@cern.ch>
BibWorkflow: fix typo in worker import path

* Fixes a typo in the worker import path.

Signed-off-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
BibWorkflow: fix redis host configuration issue

* Fixes an issue where bibworkflow assumed that Redis was always
  located on localhost. Changed to use CACHE_REDIS_URL also used
  by Flask-Cache.

Signed-off-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>