base: render field enhancement

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
groups: account settings fixes

* NOTE: please upgrade *SQLAlchemy-Utils*
  $ pip install --upgrade SQLAlchemy-Utils<0.24.0

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
utils: etree to dict translation reorganization

* Adds xmlhelpers to utils to handle custom XML.

Signed-off-by: Pedro Gaudêncio <pedro.gaudencio@cern.ch>
Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
utils: import json from arxiv api

* Adds API to import JSON data from http://export.arxiv.org with a
  given ArXiv ID.

Signed-off-by: Pedro Gaudêncio <pedro.gaudencio@cern.ch>
Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
arxiv: initial Flask extension commit

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
records: API for database querying with DOI

* Adds API to query the database with a DOI.

Signed-off-by: Pedro Gaudêncio <pedro.gaudencio@cern.ch>
Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
Reviewed-by: Jiri Kuncar <jiri.kuncar@cern.ch>
travis: less verbose output

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
installation: requirement addition for six library

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
arxiv: docs entry addition

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
docs: css theme overrides

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>
registry: keygetter value fix

* `register` method of `DictModuleAutoDiscoverySubRegistry` class
  gives original value of parameter `value` to the keygetter method
  as well as the modified value from valuegetter.

* NOTE If your registry.py is using DictModuleAutoDiscoverySubRegistry
  and you use your own keygetter method then please update your
  keygetter method to use all three parameters.

Co-authored-by: Jan Åge Lavik <jan.age.lavik@cern.ch>
Co-authored-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
Signed-off-by: Graham R. Armstrong <graham.richard.armstrong@cern.ch>
workflows: registry fix for keygetter method

* `keygetter` method of WorkflowsRegistry updated for keygetter fix.

Co-authored-by: Jan Åge Lavik <jan.age.lavik@cern.ch>
Signed-off-by: Graham R. Armstrong <graham.richard.armstrong@cern.ch>
jsonalchemy: move to `isinstace(foo, Mapping)`

* Improve type checks to use `collection.Mapping` instead of `dict` as
  this the recommended way and it gives more abstraction.

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
jsonalchemy: preserving original tags inside JSON

* When using creator rules that involves regex like `245..` the original
  marc tag is preserved inside the `__meta_metadata__` key for posterior
  needs.

* Enhances `json_for_marc` producer to use the input tag if the
  producer is defined to do so (See `json_for_marc` documentation).
  (addresses #1722)

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
records: move legacy methods to the Record object

* Moves `legacy_export_as_marc` and `legacy_create_recstruct` from the
  `SmartJson` class to the record class.

* Enhances test cases for this two methods.

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
jsonalchemy: fix for `__additional_info__` access

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
workflows: abstraction layer for workers

* Adds an abstract class to be used like a base for
  worker wrapper.

* Implements a worker wrapper for abstraction for
  celery worker.

* Starts the implementation of a worker wrapper for
  redis worker.

* Fixes PEP8 and sphinx documentation specification.

* Fixes mistake about iteritems with six.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: more unit tests

* Adds more unit tests to the workflows module, including
  new test workflows. Coverage should now be about 50%.

* Modifies API internals for unit tests conformation.

* Adds one file for tests which are using asynchronous
  worker to compute.

* Corrects unit tests.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: global improvement

* Updates workflow to comply with the modification in
  tasks librairies.

* Fixes an error with the new ObjectVersion type with
  python3.

* Fixes sphinx mistakes.

* Removes useless regex.

* Fixes PEP8 mistakes.

* Adds more documentation to some functions.

* Removes useless and obsolete code, factorize some parts.

* Changes the start workflow function to give the
  possibility to start from a new clean BibWorkflowObject.

* Adds the refresh frequency as a parameter for workflows
  synchronisation.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
oaiharvest: PEP8 and errors category

* Fixes PEP8 errors in a strict way.

* Allows log of errors of loglevel=40 in oaiharvest.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>