oaiharvest: reliability improvement and docs

* Prints not only error logs for every workflow, but also
  everything with a superior code error (40) like critical.

* Removes useless part of code.

* Adds basic sphinx documentation syntaxes to various functions.

* Reactivates mails and tickets for harvesting process.

Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
docs: how to develop modules addition

* Adds some documentation on how to develop modules in Invenio
  v2.0.0. Still work in progress.

* Fixes typo in extensions.rst and introduction.rst.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
demosite: update demosite package for create/populate

* Changes from invenio_demosite to invenio_demosite.base

Signed-off-by: Javier Martin Montull <javier.martin.montull@cern.ch>
installation: MANIFEST template fix

* Fixes the MANIFEST template to include all jsonext configuration
  files.

Co-authored-by: Esteban J. G. Gabancho <egabancho@gmail.com>
Signed-off-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
search: templates inheritance schema change

* Makes index_base.html and results_base.html inherit from
  searchbar_frame.html instead of page base.html to share
  the search-bar.

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
search: typeahead 0.10 search bar adaptation

* Adds configurable parser.

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Reviewed-by: Javier Martin Montull <javier.martin.montull@cern.ch>
search: browse.html inheritance change

* Inherits searchbar_frame.html to get shared searchbar code.

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Reviewed-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
formatter: TypeError fix in record template

* Fixes TypeError in  render_record_footer in Default_HTML_brief_macros.tpl.

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Reviewed-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
search: code clean-up and documentation

* Improves code organization in helpers_base.html.

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Reviewed-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
search: browse button fix

Signed-off-by: Kamil Neczaj <kamil.neczaj@cern.ch>
Reviewed-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
docs: Ubuntu 13.10 setup

* Adapts VM setup from greut/invenio-vagrant.

* Fixes some formatting and typos.

Signed-off-by: Yoan Blanc <yoan.blanc@cern.ch>
Reviewed-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
registry: dict-style auto discover registry

* Adds dict-style submodule auto discover registry to support
  uses cases from workflows and jsonalchemy registries.

Signed-off-by: Lars Holm Nielsen <lars.holm.nielsen@cern.ch>
workflows: revamp session handling

* Changes the way the db.session is called in
  BibWorkflowEngine and BibWorkflowObject models
  to use a session_manager decorator.

* Adds new classmethods to BibWorkflowObject to create
  objects and revisions.

* New BibWorkflowObject's are now by default initiated
  with version set to INITIAL (from RUNNING).

* RUNNING version is only set when a BibWorkflowObject is
  actually being executed in a BibWorkflowEngine.

* Removes the workflow status FINISHED, as it was too
  ambiguous compared to COMPLETED. If a workflow contains
  data objects that have not reached FINAL version, the
  workflow status is set to HALTED.

* Cleans some of the log messages in the engine.

* Reworks what happens when a workflow is asked to *fully*
  restart. It will take the INITIAL version of every
  associated object and run it through the workflow from
  the start.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: sphinx documentation and more

* Adds sphinx documentation syntax to functions in the module.

* Fixes a problem with duplication of logger, the logger was
  a singleton but not the handlers which were accumulated each time
  we were continuing a workflow, producing database burst.

* Deletes some dead code.

* Fixes a mistake which was due to limitations in extra_data
  access.

Reviewed-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@techie.com>
workflows: add more unit tests

* Adds more unit tests with new tasks and workflows.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: update generic_harvesting_workflow

* Updates the generic_harvesting_workflow.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: update API and errors

* Workers now return directly, removing some preliminary code
  to close sessions.

* Adds new exception WorkflowAPIError to alert about API usage
  errors.

* Amends the way workflows within a workflow is initiated.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: revamp testsuite

* Implements a renewed testsuite for workflows with better coverage
  overall.

* Moves the workflows and widgets loading to use registry. Tests
  also use their own test registry.

* Test specific workflows are now moved to the testsuite.

* Cleans up the tests a bit, renaming and moving the tests in
  test_halt into the renewed test_workflows suite.

* Makes the tests visible for the overall invenio testsuite by
  adding the TEST_SUITE variable.

* Fixes an issue with the custom Log handler and inheritance.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: add more docs

* Adds basic documentation for API and Holding Pen view under
  api.rst.

* Adds docstrings in various modules to be included in general
  documentation.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: registry recreation in test setUp

Signed-off-by: Jiri Kuncar <jiri.kuncar@cern.ch>