records: aggregation field definitions fix

Signed-off-by: Esteban J. G. Gabancho <egabancho@gmail.com>
workflows: move persistent_ids to extra_data

* Moves the persistent_ids column into an extra_data field.

* Changes tasks appropriately.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: new models
oaiharvest: reliability improvement and sphinx docs

* Prints not only error logs for every workflow, but also
  everything with a superior code error (40) like critical.

* Adds sphinx documentation syntaxes to various functions.

* Reactivates mails and tickets for harvesting process.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: sphinx documentation, PEP8, Bug fixes

* Adds sphinx documentation.

* Adds a security about data synchronization between,
  celery worker and the main program, by closing th db
  session, celery-side. It fixes the problem of not
  getting the last data due to SQLAlchemy transaction
  policy.

* Fixes a bug about the duplication of logger,
  the logger is well a singleton but not the handlers
  which were accumulated each time we were continuing a
  workflow, producing database burst.

* Deletes useless code.

* Fixes a mistake which was due to limitations in extra_data
  acces.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@techie.com>
workflows tasks: new tasks, bug fixes, sphinx, PEP8

* Adds sphinx documentation.

* Fixes a bug for the foreach loop, about caching in case
  of non function parameter.

* Adds a function to do simple comparison between two data.

* Adds the function set_obj_extra_data_key which allows
  to manipulate extra_data of an object.

* Adds get_data function which allows to get the data of the
  object.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: unit test updates

* Adds more unit tests with new tasks and workflows.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows workflows: up-to-date generic_harvesting_workflow

* Updates the generic_harvesting_workflow.

Signed-off-by: Guillaume Lastecoueres <guillaume.lastecoueres@cern.ch>
workflows: updates
workflows: important fix for model
workflows: tmp fixes for test
workflows: final object creation changes
workflows: revamp testsuite

* Implements a new testsuite for workflows with better coverage
  overall.

* Moves the tests in test_halt into a new test_workflows suite.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: engine and model changes

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
workflows: unit test updates
workflows: revamped engine and models

FIXME
workflows: fix workflow task
workflows: add docs

* Adds basic documentation for API and Holding Pen view under
  api.rst.

* Adds docstrings in various modules to be included in general
  documentation.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>