aodn/harvesters — pull request #135 by lbesnard (2014-12-01 09:50:36+00:00)
Title: Argo harvester - Major update

* replaced tpostgres to ipostgres on all subjobs
* added tpostjob and close connection on all subjobs
* removed reporting contexts/connections and subjobs
* removed copy_of_data subjob which copy data from one table to legacy_auv schema
* removed context and connection associated to legacy_auv
* created AODN_views job to create the following views in the argo schema
(to replace the ones from the legacy_argo schema):
argo_float_abos_aodn_vw
argo_float_oxygen_aodn_vw
* made sure there was one subjob to process one file only at a time -> fixed
* change file_id to int in jobs
* in table platform_params , convert char varying(number) to text. There was
an issue with some NC files. sometimes the global att values were bigger than 256
* In the harvestArgoProfile subjob, there was an issue with the deletion of data.
"too many connections on argo ..." . fixed. more likely caused by a semi colone
missing at the end of each tpostgres_row
* cleaned multiple contexts, including some soop_something ones
* fixed Liquibase issues and dropped unused views
* change the order to delete files first and then process new ones
* add context.include which needs to be a new databag
* various cleaning

* This new version of the harvester will take some time to run since the
harvestType context was missing for the harvest of all profiles. This means
that from now the data in the database will be clean but it will take more time
in the first run to run since it has to 1)delete profiles associated to an
emtpy harvestType and 2)reharvest them