0.9.17

Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>
Merge branch 'master' of github.com:dmwm/WMCore
0.9.18

Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>
Merge branch 'master' of github.com:dmwm/WMCore
0.9.19

Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>
Merge the Tier1PromptReco and PromptReco specs

They do the same thing, no need to mantain so much duplicated code.
I tested this against the T0 unit tests besides the usual WMCore tests.

Also added a few optional parameters to make it easier to setup
a testing environment to run the tests which involve skims

Fixes #3994
New splitting algorithm

The purpose of this algorithm is to dynamically split jobs by lumis
trying to make the jobs have a target number of events which is
ultimately the unit that determines job running time, disk space used
and memory constraints.
However this differentiates from the EventBased algorithm since
it doesn't split indiviual lumis, the basic unit for the splitting is the
lumi and every job is guaranteed to have at least one full lumi, this implies
that the number of events in a job will never be the exact target
but on average all jobs have about that number.
Also this includes the option to define a hard limit on the events
a job can process, if a file has a single lumi and more events
that such limit that job is created and marked as bad. This means
that this job will be moved from created to createdfailed by the
JobCreator.

The ErrorHandler is now configured to immediatly exhaust createfailed
jobs. The reason is that the createfailed state didn't make much sense,
the transitions defined it to go from createfailed to cooloff and then to
new. But a job that is in createfailed, had to actually be created otherwise
it wouldn't exist in WMBS. I changed the transitions to allow from created to createfailed and changed the transitions from createdcooloff/paused to new
for created instead which seems to be the logical step, since jobs in "new" state are not picked up by any piece of code.
However for now the createcooloff state is unreachable as the ErrorHandler will always take createfailed and send them to exhausted.

The algorithm takes the following inputs:

- events_per_job : This is the main parameter and defines the target events we want in each job. The algorithm will split lumis according to average events per lumi in each file
- max_events_per_lumi: Hard limit that defines jobs to fail when there are too many events in a lumi
- halt_on_file_boundaries, splitOnRun, etc..: Usual LumiBased algorithm parameters
Support for the new splitting algorithm in reqmgr

Reqmgr modifications to include the new algorithm
Bugfix for validation of well-formed dqmHarvest

Validation depends on having a self.procScenario in stdBase
this holds in all cases but ACDC workflows, this bug
prevents from creating ACDC workflows of requests
with DQM harvesting included through the webinterface (it is still
possible through script by passing a dummy 'scenario' parameter).

This patch changes the validation to check the steps in the harvesting task
and make sure the scenario was defined, this covers all workflows including
ACDC.
combine taskarchiver and cleanup manager to handle the case when reqmgr is not used
(need to be tested.)
Fix RequestManager unittests

- Add default performance parameters
- Make sure WMStats is deployed in couch during the tests
Removing extra slash from analysis spec #4131
Merge pull request #3958 from dballesteros7/fix-reqmgr-unittest

Fix reqmgr unittest
Remove debug printout from requestDb
Add ACDC integration test
Upload the toPublish.json with the user's credentials. Fixes #4135
Take the UI from the configuration instead of using the sourced one. Fixes #4141
Added status to the requestByDate monitoring view
Merge pull request #4011 from mmascher/monitoring-view

Added status to the requestByDate monitoring view
Merge pull request #4145 from mmascher/condor-proxyui

Take the UI from the conf instead of using the sourced one. Fixes #4141