Fix SiteDB service for resource control

It was broken for the call to include all T1s/T2s after
the  changes for SiteDB v2.
Avoid re-running performance killed jobs

Improve the exit code handling in the ErrorHandler,
retrieve a list instead of a single random non-zero exit code.
Compare the list against the exit and pass list.
Change the exit codes for performance kills to make them identifiable,
include them in the default configurable list or exit codes.

Fixes #4473
Include performance estimates in Condor JDLs

On the way to better provisioning, some of the splitters are able
to "predict" how long a job will take based on time per event estimates,
how much disk it will require based on estimates of output size per event
and how much memory the job will require. This information is included in
the condor JDLs in the following values:

+MaxWallTimeMins : Time in minutes, used already by CRAB2, coordinating with the FE admins if central production is already set to use it.
request_memory : Standard condor attribute to request memory during the matchmaking. It's in MB
request_disk : Standard condor attribute to request disk space during the matchmaking. It's in KB

The list of algorithms which support estimates follows:

EventAwareLumiBased, EventBased, LumiBased, FileBased: Provides estimates for all 3 values, this covers all Processing/Production/Skim tasks.
WMBSMergeBySize, ParentlessMergeBySize: Provide very accurate estimates for the disk space. This covers all merge tasks.

The tasks which provide no estimates are:

Harvest, Cleanup, LogCollect : All these tasks are very fast and with very little requirements.
We may tailor some defaults for them in the future.

The initial idea included changing the estimates for time per event, size per event
and memory dynamically using couchDB information. However after giving some thought
I realized that with the current schema this is not useful and can be harmful. The case
is when the second block is split after only fast jobs have run, these will have
low time per event and memory usage which will trigger bad pilots for future jobs
and won't allow for correction.

The estimates will be only reliable in a model where we first fully process at least a block
before splitting the rest of the workload. This would give some guarantee that the sample
used for the performance information is somewhat representative.

Fixes #4472
Catch empty lumi runs

Improve on 851acb4d94fc20d07d15b6aa3fe94d82c43aa2ed to catch
the cases where the runs dictionary is not empty but the lumi lists
are empty for a run in the report.
Merge pull request #4538 from dballesteros7/smarterErrorHandling

Avoid re-running performance killed jobs
Merge pull request #4537 from dballesteros7/fixCheckupLumis

Catch empty lumi runs
Merge pull request #4532 from dballesteros7/fixSiteDBForResourceControl

Fix SiteDB service for resource control
Merge pull request #4536 from dballesteros7/RunningTimePrediction-AgentLevel

Include performance estimates in Condor JDLs
0.9.58

Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>
changes to DQM harvesting

Change the Harvest job splitter to have two distinct modes, Periodic and EndOfRun that do not overlap.
In StdBase automatically add an additional EndOfRun DQM Harvest task if a Periodic DQM Harvest task is
specified. The EndOfRun task will trigger when the Periodic task is complete.
Implement a reasonable API to get the details - events, lumis, parents - of each file of a dataset

Conflicts:

	test/python/WMCore_t/Services_t/DBS_t/DBSReader_t.py
Merge pull request #4459 from hufnagel/harvestchanges

changes to DQM harvesting
Store correctly the SizePerEvent and friends

Make sure that SizePerEvent is stored correctly in CouchDB and Oracle
if present, it goes in hand with RequestSizeFiles and RequestNumEvents
but these two are not really present in any request schema
so they will be always zero, only SizePerEvent is mandatory.

Also fix the clone and assign of requests pre-upgrade.
Merge pull request #4540 from dballesteros7/straight-out-params

Store correctly the SizePerEvent and friends
Fixed typo in test decorator. See pull #4539
Merge pull request #4539 from bbockelm/crab3

listDatasetFileDetails API for CRAB3
Fix broken DBS3Buffer creation in Oracle

It was broken in 0.9.58, just a typo.
Merge pull request #4543 from dballesteros7/fix-oracle

Fix broken DBS3Buffer creation in Oracle
0.9.59

Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>
Add Jenkins bits to git

Currently, we have a ton of scripts in Jekins input manually, this
is a bit of a hassle since any changes need to be synced a few times.
I'm going to begin to strip the commonly used functions into shell
scripts stored in WMCore's git repository, so updating there will change
everything else simultaneously