Handle files with multiple runs in Harvest

Modifiy the Harvest JobSplitter algorithm to handle multiple
runs in a file, it adds the run/lumi information to the job mask.

Note that this patch doesn't remove the special do-not-mix-runs
behavior of the agent, that will go in another patch which
will need https://github.com/dmwm/WMCore/pull/39 to
be applied.
Switch from dumpPython to pickle.

Fix some simple mistakes in PSet tweaking. Unit test for multicore
Fixes #4009
add unittest for couch view
log error message instead of stoping the process when request is not exist in wmstats
when status is updated
Fix unit tests for UnpackUserTarball_t

Fixes #3680
Fix MonitoringDAO BossAir test
simplify and cleanup DQMUpload code
Include RequestName in the Condor JDL

Changes:

- Modify the ListForSubmitter DAO to fetch the workflow name
- Pass the request name in the RunJob objects
- Include the line +WMAgent_RequestName in the condor JDL
Merge pull request #4035 from dballesteros7/fix-monitoring-dao-test

Fix MonitoringDAO BossAir tests.
Merge pull request #44 from dballesteros7/dqm-harvest-multirun

Handle files with multiple runs in the Harvest job splitting algorithm.
Merge pull request #4031 from ericvaandering/fixTarballTest

Fix unit tests for UnpackUserTarball_t.
Merge pull request #4023 from ericvaandering/dropDumpPython

Use pickle to write out configs instead of the unreliable dumpPython().  Cleanup white space and add a unit test for MultiCore config handling.  Fixes #4009 and #2847.
Create an inactive doc in the old active db and not in the actual active one.
WorkQueue: Handle open blocks.

Fix #3980

Periodically check open blocks for new files and inject into wmbs. Once
block closed in dbs close fileset in wmbs.

Signed-off-by: stuartw <stuart.wakefield@gmail.com>
fix some issues in the fwj report

When calling the constructor of the Report class with
a step name, it adds the step and initializes self.report
to point at the data for that step. Some other methods
to modify the report then work off self.report.

When unpersisting the report (ie. loading data from
a file and overwriting the internal data), self.report
will now no longer point at the internal data structures.
The modification moethds will still work, they just
modify some data that is no longer attached to the
class and will not be written out when the report is
persistet again.

Fix this by allowing to pass a step name to the unpersist
methos, it then initializes self.report to point at the
data for that step after loading the file. Also fix a
few places where the Report class is used to not pass
a stepname to the constructor if it's not used.
Merge pull request #4042 from hufnagel/fwj-report

fix some issues in the fwj report
add support for Alca Harvesting

Add AlcaHarvest step which goes through the analysis files
the CMSSW job wrote out and checks for sqlite condition files.
If it finds any, it does create metadata text files and copies
each condition file and it's corresponding metadata text file
to a special directory (POSIX path) and then adds both files
to the FJR under a fake 'Sqlite' output module.

Also make a change in StdBase to support output from
AlcaSkimming with data tier 'ALCAPROMPT'. This is needed
to support AlcaHarvesting.
have TaskArchiver accept missing working directory

Currently, the TaskArchiver will print an error message and send
an alert if it cannot delete the working directory for a task due
to that working directory not existing.

If the input for a task never gets any events, the task itself is
never initialized by the JobCreator and therefor it's working
directory is never created in the first place. This is normal
for these cases and not an error.

Downgrade the error message to a debug message and remove the alert.

While modifying this code anyways, make the consistency check more
specific: If the working directory does not exist, handle it in
the described way. If it exists, but is not a directory, bail out
with an exception because at that point something is screwed up.
Merge pull request #4046 from hufnagel/taskarchiver-workdir

Have TaskArchiver accept missing working directory.
Merge pull request #4045 from hufnagel/alcaharvest

Add support for Alca Harvesting.