Squashed commit of the following:

commit e3cae2bdf053530f84e3df73f822bc7c7405a9be
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Jan 30 15:26:38 2013 -0600

    check components are alive using pid and also thread status

commit e6fac9d603eaa882c8830b28a7913b37d56269b3
Merge: 5d42249 5fc3e3d
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 30 11:42:14 2013 -0800

    Merge pull request #4379 from zdenekmaxa/reqmgr-custodialsites-bugfix

    ReqMgr CustodialSite bug fix.

commit 5fc3e3d6d2fbb61f9355dad356d9763644dcdfd2
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Jan 30 16:30:02 2013 +0100

    ReqMgr CustodialSite bug fix.

    CustodialSite is an argument not used for long. (Non)CustodialSites have
    superseded this, however no one bothered to remove all dead functionality and
    parameter definition. On cloned requests, the default value (None) of
    CustodialSite request parameter appared and PhEDExInjector was failing on
    that. This patch removes CustodialSite from ReqMgr only.

commit 5d4224997378fa1778e79d59acd7dc7f97594b93
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Jan 29 16:23:38 2013 -0600

    0.9.39

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit be297dcd343990d82b1b1051af0b099090b06128
Merge: faf418a aeb4645
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Jan 29 14:20:33 2013 -0800

    Merge pull request #4371 from hufnagel/runcompletedqm

    fix cmsrun config for endofrun dqm harvesting

commit faf418a34c3fa032eec3cc4c1c1df6f01a0351a4
Merge: 10dbe1c eb9c657
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Jan 29 14:20:04 2013 -0800

    Merge pull request #4377 from zdenekmaxa/reqmgr-clone-prioritybugfix2

    ReqMgr cloned requests priority fix

commit eb9c65720253f722c3872427ad994aca9e07df89
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Jan 28 17:59:41 2013 +0100

    ReqMgr cloned requests priority fix

    When cloning a request which had changed priority, the newly cloned request
    priority is on default values. Fixes goes via MakeRequest.createRequest() to
    database DAOs Request/New.py
    Also reqmgr.py checks the case.

    Modified default value SoftTimeout on Ops request.

commit 10dbe1ca03c8a31c2c64642c082363c83875e407
Merge: d7a3d13 694b379
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Jan 29 08:20:07 2013 -0800

    Merge pull request #4374 from ticoann/tier0_state_warning

    add status duration

commit aeb4645a5395a3e252247e2009192f7815c61197
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Mon Jan 28 16:44:44 2013 +0100

    fix cmsrun config for endofrun dqm harvesting

    EndOfRun DQM harvest jobs need a special cmsRun configuration.

commit 694b3798b0a894c6c7172e2381f6a55faf069e59
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Mon Jan 28 16:03:11 2013 -0600

    add status duration

commit d7a3d138ba63507658109b582770de2c93876ff9
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 15:34:15 2013 -0600

    0.9.38

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit e186453679fbdd4a611ab6eb1b3b817739821291
Merge: 6fcb77f a61239f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 13:30:28 2013 -0800

    Merge pull request #4369 from dballesteros7/fix-WorkQueue-lumicount

    Fix Block policy accounting of lumis

commit 6fcb77f2474b768a7ca6112e1f880cd67f9bf816
Merge: cfed687 9c46dd9
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 13:29:52 2013 -0800

    Merge pull request #4370 from ticoann/add_queue_column

    add queue column in the job number table view

commit 9c46dd9331509d3dcce572d2f67a2807b6dcd875
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Jan 25 15:09:24 2013 -0600

    add queue column

commit cfed687fbc58c4c478a134b1ad2a3721541756ee
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 15:06:21 2013 -0600

    0.9.37

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit e090caf716f71ecc2853c0221dff0f12692b095d
Merge: a852056 354ae40
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:49:02 2013 -0800

    Merge pull request #4359 from ticoann/remove_t0wmstats_dependency

    remove t0wmstats dependencies - just use reqmon to create

commit a852056c49ca61808ffbf8f71e79af97e22d08fa
Merge: a58ac10 c52a14c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:48:47 2013 -0800

    Merge pull request #4360 from dballesteros7/fix-tier0-plugin-races

    Fix another possible glitch in the Tier-0 plugin

commit a58ac10ab120b0d480c9024d187f937777b34d0c
Merge: 39db917 feb0723
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:47:52 2013 -0800

    Merge pull request #4361 from dballesteros7/Fix-EventBased-Tails

    Fix EventBased algorithm dropping zero event lumis

commit 39db917a0a128a3c6b32972a567131dd569521df
Merge: 2110ca5 928dd90
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:47:30 2013 -0800

    Merge pull request #4362 from ticoann/replication_retry

    retry replication when it fails

commit 2110ca5d9a2ca1afe58e60be30c8135ead5547f2
Merge: 8f36704 7622553
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:47:09 2013 -0800

    Merge pull request #4365 from ticoann/t0_operator_permission

    add t0-operator permission for wmstats

commit 8f36704550f719486c46cb04ea6c81a99de1d871
Merge: 8cf925b 606862c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:46:20 2013 -0800

    Merge pull request #4367 from ticoann/prep_id_fix

    fix prep id search and make detail page not draggable

commit 8cf925ba2bc02ae7693b0530e6006ab5dbe84af0
Merge: 4262559 75e9737
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:43:58 2013 -0800

    Merge pull request #4368 from sfoulkes/run_whitelist

    Make sure the input run white list is passed to the job splitting

commit a61239ffe1ed859d39b485c96df110ecc8d09f44
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Jan 24 16:34:46 2013 -0600

    Fix Block policy accounting of lumis

    When a run white/blacklist is present, the accounting of lumis is screwed up
    and it only reports a number of lumis equal to the number of runs accepted in a block.

    Unfortunately, DBS doesn't return block level information on runs and lumis
    when calling the listRuns function. So the solution is the following:

    - Check first with listRuns and apply run black and whitelist.
    - If there is a single run in the block, and the block passed then no need to recalculate lumis.
    - However if the block has more than one run, and the list of blocks in the run was affected
      by the run white/blacklist then recalculate the event/file/lumi counts by pulling full file
      information from DBS. This is expensive but we expect the case to be rarely present.

commit 75e973759951ac44970cf967583445b72561fcbb
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 14:40:12 2013 -0600

    Improve the code that passes run white and black list to the job splitting
    parameters so that it doesn't break other things.

commit 4262559d92516a2a25794d0736246703e223eacf
Merge: f2080b3 99b0f34
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 12:29:03 2013 -0800

    Merge pull request #4358 from sfoulkes/parallel-dbs3

    Fix up the DBS3Uploader and modify it to support an upload only mode.  K...

commit 99b0f34ee6d5e9e2a29d4ad5659cd32cb2d096e7
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 14:27:58 2013 -0600

    Fix issues with Oracle and DBS3Upload.

commit 91683197d3925dc4745cb3107214ec5be3e94847
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 25 10:12:05 2013 -0600

    Make sure the input run white list is passed to the job splitting
    algorithm.

commit 606862c5155deeee491d84535aacdbe095d54219
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Jan 24 15:54:12 2013 -0600

    fix prep id search and make detail page not draggable

commit 76225534924e3cc2aa9814330305f2f9c3802b6a
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Jan 24 13:22:26 2013 -0600

    add t0-operator permission

commit 928dd90313d9da924b17f906ad1b439305d20b76
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Jan 24 11:20:35 2013 -0600

    retry replication when it fails

commit feb0723145854127e2e214a5df1022e8448ac75d
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Jan 23 23:55:02 2013 -0600

    Fix EventBased algorithm dropping zero event lumis

    The current mask definition performed by the EventBased splitting
    algorithm on non-mc input files can lead to missing lumis in the output.
    This occurs when an input file has zero events in the last lumis, because
    the cmsRun process will stop once it has reached the max events and won't process
    those eventless lumis. To fix this, avoid passing the maxEvents parameter
    when configuring the mask of the last job in a file, the skipEvents parameter
    will suffice and the job will run over the remainder of the file.

    Many assertions changed in the unit tests, since we change from having
    always a max events to having it only on intermediate jobs in a file.

commit 354ae40a1179f8bc02af0a0a6a15a3717c1b1f14
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Jan 23 13:45:32 2013 -0600

    remove t0wmstats dependencies - just use reqmon to create

commit e6988496fd2be7410c32eaa8a14a22055e1686b7
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 23 09:22:35 2013 -0600

    Fix up the DBS3Uploader and modify it to support an upload only mode.  Kill
    off SQLite in DBS3Buffer and DBSBuffer.

commit f2080b3ca2b54c0af756462c113f693b2c0e15bb
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 23 09:09:27 2013 -0600

    0.9.36

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit aab5e088feebc1432c97739b6a5fef17a38ad116
Merge: bb56bf8 083857c
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Jan 23 06:02:18 2013 -0800

    Merge pull request #4357 from zdenekmaxa/reqmgr-feb-opsadditions

    Reqmgr feb opsadditions

commit 083857ccc0e43773fa7fe18e5127f5a4031f8753
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Jan 23 14:47:41 2013 +0100

    ReqMgr - PhEDEx subscriptions,cloned request state

    - PhEDEx subscriptions arguments added into JSON request templates
    - cloned requests will not have new state but assignment-approved

commit bb56bf8c9953689dfcd878203ec7802dc9dd9207
Merge: 2be5654 95da3bd
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Jan 23 05:43:10 2013 -0800

    Merge pull request #4355 from dballesteros7/flexible-ReReco

    Allow keeping ReReco outputs unmerged

commit 95da3bd42a6e8b11567ecf39a6220b6781fa003b
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Jan 18 17:07:35 2013 -0600

    Allow keeping ReReco outputs unmerged

    Add the option to the ReReco spec to keep some output module unmerged
    but still run skims on it if necessary, if no skim is defined for it
    the output will be produced, stored in unmerged area and cleaned up
    afterwards.

commit 2be5654eac4ba621245514ea959fc460507bb5cb
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Sun Jan 20 11:50:50 2013 -0600

    0.9.35

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 64cfdc0b534724f13e2d295c0e9c79f3956a9653
Merge: bc67672 d66df52
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 18 14:27:54 2013 -0800

    Merge pull request #4354 from dballesteros7/changePhedexSubscriber

    Add an option to the Phedex subscriber

commit d66df52dee894e4cc046fff612c2f731eb09652d
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Jan 18 15:39:34 2013 -0600

    Add an option to the Phedex subscriber

    Add the option to make only replica subscriptions, this
    for agents which don't really require move subscriptions
    such as those only dealing with T0 or T1s

commit bc67672266a502688bab123fef72812e5b14a10e
Merge: 71f5e6c 3fb456f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 18 13:37:12 2013 -0800

    Merge pull request #4353 from ticoann/running_open_update

    fix bug for updating data for running-open and running-closed status

commit 3fb456fdf7e6c87f9a79d0267f958806c4d5aae8
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Jan 18 11:28:06 2013 -0600

    fix bug for updating data for running-open and running-closed status

commit 71f5e6cb53c572f132a4cd90856877b5f0d14362
Merge: f1e7c86 7120dec
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 18 07:19:16 2013 -0800

    Merge pull request #4352 from ticoann/pull_request_feb2

    wmstats updates for feb, 2013 deployment

commit f1e7c86c19c58ee629935eeeb4037515eb71cfc3
Merge: 41ca82a dd94f64
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Jan 18 07:18:59 2013 -0800

    Merge pull request #4350 from hufnagel/newdropboxupload

    changes to alcaharvest runtime code to support new dropbox

commit 7120decca4277ebb93e5884e2b735dbf1ca3327c
Merge: bf8035e 32bf6c3
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Jan 17 11:41:04 2013 -0600

    Merge branch 'report_couch_replication' into pull_request_feb2

commit 32bf6c3ce0e20b610cb56c394115beb57251ff5f
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Jan 16 16:31:55 2013 -0600

    add warning for couch server status

commit 6f7cdadd06cdbec290746007757115d70754ce97
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Thu Jan 17 09:34:43 2013 +0100

    Merge branches 'upmaster' and 'reqmgr2-databasecleanup'

commit bf8035eec188dd081186f980650004564ded362e
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Jan 16 15:19:01 2013 -0600

    Squashed commit of the following:

    commit 154b26deff714575f1abbdac86d2de67441147cd
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Wed Jan 16 15:16:26 2013 -0600

        add job number views for request view

    commit b6820727939f58f4c236b54573226b0904da8f47
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 11 16:58:02 2013 -0600

        cleanup view related to analysis use case

    commit 87ae24252e6d050e3c477a7edd067904529db640
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 11 16:56:54 2013 -0600

        search in wmstats instead of workloadsummary
        add prepid serarch
        fix reduce error on performanceSummaryByTask

    commit 7505adea29c5f676cbd44b4d57556fec816c0502
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 11 10:41:16 2013 -0600

        view switch

commit 41ca82a853325f94865f8129a89a11c7666db001
Merge: d525feb 14700df
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Jan 16 10:54:36 2013 -0800

    Merge pull request #4351 from zdenekmaxa/reqmgr2-databasecleanup

    ReqMgr2 - Oracle, CouchDB database cleanup

commit 14700df6ed6e1b299712cee53558fdc0cd61031c
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Jan 16 19:45:53 2013 +0100

    ReqMgr2 - Oracle, CouchDB database cleanup

    -utils for Oracle dump, Oracle vs. CouchDB consistency comparison
    -CouchDB requests cleanup
    -temporary logs and notes, all helpful before dropping Oracle for ReqMgr2

commit dd94f6484dc6c6f261179acaa81ef1af3c47401c
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Fri Jan 11 11:04:28 2013 +0100

    changes to alcaharvest runtime code to support new dropbox

    The new dropbox expects a different metadata format for
    uploaded payloads. TO produce that format the format of
    the information returned in the analysis files section
    in the framework job report has been changed. Make the
    needed code changed to deal with that new format.

commit d525febdd8c525e8594512c61d38bd50fc5ae9be
Merge: 2b49c88 aa7a866
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Jan 10 14:03:00 2013 -0800

    Merge pull request #4348 from dballesteros7/fixHarvestDuplicate

    Fix Harvest splitter creating duplicate jobs

commit 2b49c88fb3b5ac1f7d729894cd63d46f6d1339bd
Merge: 479c1b1 67c268f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Jan 10 14:02:39 2013 -0800

    Merge pull request #4349 from dballesteros7/fixCERNStagein

    Fix EOS StageIn at CERN

commit 67c268f7eb1246eed5aa6a735376e69cba39d2dc
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Dec 7 10:56:02 2012 -0600

    Fix EOS StageIn at CERN

    Plugin was using rfcp to stageIn files from EOS at CERN.
    Check properly if remotePFN is EOS, instead of targetPFN
    as in stageIn the targetPFN will be the local.

    Also fix deletions, in stageIn case it was going to
    delete the remotePFN instead of the faulty local
    copy. Include deletions for failed CASTOR stage-outs.

    Finally, only create directories on remote PFNs.

commit aa7a8669492a4aa5ef1721da8bff27743b46b2c6
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Jan 9 23:25:46 2013 -0600

    Fix Harvest splitter creating duplicate jobs

    This because of multiple SEs for a single site.

commit 479c1b1c9d1da197bb58cbcce4e9bf6d7903e376
Merge: 1c64de1 bb34e7c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 9 11:37:38 2013 -0800

    Merge pull request #4347 from dballesteros7/fixMergeDuplicate

    Fix WMBSMergeBySize causing duplicates

commit bb34e7c8f4641191a048574236c188c233a9fed7
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Jan 8 13:00:58 2013 -0600

    Fix WMBSMergeBySize causing duplicates

    This occurs when a site has multiple SEs, fix
    changes the discovery query to return a file only once
    with a tuple of the possible SEs.

commit 1c64de1072504c524c57d5954c445395b8be663f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 9 09:27:29 2013 -0600

    0.9.34

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit abf282481bd03728345066aa85ef5f1a3920941d
Merge: d076564 5cd9564
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 9 07:03:29 2013 -0800

    Merge pull request #4339 from PerilousApricot/bugfix/zipimport

    Fix sandbox creation

commit d0765642d0921d935d304a05c65754282c2377d7
Merge: a8da4ac 0596ed1
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 9 07:02:51 2013 -0800

    Merge pull request #4340 from giffels/restapi-docstrings

    Display docstrings in RESTAPI documentation

commit a8da4ac3d680d937b8c46ab2501b5f088edfcc46
Merge: 9f60a15 5d5e845
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 9 07:02:18 2013 -0800

    Merge pull request #4342 from dballesteros7/fix-errorSite-hash

    Fix issue with site parameter in error summary

commit 9f60a151e19eb29edd9da9adb562605ebc300ad1
Merge: 8f5e565 04abda2
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Jan 9 07:01:47 2013 -0800

    Merge pull request #4345 from ticoann/pull_feb_deploy_1

    major wmstats change for feb deployment

commit 04abda2e80a0e20184b431fba7f9e4fee8cee17e
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Tue Jan 8 14:46:20 2013 -0600

    add running-open and running-closed status and update event, lumi function.
    add workload summary rewrite rules.

commit fe4f6830d18440121d6ef98ab6e9165ca7086f72
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Mon Jan 7 17:18:20 2013 -0600

    add basic search page

commit a3298d0f2215ba3048f4679765f7e51704efc90d
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Jan 4 16:22:48 2013 -0600

    Squashed commit of the following:

    commit 37505121c613f5c03821d2814931c0e488e872a5
    Merge: d9a63f4 8599684
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 4 15:46:26 2013 -0600

        Merge branch 'fix_agent_info_update' into rpm_test

        Conflicts:
        	src/python/WMComponent/AnalyticsDataCollector/AnalyticsPoller.py

    commit d9a63f49a03f5a9ab74fff0a475bdf52243e6ab2
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 4 15:43:08 2013 -0600

        add couch server alert

    commit 85996843562d4a34a183742d9324f876e9dc2bf2
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 4 11:21:32 2013 -0600

        directly upload agent info to remote wmstats to prevent conflict when agent redeployed

    commit 900174f6fdd8b022d364d01224a184dbcbc22109
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jan 4 11:17:36 2013 -0600

        refactor Controls view and fix UTC

    commit 1b328477033283a056c5156a547292dcffbd4a89
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Wed Jan 2 10:43:09 2013 -0600

        fix average event and lumi calculation

    commit 61a873f549cff765880118ff32fef79523de7c67
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Mon Dec 31 11:54:34 2012 -0600

        lumi and event progress.
        need to find how to handle the same out put dataset names for different workflow.

    commit 31e875136da6038a3d3ec98451ea0d9d15f1bc06
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 28 13:59:07 2012 -0600

        lumi based progress bug fix and refactor request struct

    commit f292faf502aa888e8471112789425be3684b930a
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Wed Dec 19 11:14:00 2012 -0600

        hard code summary link

    commit 95162066eb99fb546c527293bc4d7d85ec9b4617
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Tue Dec 18 16:53:21 2012 -0600

        add minified version

    commit 2a8a021ac8aae219adb4defebee2ccb3433a9705
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Tue Dec 18 16:52:14 2012 -0600

        12/18 vocms236 test version

    commit 63a96526b30aaabb2c8c63f099157e5476872d83
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 14 17:07:47 2012 -0600

        fix site missing error for display this is temporary patch to handle old wmagent interaction

    commit 1904760f02e8f44b13664184e122cbacaa8f6a61
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 14 16:20:36 2012 -0600

        hard coded value

    commit b7beba9d43f6708e6087b6d40ed8ed46a90ef1ab
    Merge: efdfcaf 97ecf83
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 14 14:01:46 2012 -0600

        Merge branch '2012-12-4-merge' into 2012-12-14-merge

    commit 97ecf837a1740637de65ab5ff83f3e57cd89d25f
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 14 13:55:35 2012 -0600

        add search page work in progress

    commit ffd317715345ae06cf57141c22466769256712eb
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 14 13:51:52 2012 -0600

        add total lumi calcuation

    commit 1c67f0b23458771de3dc81f0d7ffe183fdc577d7
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Dec 13 15:53:32 2012 -0600

        fix typo

    commit b0d3b5d534a0d2cf1d83bc0512563b97550defc9
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Dec 13 15:36:37 2012 -0600

        enhance the loaing time

    commit 03fb87b883ce27ac281435deb640aeb67733535d
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Dec 13 15:35:28 2012 -0600

        add file merge status in the job report

    commit dae5e248c071dc70c86741e589ff55d4aea335ca
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Dec 7 15:23:18 2012 -0600

        remove none used view

    commit a49ed0e6d30c8178f511651017f48e524569f2ef
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Tue Dec 4 16:59:01 2012 -0600

        Conflicts:
        	src/python/WMComponent/AnalyticsDataCollector/AnalyticsPoller.py
        	src/python/WMComponent/AnalyticsDataCollector/DataCollectAPI.py
        	src/python/WMCore/Services/WMStats/WMStatsWriter.py

commit c52a14c5d96d2dfc8decfb51f0920d730ad64b42
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Jan 2 13:38:39 2013 -0600

    Fix another possible glitch in the Tier-0 plugin

    normal-archived and aborted-archived are final status and therefore
    no transition from them should be admitted.

commit 5d5e845ce30b50bd2d5b6fe4ef9d0d27cf29ec20
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Jan 2 08:18:22 2013 -0600

    Fix issue with site parameter in error summary

    It can be an empty dictionary which causes problems
    when trying to create the summary.

    Fix that by ensuring it's a string.

commit 0596ed162954f642a4117aa0118f5825f4ab899c
Author: Manuel Giffels <giffels@gmail.com>
Date:   Wed Dec 19 17:16:27 2012 +0100

    Display docstrings in RESTAPI documentation

commit 5cd9564b9ba3d6e29a9de81e018bd676436b2099
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Sat Dec 15 17:08:21 2012 +0100

    Fix sandbox creation

    The old way of finding the base of the WMCore tree didn't work with
    patched installs (the sandboxes were getting made from the base RPM, not
    the patched location)

commit 8f5e5653ed6199a8e0b203a7d10ba19e479c2d58
Merge: 6022c3a 7a9a74c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Dec 14 12:26:08 2012 -0800

    Merge pull request #4336 from ticoann/add_merged_flag_frwj

    add merged flag for framework job report

commit 6022c3a268bd7eae49e3024485568b1e02dbae6d
Merge: be7bd5c 3813be9
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Dec 14 12:25:51 2012 -0800

    Merge pull request #4335 from dballesteros7/fixLastFix

    Fix cleanup of timestamp cache on the JobSubmitter

commit be7bd5c0a2d5f6d6b503c6f6a0f1c8eac9e26be3
Merge: efdfcaf f3180de
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Dec 14 12:25:29 2012 -0800

    Merge pull request #4334 from dballesteros7/fixSmartLumiSplitter

    Fix bug in smart lumi splitter

commit 7a9a74ceef7a09f96bf4c407ef0bf5ca9ec0c7be
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Dec 13 15:44:57 2012 -0600

    add merged flag for framework job report

commit f3180ded237f2f9dddb1ee910176ec886c9cb765
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Dec 12 10:55:11 2012 -0600

    Fix bug in smart lumi splitter

    It failed if the conditions aligned and it was trying to process
    a too-big single-lumi file.

    Add unit-test that fails without the fix

commit efdfcafc39f5df2144f060b18a47e73dd51f575d
Merge: 1529eac 4ba9657
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Dec 11 13:03:54 2012 -0800

    Merge pull request #4332 from zdenekmaxa/reqmgr-couchapps-reqmgr-cleanup

    Cleaning up ReqMgr couchapp

commit 4ba96577766ddcb1a2c708593ef01d269bb2a4cc
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Dec 11 14:50:47 2012 -0600

    Cleaning up ReqMgr couchapp

    (what gets currently installed in reqmgr_workload_cache database)
    - index.html not functional, could not work behind cmcsweb frontend
    - most but 1 views removed and list function, not referenced from anywhere
    - entire evenly library was checked in without any purpose

commit 3813be999bbf5a8fb4a4581403ee6021546e6862
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Dec 11 13:16:10 2012 -0600

    Fix cleanup of timestamp cache on the JobSubmitter

    Last patch has a bug when cleaning up the memory cache on
    workflow timestamps, fix this here.

commit 1529eac85263a55977d8f25b6723d3976c52e86c
Merge: c616173 aa3bfc5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Dec 7 14:24:50 2012 -0800

    Merge pull request #4232 from ticoann/remove_workloadsummary_remgr

    remove workloadsummary installation from reqmgr and move to reqmon

commit c61617307c8b2ecf506fc0990d1a03178a794896
Merge: 501aac6 4a74365
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Dec 7 14:24:32 2012 -0800

    Merge pull request #4325 from dballesteros7/fixSubscriptionErrors

    Avoid crashes on custodial subscription to non-MSS

commit 501aac6d0e04eb6e4d5e36eca7fe803657d40359
Merge: df5d482 f37a613
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Dec 7 14:24:18 2012 -0800

    Merge pull request #4326 from ticoann/cleanupcouch_fix

    handle when there is no jobs in request

commit aa3bfc59e1e041525ed81f167ed8bb4c90fe2377
Merge: efbcaa7 df5d482
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Dec 7 15:29:39 2012 -0600

    Merge branch 'rm_workladsummary' into remove_workloadsummary_remgr

    Conflicts:
    	setup_dependencies.py

commit df5d4822703e9fcf3624939c926f931ceee0b794
Merge: 5b46f06 1e12bfa
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Dec 6 13:22:47 2012 -0800

    Merge pull request #4327 from dballesteros7/ensureJobOrderByTime

    Ensure jobs are submitted, oldest workflow first

commit 5b46f06ddf87efc930605d0921af62e342403177
Merge: e030e4a b183120
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Dec 6 13:22:22 2012 -0800

    Merge pull request #4328 from dballesteros7/addTimestampToStatusView

    Add update_time to result of requestByStatus view

commit b18312054395b2bd6d6913191a3f9750ea46e0ed
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Dec 5 17:19:29 2012 -0600

    Add update_time to result of requestByStatus view

    Add a WMStatsReader method that allows us to get
    the request sorted by state with timestamp
    of the state.

commit 1e12bfa8feae7bbdcbcf698f2f0e5bc3ff47a3e0
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Dec 5 14:51:13 2012 -0600

    Ensure jobs are submitted, oldest workflow first

    In Oracle we don't have guarantee of strictly increasing
    indexes for the workflows and the JobSubmitter relies
    on this to order the workflows and submit older workflows first.
    Change the sort to use the timestamp in the last_update column
    from the subscriptions.

commit e030e4aede1c7c1d52c6e91a6918025e1ff7ff52
Merge: 6cbc2e9 a45dfa8
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Dec 5 12:46:59 2012 -0800

    Merge pull request #4324 from zdenekmaxa/reqmgr-couchapps-configcache-cleanup

    Cleaning up ReqMgr ConfigCache couchapp

commit f37a61335a89b24dc1be9ef0bc1f7b7bcf52db4a
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Dec 5 13:31:33 2012 -0600

    handle when there is no jobs in request

commit 4a74365217e4138689cbdde26ce7579a1416e5d9
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Dec 5 12:54:50 2012 -0600

    Avoid crashes on custodial subscription to non-MSS

    The PhEDEx subscriber must check if a custodial
    subscription will be made to a MSS node before doing it
    otherwise PhEDEx will reject it and the poller will crash.

    If a custodial subscription is made to a site without MSS
    phedex node (no tape backend) then it defaults to a non-custodial
    subscription.

commit a45dfa87e53a7f378f8a4c3af663be728b044b82
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Dec 4 18:27:13 2012 -0600

    Cleaning up ReqMgr ConfigCache couchapp

commit 6cbc2e9fc053ac134a47696018e4cd5300a28aab
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Dec 4 16:52:59 2012 -0600

    0.9.33

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 201ef72c104725b73e2a2a3e14330bd06b27ac32
Merge: 8d35fbb badad7b
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Dec 4 14:46:09 2012 -0800

    Merge pull request #4308 from mmascher/accountant-location

    Do the SE name => Site name conversion also for failed jobs

commit 8d35fbb1062374e75c7ea1d131276ad99917d91a
Merge: bb75562 f2dc186
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Dec 4 14:45:45 2012 -0800

    Merge pull request #4311 from ticoann/job_report_change

    add job state transition information

commit bb755623f09d706ac42140264addb957c3cdf28a
Merge: 3fb41a2 ad6f726
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Dec 4 14:27:23 2012 -0800

    Merge pull request #4319 from zdenekmaxa/reqmgr-remove-opsclipboard

     Removing already dead ReqMgr/OpsClipboard

commit 3fb41a2070a04eb9e57f6dae97d6ad1da5a12173
Merge: 330e38f 2d58a70
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Dec 4 13:08:14 2012 -0800

    Merge pull request #4322 from dballesteros7/fixLSFTimeout

    Fix timeout kills in LSF plugin

commit f2dc1862866c72d7c4c3295dfe783e0ea034bf89
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Mon Nov 26 14:19:48 2012 -0600

    add job state transition information

    add agent info detail (handle thread is crashed)

    restrict the state transition to prevent race condion

    bug fix for state transition

commit 2d58a7017fba27692e203f5aece742d9929ba8dc
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Dec 4 11:28:00 2012 -0600

    Fix timeout kills in LSF plugin

    Add the Timeout state to the LSF plugin
    state map, needed when a job is killed by timeout

commit ad6f7266063197ca8dc75660881464c950e8d6d2
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Dec 3 17:48:32 2012 -0600

     Removing already dead ReqMgr/OpsClipboard

commit 330e38fb6f4eb98fd526cae734ecbeb8a8051346
Merge: 354a0aa 8985433
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Dec 3 09:37:30 2012 -0800

    Merge pull request #4316 from zdenekmaxa/remove-remotemsg

    Removing dead messaging feature RemoteMsg.

commit 354a0aa12cd6b71495ebde3eb8b36ef09f80f342
Merge: 54aa86f ce21f26
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Dec 3 09:34:43 2012 -0800

    Merge pull request #4313 from zdenekmaxa/reqmgr-security-issue-permissions

    Reqmgr - fix permission problem

commit 8985433f69b35eb651651fc6912f94de2e6e0021
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Dec 3 11:29:55 2012 -0600

    Removing dead messaging feature RemoteMsg.

commit 54aa86f9a0e0736d980eae0517163e7dec3df12d
Merge: 4a70451 5dfd4a1
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Nov 30 13:35:46 2012 -0800

    Merge pull request #4312 from dballesteros7/changeLogCollectCollectionToList

    Change LogCollect reporting from a reduce to list

commit ce21f26d0bd67712d0e619f6dde56842910d8c14
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Fri Nov 30 14:21:42 2012 -0600

    Reqmgr - fix permission problem

commit 5dfd4a1503cfe915b770e48c2faca127e903b94e
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 30 14:03:48 2012 -0600

    Change LogCollect reporting from a reduce to list

    Growing arrays in a reduce can blow up couch, even if
    bounded. Use a list function to collate the results,
    make the appropiate changes
    in the TaskArchiver and add a test for this in the
    TaskArchiver unit test.

commit 4a7045138444c4274e76b3c0d0836b81431633cd
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Nov 30 13:30:31 2012 -0600

    0.9.32

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 9ac969a277b43b738fd44c5eb02e64588f948443
Merge: 8ef4aed 8e89393
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Nov 30 11:24:00 2012 -0800

    Merge pull request #4310 from dballesteros7/changeReduceToList

    Change approach to getting tasks by dataset

commit 8e89393cf0ae9b0ea840760cb337b9081f70009b
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 30 07:45:54 2012 -0600

    Change approach to getting tasks by dataset

    It was abusing of couchdb reduce by creating growing lists
    (doesn't matter if bounded). Change the approach to using
    a list with generates the desired mapping of tasks
    that produced a dataset.

commit 8ef4aede97fafc69723ea3f931f82c0bfe0e1781
Merge: 5185905 3e18c4b
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Nov 29 13:26:42 2012 -0800

    Merge pull request #4309 from dballesteros7/fixReduceFunction

    Fix Reduce function for outputByWorkflowName

commit 3e18c4bc70f4f40d287ba80f42c6a590e596fe1f
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 29 14:45:39 2012 -0600

    Fix Reduce function for outputByWorkflowName

    It was generating a growing object in the output,
    although it would only grow to size 2 at any given time.
    Couchdb reduce_limit check throws an overflow
    error on this function, changing the type to an array
    fixes the problem. Added an object variable
    that only lives in the reduce function to
    keep track of duplicates.

    Changed the TaskArchiver accordingly.

commit 51859054a71a8f83bc8fe8bb866cae9c0305a16f
Merge: 70e349d da1c927
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 28 09:22:33 2012 -0800

    Merge pull request #4307 from dballesteros7/fixLogArchives

    Include the new PSet pickle in the logArchs

commit badad7ba3135162227f2fac8dc9c376008dba144
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Wed Nov 28 17:57:31 2012 +0100

    Do the SE name => Site name conversion also for failed jobs

commit da1c9278b927393d3e28afb545fbfaf2e489da67
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Nov 28 00:19:29 2012 -0600

    Include the new PSet pickle in the logArchs

    Since we changed to having a pickle file
    with the configuration, the logArchs only have
    the .py file which is useless without the pickle.

commit 70e349d36597ad0f4d94ceebe534807a6c730197
Merge: 30afbb6 25f5a8e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Nov 27 07:13:22 2012 -0800

    Merge pull request #4306 from giffels/dbs-lexicon-changes

    Added basic http(s) url validation for DBS3

commit 25f5a8ec8ab5ad3cf475c865edb9175d028bdf5c
Author: Manuel Giffels <giffels@gmail.com>
Date:   Tue Nov 27 13:51:29 2012 +0100

    Added basic http(s) url validation for DBS3

commit 30afbb68d5ff4d08276b994f3e9e61731348c3f2
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 14:27:57 2012 -0600

    0.9.31

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 6ffbbb163782a5e940cedf7726ade1c8b79e1937
Merge: fc96eec db50ae8
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 11:09:50 2012 -0800

    Merge pull request #4304 from dballesteros7/fix-unittests-workloadsummary

    Fix unit tests for new data structure

commit db50ae859bbec64d3a38be3182aadf495c8ac18d
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Nov 26 11:59:43 2012 -0600

    Fix unit tests for new data structure

    Test was incompatible with 2.6 due to the delta
    parameter in unittest.TestCase.assertAlmostEqual
    that was introduced only until 2.7. Replace it
    with places instead.

commit fc96eec87859b81a6daa71ab91c1b59551980f9b
Merge: d019cb5 18e1366
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 09:02:34 2012 -0800

    Merge pull request #4302 from sfoulkes/himc

    Add /store/himc as an assignment option.

commit 18e1366c6e4adddd1c45ecfe925552a495ee9ec6
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 11:00:55 2012 -0600

    Add /store/himc as an assignment option.

commit d019cb56c2240d5cd6ed69c4cd3fac7f53b32057
Merge: 82d8d8e 0c2d9ba
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:37:41 2012 -0800

    Merge pull request #4222 from dballesteros7/implement-checksum-check-lcgImpl

    Implement checksums in LCGImpl backend module

commit 82d8d8edf8a476546cbd3a081749be87bba404cb
Merge: 4631426 b8870c5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:37:08 2012 -0800

    Merge pull request #4285 from mmascher/check-lumi

    Check that files contains at least one good lumi before adding them to W...

commit 4631426d017a4887349c8e94544009885064918c
Merge: 71c8eb4 7409d07
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:36:14 2012 -0800

    Merge pull request #4246 from PerilousApricot/feature/subclassable-worker-sleep

    Make component sleeping configurable

commit 71c8eb4e712d66a8a02ccb2828ecdbc0302178e6
Merge: 8e5644c 209e6bf
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:35:28 2012 -0800

    Merge pull request #4283 from dballesteros7/bugfix-insert-thresholds

    Minor bugfix for the resource control script

commit 8e5644cd94614b02041c54ee6c285b5c56215e9f
Merge: b5a6cba bc5b4e0
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:35:06 2012 -0800

    Merge pull request #4284 from dballesteros7/workloadsummar-nullauth

    Allow nullauth for WorkloadSummary in local couch

commit b5a6cba85cb840cc498a7fffc776e6e2432485c7
Merge: b35cab4 aaa4d35
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:34:19 2012 -0800

    Merge pull request #4288 from dballesteros7/phedexSubscriptionFix

    Remove restriction of autoapprove on custodial

commit b35cab426a8c6ed388fe3e878fdfbc07bc2b84e3
Merge: 843c0a9 ca0db2c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:27:49 2012 -0800

    Merge pull request #4298 from samircury/failed-jobs-histograms

    Failed jobs histograms

commit 843c0a98efecfad19fc6f603a885fcc261c67981
Merge: aaa3ca1 88cf145
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:21:42 2012 -0800

    Merge pull request #4295 from dballesteros7/dqmharvest-configcache

    Allow ConfigCacheIDs for DQM harvesting

commit aaa3ca15942f373c3d8193479652fe0f219d4234
Merge: 70b56ea 2d116fb
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:21:04 2012 -0800

    Merge pull request #4301 from dballesteros7/fix-update-jobs-dao

    Fix status_time type for oracle DAO

commit 70b56ea54e7ede719f77f4951dbf614dae20b87a
Merge: 45d69ac bb8afc1
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:20:23 2012 -0800

    Merge pull request #4299 from dballesteros7/exploit-xrootd

    Allow requests to run at sites without the data

commit 45d69ace6842931e6f8cad41ef98c67cbdca5ddf
Merge: 06e7439 1a2f077
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 08:18:36 2012 -0800

    Merge pull request #4300 from sfoulkes/hidata

    Add /store/hidata as a possible LFN base for single request assignment

commit 2d116fb41d496c93931260c981878c55c512034c
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Nov 26 10:18:17 2012 -0600

    Fix status_time type for oracle DAO

    Oracle optimization require a constant type on the binds
    when calling a query many times. Ensure this from the CondorPlugin
    track method.

    Fixes #4297

commit 1a2f0774073d05c19ef07975fa9096a7ca34630c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 10:15:46 2012 -0600

    Add /store/hidata as a possible LFN base for single request assignment
    and add tests to the Lexicon tests verifying that works as well.

commit 06e74393872519414dcd6b1a5230fc153893d4ff
Merge: fed19d2 b8a6bd5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 26 06:48:31 2012 -0800

    Merge pull request #4164 from mmascher/poller-errors

    When killing a job for timeout, check if a report exist before writing t...

commit ca0db2cd524cab9b094c5f989a05a4a833e39d41
Author: Samir <samir@cern.ch>
Date:   Fri Nov 23 18:16:54 2012 +0100

    * Adds histograms for failing jobs per task/step under the label of errorsHistogram
    * Improves TaskArchiver test adding the feature where half of the jobs get the FWJR of a succeeding job, just to have variety in the job set and simulate prod situations where there are failed jobs, but those are not all jobs.
    * Adds jobTime as a histogram key, so now we will actually generate histograms in the summary of this test, before we weren't

    Fixes #4215

commit 7d6bc96c2be415dc1c306cba3ab8be5b15af28bc
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Nov 20 10:05:11 2012 -0600

    Implement summary of failure per site

    Include two type of histograms with the number of failed jobs
    per site. One given per site at a workflow level, and other
    per step and exit code.

    This is included in the summary using the new Histogram data structure
    that allows generic histograms to be built with a simple interface.
    This strutcture takes care of statistic calculation and generation
    of a dictionary suitable for couchdb, also is meant to provide
    support for multiple agents which will require merging summaries.

    Fixes #4213

commit 06ad15739c35db813bdfaacd9ab2b5bf87b4bb10
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 16 13:26:47 2012 -0600

    Include REST API for missing lumis in the summary

    Add a show function which works as a REST API
    to get the missing lumis for a completed workflow from
    the workload summary.

    URL is ugly at the moment:

    /_design/WorkloadSummary/_show/lumiMissingByWorkflow/<workflowName>?outputDataset=<string>

    The outputDataset parameter is optional and is used to filter the output. The format of the output
    is:

    {<outputDataset:string> : { <run:string> : [ [<startLumi:int>, <endLumi:init>], ...] , ... }, ...}

    Note that for summaries produced by agents without the TaskArchiver changes in this patch, the outputDataset won't
    be returned as a key, instead it will be the task where the lumis were lost.

    Fixes #4216

commit bb8afc18ec02cf3ca91c1fc19a17eb3e0ca114e5
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 22 03:45:26 2012 -0600

    Allow requests to run at sites without the data

    Include a flag at assignment time that
    allows ops to decide if a request should run
    in all CMS sites filtered by whitelist/blacklist
    instead of where the data is. This assumes Ops knows
    what is doing (sites have xrootd to get the data).

commit fed19d28cc69db9ee4ad8c65d0cfbc3e3368d8f5
Merge: 5361dc7 d97f8c3
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Thu Nov 22 16:26:57 2012 -0800

    Merge pull request #4291 from samircury/logarchives-listing-workflowsummary

    Adds PFN of logArchives@CERN per task to the summary

commit 88cf145ffbf7b4f7b0887d071cc43bac7e4ecf80
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 22 00:14:31 2012 -0600

    Allow ConfigCacheIDs for DQM harvesting

    Add the option in the ReqMgr and handle the specific tweaks
    for harvesting

    Fixes #4262

commit 5361dc7d38838446ccb8b9670cb985e37270b24d
Merge: 51aafc3 130df0d
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Nov 21 09:49:53 2012 -0800

    Merge pull request #4293 from zdenekmaxa/reqmgr-ops-scrips-02-1

    ConfigCacheUrl, extented reqmgr.py Ops script

commit 130df0de620305d163a43610021c7e259f8ff395
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Nov 19 23:17:05 2012 +0100

    ConfigCacheUrl, extented reqmgr.py Ops script

    * ConfigCacheUrl implemented for all worklflows types (allows to load
      ConfigCache document from an arbitrary CouchDB server, not necessarily
      the one coupled with ReqMgr (whose URL ReqMgr passes in CouchURL
      argument)).
      ConfigCacheUrl, if specified, overrides CouchURL. Is backwards compatible,
      if not specified, CouchURL is used as before. reqmgr.py Ops ReqMgr client
      script enhanced - more workflows template arguments defined
      (test/data/ReqMgr/requests)
      Fixes #4269

    * Ops ReqMgr client script allows to create and assign requests in one go or
      separately. Approve is always coupled with create. If necessary can be
      separated too, however, if create succeeds, approve must too. Unfortunately,
      assignment lacks REST API so the client talks to a webpage.
      Ideally (for ReqMgr2), REST API shall support all possible status
      transitions, this will then be more easily exposed to reqmgr.py.
      Fixes #4247

     * Add /store/hidata to the lfn base allowed in the assign page of the
       ReqMgr. Fixes 4287

     * minor reqmgr.py bugfix, template default values agreed by the Ops, tested
     * by the Ops on a private VM instance

commit d97f8c34f500cf8363eff158dac1e2ee94774cc4
Author: Samir <samir@cern.ch>
Date:   Mon Nov 19 13:11:31 2012 +0100

    Adds a real LogCollect fwjr to the TaskArchiver test and changes to support it better. Changes in the summary document. fixes #4214

commit aaa4d3589157cd1637019242d3a39f6667a414b4
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Nov 19 11:32:51 2012 -0600

    Remove restriction of autoapprove on custodial

    Removing an unnecessary restriction I added that prevented
    custodial subscriptions from being auto approve as well.
    The Tier-0 needs it since we auto approve the custodial
    subscriptions to CERN.

commit 209e6bf8078c356ef13f16d8dea6d8f576ad46ce
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 16 13:44:01 2012 -0600

    Minor bugfix for the resource control script

    The update threshold option had the pending and running
    slots for a task swapped.

commit 51aafc38fc36b612decce10fc49f15eb493a6af7
Merge: d2e5843 70d790a
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Nov 16 07:10:55 2012 -0800

    Merge pull request #4279 from ticoann/add_output_datasets

    add output dataset

commit bc5b4e0336de9e4827248a6e04970ca4fa5a9bb7
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 15 13:07:13 2012 -0600

    Allow nullauth for WorkloadSummary in local couch

    Check for null auth and allow it,
    also allow the update of existing documents
    this is to provide support for multiple agents
    and eventually cleanup. The TaskArchiver already takes care
    of not doing funny stuff and uploading stuff twice
    unless it's necessary.

commit 70d790aad244d091db7737203545d368439eed70
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Nov 15 15:26:35 2012 -0600

    add output dataset

commit d2e58432dba15e2ba3962ca62cebf0fe677baae2
Merge: 7adcc2f 2e52b91
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Nov 15 08:48:36 2012 -0800

    Merge pull request #4147 from dballesteros7/job-creator-mysql

    Remove Autoincrement check for MySQL

commit 7adcc2f93760c2cf22b911f3cb8446dc66a50e4e
Merge: da3b9f7 41eda72
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Nov 15 08:39:20 2012 -0800

    Merge pull request #4277 from dballesteros7/fixResourceControl-binds

    Repair resource control interface

commit 41eda723420d9cbf3eabc29e9779b63813ef709f
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 15 09:32:36 2012 -0600

    Repair resource control interface

    It was blowing on Oracle, MySQL didn't complain but
    it wasn't doing the right thing

commit b8870c586c351c20ee2cb6b9600f84b2eea1a236
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Thu Nov 15 14:52:13 2012 +0100

    Check that files contains at least one good lumi before adding them to WWMBS. Fixes #4267

commit da3b9f780520c74f19ea715cb07d505c14b6886d
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 14 16:36:57 2012 -0600

    0.9.30

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit d439c7d52a17470083e8fa4ac82d2e72ae017c2d
Merge: cea9f19 c67eaf5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 14 16:36:15 2012 -0600

    Merge branch 'master' of github.com:dmwm/WMCore

commit c67eaf550a3342c696d6d6dd12d43bbeb7761c13
Merge: 6f60062 f3ddd33
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 14 14:35:56 2012 -0800

    Merge pull request #4272 from dballesteros7/fix-unit-tests-rc

    Fix unit tests broken

commit f3ddd33634a2bb2630d3469f5e401a040cb42292
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Nov 14 16:17:59 2012 -0600

    Fix unit tests broken

    Change of interface that many test use

    Fixes #4271

commit cea9f19c690b49953cd662640da1f9a44b607fc5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 14 16:25:41 2012 -0600

    0.9.29

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 6f600623d99924c8dd1e07c6bd08f1d3cf91c2ef
Merge: aeb3130 21f9bd4
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 14 14:01:59 2012 -0800

    Merge pull request #4270 from dballesteros7/pending-jobs-per-task

    Implement pending jobs per task/site functionality

commit 21f9bd486a77d455e4efa8d9ec17a4c72a3f790e
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Nov 12 12:52:35 2012 -0600

    Implement Pending jobs per task/site functionality

    Add a new column to the rc_threshold table, this indicates the number of pending
    jobs to keep for an individual task.

    Modify all the concerned DAOs and the JobSubmitter.

    Rewrote the JobSubmitterPoller unit tests to make them all non-integration,
    break the dependecy of having condor available and add tests for the different
    mechanics happening in it such as threshold management, site status,
    site whitelists/blacklists. Also includes a performance test.

    Finally add a special option to setup the Tier-0 thresholds in one line from the
    resource control script.

commit aeb31307e5e8c93ad78bb5bb712954ecdb4d503c
Merge: dc9553b 8258774
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Nov 14 06:40:48 2012 -0800

    Merge pull request #4265 from dballesteros7/tier0-runStates

    Tier0 run states

commit dc9553b09f91dfa020175c0737258bc73db7ed8a
Merge: 61a1fe3 0962526
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Nov 14 00:02:37 2012 -0800

    Merge pull request #4237 from zdenekmaxa/reqmgr-request-clone

    ReqMgr requests clone implemented.

commit 61a1fe3b0010da4144f40507839cffd6f976a8b3
Merge: d6c68d7 198da7c
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Nov 13 14:34:27 2012 -0800

    Merge pull request #4253 from PerilousApricot/bugfix/revert-a-revert-a-revert

    revert a revert a revert

commit 8258774df9f467bc9b6824e0f366f94a95942fa7
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 9 01:13:25 2012 -0600

    Tier0 Plugin implementation

    The Tier0 plugin for the AnalyticsDataCollector components
    calculates the state of Tier-0 workflows based on subscription
    information found in WMBS and updates the request document
    in central WMStats with this status.

    Additionally this patch fixes some bugs with PluginInterface implementation
    patch and renames it from PlugIn to Plugin to conform with the same
    name as the BossAir package (PlugIn is a bit harder to remember).

    Fix the AnalyticsDataCollector unittest and removed it from the
    allowed failing tests.

    This requires #4227 to work, it also fixes the unit test breaks from it.

    Fixes #4264

commit 2f5e7f87ab99b679062fcabdcaab86d4920dcc5e
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Nov 1 13:29:02 2012 -0500

    add PlugIn structure in AnalyticsDataCollector

commit d6c68d7bd75d8c0750ca3278a9b608923567bd44
Merge: 27e0f3d ca944fe
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Nov 13 13:55:56 2012 -0800

    Merge pull request #4227 from ticoann/merge_wmagent_summary_delete

    deleting jobsummary from wmagent summary  when request is announced

commit 27e0f3d742d6f46a39d5f24f926a3953380919ea
Merge: 25a4f3c d08bd4b
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Nov 13 09:29:36 2012 -0800

    Merge pull request #4244 from dballesteros7/display-errors-workload-summary

    Fix error display in workload summary

commit 25a4f3cb848384d86059f2736fbabd3db21932bd
Merge: 928c20b 2b13ba9
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Nov 13 09:11:35 2012 -0800

    Merge pull request #4242 from dballesteros7/input-any-acdc-collection-name

    Allow arbitrary collection name on ACDC request

commit 928c20b8fc2ee53eef092e1a4d9e014349a39579
Merge: a05e9a4 22b5967
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Nov 13 08:09:40 2012 -0800

    Merge pull request #4239 from yuyiguo/lexicon_fix

    lexicon add dbs user

commit a05e9a4bd9b8d52ed9b96f2d68c59b19f6271d58
Merge: 9ffda12 b23c608
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 12 12:44:48 2012 -0800

    Merge pull request #4220 from ticoann/jobstate_typo

    fix the typo in the jobstatus (affect monitor update in submit paused status)

commit 9ffda1287a32950fa3370a0fd5777bff0cfba3c9
Merge: 1b60a96 af54eb6
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 12 12:44:23 2012 -0800

    Merge pull request #4260 from dballesteros7/phedex-inject-override

    Implement override for PhEDEx injections

commit 1b60a969e95731c164618e2ff9080d24edbb7320
Merge: c64810b 8d0d93c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 12 08:08:04 2012 -0800

    Merge pull request #4261 from dballesteros7/stageout-code-fix

    Fix broken stageout code

commit 8d0d93cbc1a214926b8a6d108ef12eb85ce50503
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Nov 12 09:56:24 2012 -0600

    Fix broken stageout code

    Combine fixes from @PerilousApricot and @hufnagel, the check
    on the new stageout code is done without the use of a helper
    to keep consistency with the rest of the executor code as
    implemented in #4259.
    The unit tests from #4251 are included and working.

commit c64810b216a9c24051ded4965cdf870e9c66c037
Merge: 7bad5fd a83d935
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Nov 12 07:18:51 2012 -0800

    Merge pull request #4258 from bbockelm/cvmfs_support

    Add support for CVMFS, based on the CRAB implementation

commit a83d9358f0738e33f690158446d697e5fa3e3596
Author: Brian Bockelman <bbockelm@cse.unl.edu>
Date:   Sun Nov 11 14:10:25 2012 -0600

    Add support for CVMFS, based on HN conversation and identical to the CRAB bootstrapping code.

commit af54eb6de21aced49b5f61176e65c5f8b5941442
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 9 17:57:28 2012 -0600

    Implement override for PhEDEx injections

    Due to the special setup of the T0/T2 at CERN
    and the Bulk/Express processing in the Tier-0. We need
    the ability to specify where the files should be registered
    to in PhEDEx. This is done per spec, and all files
    associated with the workflows from that spec
    will be registered to the given site, regardless of the SE
    available in the dbsbuffer table or custodial site
    entry in dbsbuffer.

    Also fix a unit teste in WMWorkload_t

commit ca944fe957e3c3171b1c1621fc50365720040852
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Mon Oct 29 11:34:20 2012 -0500

    deleting job summary from wmagent summary on given condition

commit 7bad5fdc7e9526dfcd030e528c10f756795fe88b
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Wed Nov 7 11:17:50 2012 -0600

    0.9.28

    Signed-off-by: Andrew Melo <andrew.melo@gmail.com>

commit e4d8ecd822884c36a7a1c4aed738895bd93ff403
Merge: 3126707 322f97a
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Wed Nov 7 08:59:40 2012 -0800

    Merge pull request #4252 from PerilousApricot/bugfix/revert-a-revert

    Do something crazy

commit 198da7c291c5d62708bdd1b61db6f0e835720133
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 17:56:15 2012 -0500

    Revert Revert Revert

    This reverts a previous commit that reverted a previous commit. Solid.

commit 322f97af6b7160446da3e821baeb5f965a03aa1d
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 17:53:01 2012 -0500

    Do something crazy

    I will now revert a revert so something something something

    After that I will revert this reverted revert so I can get things back
    to normal.

commit 3126707d8e87b0d86c52e9bdb8faaca9b0979bf3
Merge: eb65f8b 2f16328
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Nov 7 07:34:45 2012 -0800

    Merge pull request #4240 from dballesteros7/fix-reqmgr

    Bugfix StdBase DQMHarvest validation

commit 7409d07461039b26f090ca525d005a0ea73a19b0
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Mon Nov 5 11:03:49 2012 +0100

    Make component sleeping configurable

    Factor out the sleeping part of the component of the component harness.
    In some cases, simply sleeping isn't the best idea, this change makes it
    so subclasses of BaseWorkerThread can choose how they sleep.

commit d08bd4be17394099726f17e5466574911aec5253
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 1 18:28:28 2012 -0500

    Fix error display in workload summary

    Display all errors correctly

    Fixes #4217

commit 2b13ba953f22ba2f072304615f3f3b2c8588cfbc
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Nov 2 10:08:08 2012 -0500

    Allow arbitrary collection name on ACDC request

    A CollectionName parameter can be specified in the request arguments
    if not specified the  collection name defaults to the old request name
    as it has done so far.

commit 2f16328c4d16ff24805c1b946aca9f17b27409d5
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Nov 1 15:38:44 2012 -0500

    Bugfix StdBase DQMHarvest validation

    Simply a code error

commit 22b5967677b08dc5cf4286629c4ad68520944951
Author: yuyi@fnal.gov <yuyi@fnal.gov>
Date:   Wed Oct 31 16:07:46 2012 -0500

    lexicon add dbs user

commit 0962526adf0e6a0798a24e25db508ca8c492680e
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Oct 31 20:58:33 2012 +0100

    ReqMgr requests clone implemented.

    A few functions / methods signatures changed in the ReqMgr files.
    Requests clone implemented: PUT, /reqmgr/reqMgr/clone/requestName, unittest
    for clone and reqmgr.py clone interface.
    reqmgr.py has --allTests to run over everything the script is capable of for
    testing.
    Clone request shall fix: #3984, #3944

commit eb65f8bb5da3b8dff7bf5ed0932b3505053951ec
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Oct 31 15:16:45 2012 +0000

    Remove updates in cancellation - stop conflicts

commit 1df71373fe53a9c37c2e33d6b14802f854889304
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 17:56:15 2012 -0500

    Revert "Revert "Ops ReqMgr interaction script, ProcConfigCacheID renaming""

    This reverts commit 138bf6cf591a23af5cf99ff0d03d0ae1f9fbb0f2.

commit 504fba6863a38c0c3d0b3764a54b3eed1b4cb242
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 17:55:04 2012 -0500

    0.9.27

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 138bf6cf591a23af5cf99ff0d03d0ae1f9fbb0f2
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 17:53:01 2012 -0500

    Revert "Ops ReqMgr interaction script, ProcConfigCacheID renaming"

    This reverts commit d01c12d692825c485dab43849765e08e59d237ba.

commit efbcaa71c7a62c6958a05e5142af714eb367f339
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Tue Oct 30 14:29:32 2012 -0500

    remove workloadsummary installation from reqmgr and move to reqmon

commit 0bdfa197e0e4eefda3a68ce03bd70c1db8974ad9
Merge: 9d4beeb db9a6a4
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 12:27:21 2012 -0700

    Merge pull request #4231 from dballesteros7/fix-wmagent-config-phedex

    Bugfix for wmagent config

commit 9d4beeb66a702dbb4c09f7e0caefdbf506d562db
Merge: 60c5772 d6ea9d2
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 10:52:35 2012 -0700

    Merge pull request #4194 from PerilousApricot/feature/stageout-switch

    Add per-request switch for stageout

commit db9a6a417b227bd26fd5668bd5354275d7609bc3
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 30 11:23:23 2012 -0500

    Bugfix for wmagent config

    The spec cache path was not correctly set by the mod-config

commit 60c5772d339f49a167c66f7960a4ee04e3745eca
Merge: 00cf065 0dcb89f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 30 08:17:59 2012 -0700

    Merge pull request #4228 from dballesteros7/fix-taskchain-bug

    Bugfix for PrimaryDataset handling in RelVal

commit 0dcb89fae2ea2a77880dbb95f344b426af826207
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Oct 29 13:47:44 2012 -0500

    Bugfix for PrimaryDataset handling in RelVal

    The PrimaryDataset argument wasn't really suited to be included in the
    ParameterStorage decorator used to modify global parameters according
    to task-level specifications. Since the PrimaryDataset
    is not really a global parameter and always a task one, changed this support
    for PrimaryDataset in order to make this actually work.

    Added unit test verification of the output datasets to catch these errors.

    Fixes #4225

commit 00cf0653636e30ebc46d082956657db8a921036d
Merge: 8820331 d01c12d
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Sat Oct 27 16:49:59 2012 -0700

    Merge pull request #4223 from zdenekmaxa/reqmgr-ops-scrips-01

    Ops ReqMgr interaction script, ProcConfigCacheID renaming

commit d01c12d692825c485dab43849765e08e59d237ba
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Sun Oct 28 01:40:01 2012 +0200

    Ops ReqMgr interaction script, ProcConfigCacheID renaming

commit 0c2d9ba115cbc39ccf0d740840d5aa452745ee55
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Oct 26 13:31:28 2012 -0500

    Implement checksums in LCGImpl backend module

    Add support for verifying the adler32 checksum when available
    through the lcg-ls -l command. Also add general support
    for passing the locally calculated checksums of the files
    to the StageOutImpl objects, this will enable the checksum
    verification in the rfcp-CERN backend plugin as well.

    For the LCGImpl plugin the checksum is obtained as follows:

    echo "$LCG_OUTPUT" | sed -nr 's/^.*\s([a-f0-9]{8})\s*\([aA][dD][lL][eE][rR]32\)\s*$/\\1/p'

    Where LCG_OUTPUT=`lcg-ls -l -b -D srmv2 <SURL> 2>/dev/null`

    The expected format of LCG_OUTPUT is a multi-line string where one of the lines has the following format:

    * Checksum: <8-hex character string> (ADLER32)

    Or

    * Checksum: <8-hex character string> (adler32)

    If this line is not present then it is assumed that the checksum is not supported and the check is skipped,
    but if a checksum is present then it will be compared and will fail with exit code 63011 if it doesn't match.
    Error code is the same as for size mismatch.

    Fixes #4185

commit b23c6085b5cdd4a6210c53ad216173ce928158fb
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Thu Oct 25 15:20:40 2012 -0500

    fix the typo in the jobstatus

commit 8820331a4e14d54250b9b3d67f1b67c1c17f7172
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 14:04:21 2012 -0500

    0.9.26

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit cebfe33d0997dcadeee09498b0932c9d93cca417
Merge: aefaa40 26e9e05
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:57:38 2012 -0700

    Merge pull request #4196 from PerilousApricot/bugfix/detonating-logs

    Fixing giant log files/memory leaks

commit aefaa4070158316aecbbf8352ead53657213e9a0
Merge: 4585204 e0f957d
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:56:57 2012 -0700

    Merge pull request #4203 from ticoann/t0wmstats_deploy

    add t0wmstats dependency

commit 4585204e72df4518b21a1af3266191a40d387a67
Merge: 5e2fb19 5cd7a35
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:56:13 2012 -0700

    Merge pull request #4206 from dballesteros7/create-error-report-on-createfailed

    Create error report for createFailed jobs

commit 5e2fb1926813bd664383a3b26b23db101b53c9e7
Merge: b239f3c 5da3b5f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:55:42 2012 -0700

    Merge pull request #4218 from dballesteros7/avoid-dashboard-runtime-error

    Improve error handling in Dashboard runtime

commit b239f3c629f1c7340d8e1df2083a02b6917c2eeb
Merge: 793e61c 8b6db87
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:55:21 2012 -0700

    Merge pull request #4209 from hufnagel/filesetclosing

    implement alternative fileset closing policy

commit 793e61c5c521ab6acc8c14f48d950fa327f96d08
Merge: be1a981 18d9d91
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:53:58 2012 -0700

    Merge pull request #4137 from dballesteros7/keep-error-handler-flexible

    Make maxRetries flexible between job types

commit be1a981bfb1d698c9faaf688479ac6c5fc22a91d
Merge: e93da7b f3d76f4
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 25 11:53:22 2012 -0700

    Merge pull request #4208 from hufnagel/subfinished-daobug

    fix a bug in GetAndMarkNewFinishedSubscriptions

commit 5da3b5f19fc21708fad6d2d24e60b1872649fcef
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Oct 25 09:48:29 2012 -0500

    Improve error handling in Dashboard runtime

    If the command to get the user DN in the dashboard code
    hangs when opening the pipe an error can occur later
    in the code

commit e93da7bd60c23b1cef3c893bcbd3a7d09e15d95f
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Thu Oct 25 14:01:55 2012 +0100

    Skip integration tests if infrastructure down

    Also, skip rest of tests after an error or skip

commit 2e52b9114190ebccdc07704796400ec59749c4ed
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 9 11:02:04 2012 -0500

    Remove Autoincrement check for MySQL

    From the MySQL documentation:

    ```
    InnoDB uses the following algorithm to initialize the auto-increment counter for a table t that contains an AUTO_INCREMENT column named ai_col: After a server startup, for the first insert into a table t, InnoDB executes the equivalent of this statement:

    SELECT MAX(ai_col) FROM t FOR UPDATE;
    InnoDB increments the value retrieved by the statement and assigns it to the column and to the auto-increment counter for the table. By default, the value is incremented by one.
    ...
    If the table is empty, InnoDB uses the value 1.
    ```

    In case of a server restart we have security that the auto-increment will be preserved to the max job_id present, the only case where removing this check might cause problems
    is if the most recent workflow was deleted and so the index in wmbs_job is lower than before and then you restart MySQL before the TaskArchiver/CleanUpPoller cleans couch. To account for this, this patch adds support in CMSCouch to specify a callback function on conflicts during a commit or queue of documents. For the ChangeState, job and fwjr jobs document are overwritten when a conflict occurs without generating conflicts (i.e. not using all_of_nothing).

    The benefit of this change is avoiding errors when the JobCreator is restarted after a couch crash.

    Fixes #4005

commit f3d76f465cb0f50ca8b3dcfd8337f5d04f42003b
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Wed Oct 24 16:15:08 2012 +0200

    fix a bug in GetAndMarkNewFinishedSubscriptions

    The query in the GetAndMarkNewFinishedSubscriptions DAO fails (at least on Oracle).

commit 5cd7a3540d34b373b7ed7489eb3d106723f4c2af
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 23 09:39:32 2012 -0500

    Create error report for createFailed jobs

    Make sure that createFailed jobs get an
    informative error report for the summary

commit 18d9d91681f5c9af864c4098b5970c5c785d7e44
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Oct 3 16:47:24 2012 -0500

    Make maxRetries flexible between job types

    In #3961 the retry manager was modified to work differently on each
    type of job, however there is one parameter which affect retries but
    it is in the ErrorHandler.

    This should be as flexible as the RetryManager parameters

commit 1143d90370cdf310143efa7d769819e71708989e
Merge: 4fab718 4d4e580
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Tue Oct 23 11:37:29 2012 -0700

    Merge pull request #4201 from stuartw/patch/WorkQueue/4189

    Change workqueue test to match new exception

commit 8b6db87a36c1ad192e18158944674856f7e99142
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Tue Oct 23 20:34:05 2012 +0200

    implement alternative fileset closing policy

    Normally the JobArchiver will only start closing filesets if
    the corresponding workflow has been marked as injected. Implement
    an alternative policy where fileset closing ignores the injection
    status of the workflow and only depends on whether the top level
    fileset is closed or not. This is for workflows where the
    chain of subscriptions only goes down, that is, we do not have
    multiple different top level filesets and subscriptions feeding
    into the same lower level fileset/subscription.

commit e0f957d9e7ea17e7f5e4b3f3f4eb29959f46b4d3
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Tue Oct 23 13:27:12 2012 -0500

    add t0wmstats dependency

commit 4fab7187465a4cdd862e5dc4829f4ea61f05dc9d
Merge: 1dcb577 839bd64
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Oct 23 10:19:51 2012 -0700

    Merge pull request #4202 from zdenekmaxa/reqmgr-unittest-fixes-cleanup

    ReqMgr unittests fixes, removing hardcoded CMSSW version in tests

commit 839bd64e7679e741d51438ec39ccd3883caccf2d
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Tue Oct 23 19:15:14 2012 +0200

    ReqMgr unittests fixes, removing hardcoded CMSSW version in tests, tests cleanup

commit 4d4e580727bb6b53708b88b31f10b61a3f115fc8
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Tue Oct 23 18:00:17 2012 +0100

    Change workqueue test to match new exception

commit 26e9e05733b7a6e83e7895cb2b501f90cd453b11
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Tue Oct 23 11:48:30 2012 -0500

    Fixing giant log files/memory leaks

    WMException was printing traceback.format_exc(), which spits out all the
    information for an exception. In the case of nested exceptions, this
    would cause the size of the error message would grow geometrically.
    Also, WMException was looking for member variables it didn't have, so it
    would end up failing in __init__ (classy)

commit 1dcb5772ef47d155ff9f989e9d406be55f257cbe
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Tue Oct 23 15:17:22 2012 +0100

    Fix buildrelease for modern mktemp versions

commit 20af7e4356fcb6c9e60271e13880b3cc4fa79e5d
Merge: 17f2e50 5f28db2
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 23 07:06:58 2012 -0700

    Merge pull request #4150 from PerilousApricot/bugfix/whitefix-fixes

    Whitespace fixes

commit 5f28db2aa1fab075adf0387d1a9b757c20702b9f
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Mon Oct 22 16:21:30 2012 -0500

    Whitespace fixes

    There's quite a few whitespace errors. Someone apparently
    has their editor set to use "tabs = 2 spaces"

commit 17f2e509ccfe179563ca86a36691ca7240d5ff31
Merge: 71e7aed b1b4449
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Mon Oct 22 13:09:31 2012 -0700

    Merge pull request #4195 from PerilousApricot/bugfix/blacklist-flaky-test

    blacklist a flaky test

commit b1b444977dad34d8909507650a3f16023c82a474
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Mon Oct 22 15:07:47 2012 -0500

    blacklist a flaky test

commit d6ea9d2d072c5e39bdd03f61c4a9a9a46e2fc61f
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Mon Oct 22 14:48:21 2012 -0500

    Add per-request switch for stageout

    The old switch is hardcoded per-agent. This makes it so you can set
    EnableNewStageout to cause StageOut, LogArchive, LogCollect to use the
    new values

commit 71e7aed36658d872b778baed9cad356541c52da4
Merge: 69c9011 21534f2
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Oct 22 12:13:02 2012 -0700

    Merge pull request #4193 from zdenekmaxa/master

    0.9.25

commit 21534f26476dc3e49d8fb6efa738312bfb756031
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Mon Oct 22 21:09:31 2012 +0200

    0.9.25

commit 69c9011211d72a8ae122dad2e64ec111b54bbe7e
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Mon Oct 22 13:05:19 2012 +0100

    Change acdc integration test to look at all previous requests for one to resubmit

commit 704a4573ff8d5255992c5422d3fbf82fe35c2019
Merge: feaeeb4 8359e91
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Sun Oct 21 07:07:35 2012 -0700

    Merge pull request #4191 from zdenekmaxa/reqmgr-interaction-scripts-logging-01

    Test inject script fixed, unittests fixed, some improved logging and exception handling

commit 8359e91829c88bdfd7ffd89eefb67d96ed07afd3
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Sun Oct 21 15:58:45 2012 +0200

    Test inject script fixed ; Services_t/RequestManager_t/RequestManager_t.py unitetest fixed ; create request - improved logging and exception handling

commit feaeeb4b443e3d54497c8da959e4d94d493102b1
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 16:21:27 2012 -0500

    0.9.24

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit ab7c7c2c1ff7434cf363a77acc5d3661605936d2
Merge: 33240f3 55e2e9b
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 14:20:16 2012 -0700

    Merge pull request #4128 from dballesteros7/fix-default-split-on-run

    Set SplitOnRun back to true in LumiBased

commit 33240f3e1c395b22c67655d50e9fca88d85ae773
Merge: a67d75c 086c084
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 14:18:18 2012 -0700

    Merge pull request #4177 from stuartw/patch/WorkQueue/4160

    Ensure canceled workflows get cleaned up.

commit a67d75cf0ea1f0e8154ec2eb84844eee39b43108
Merge: 9e0cda7 e0ece1e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 14:18:03 2012 -0700

    Merge pull request #4099 from dballesteros7/ignore-output-modules

    Ignore output modules in ACDC requests

commit 9e0cda7bc05b7a0df19cb8d33ae9c345ea8b37de
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 16:16:46 2012 -0500

    0.9.23

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit e0ece1e391fea25cca1591af744ca0d706ee2e7c
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Oct 19 16:05:24 2012 -0500

    Ignore output modules in ACDC requests

    ACDC requests can specify a list of output modules which will be ignored
    when running th ACDC. It works as follows:

    - Any task that uses one of the output modules in the ignore list as input is removed from the workload, that means that task and its children are gone
    - Tasks that produce output modules in the lists store the list of "bad" output modules in the CMSSW step information
    - CMSSW steps don't add information about the files belonging to modules in the bad output modules list to the Report pickle, so those files don't get staged out

    Fixes #3976

commit 879ac8db68aa5da77b61ec32a63a92388011c97c
Merge: e241264 e1cee43
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 14:02:27 2012 -0700

    Merge pull request #4133 from cinquo/master

    Remove redundant slash from log file LFN

commit e24126480252d78e248b032a04c95a56002ff1ea
Merge: d12acee 9ad10a5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 14:01:13 2012 -0700

    Merge pull request #4188 from ticoann/0.9.22

    fixes #4187

commit 9ad10a59bdf32386b10ecb3a982fed54dfef7af6
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Oct 19 14:45:06 2012 -0500

    fixes #4187

commit d12acee96301a487639be100f834edb5989af402
Merge: 1e5960a 489f414
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 09:17:17 2012 -0700

    Merge pull request #4180 from dballesteros7/fixReDigiPileupTest

    Test error was masked before

commit 1e5960a780b3313c1aae51b438a42dfe440d4058
Merge: 7214ec0 e15f8f2
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Oct 19 09:16:55 2012 -0700

    Merge pull request #4129 from dballesteros7/fix-dqm-harvest-validation

    Bugfix for validation of well-formed dqmHarvest

commit 7214ec0a38ee7b2ba929c586afbb2e4323ef6e16
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Oct 19 13:59:07 2012 +0100

    Retry request submission in integration tests - get round intermittent problems

commit 086c084378ea656b84c8abbf046f65df03ced62f
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Thu Oct 18 13:43:28 2012 +0100

    Ensure canceled workflows get cleaned up.

    Fixes #4160

    Fix wmbs injection query to report canceled as injected.
    If workflow unknown to workqueue let job archiver clean up.

commit 489f41466366ac482021cd0c7789a33f1d60ddef
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Oct 18 17:30:13 2012 -0500

    Test error was masked before

    The sanboxes from previous tests were not cleaned up

    Fixes #4179

commit 38c39c43f7237fd316930839674ac9be3c0ee8cc
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 17:30:34 2012 -0500

    0.9.22

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 433f93c071a36423fbd5235b4f94e46cd7714b07
Merge: 55588c7 c46acea
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 15:20:03 2012 -0700

    Merge pull request #4118 from dballesteros7/merge-promptreco-specs

    Merge the Tier1PromptReco and PromptReco specs

commit 55588c7550c0d762b83872760409e18f69d41260
Merge: 81c1fa2 815a70e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 15:19:30 2012 -0700

    Merge pull request #4127 from dballesteros7/smart-lumi-splitting

    Smart lumi splitting

commit 81c1fa20741813e39d8d2c3e18ada6ff98f7332e
Merge: 08b0176 498859e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 15:15:27 2012 -0700

    Merge pull request #4167 from dballesteros7/taskarchiver-reverse-order

    Start finishing subscriptions from the top

commit 08b01762b6c9f4c50d4327bb3ae27c776cb2b02d
Merge: c0a5e8b ab56b64
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 15:14:46 2012 -0700

    Merge pull request #4171 from dballesteros7/phedex-fancy-subs

    Phedex fancy subs

commit c0a5e8b61b22f47c78f44ae1c494399d70730de5
Merge: 9f99629 e77f783
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 15:13:15 2012 -0700

    Merge pull request #4174 from dballesteros7/increase-lfn-counter

    Modify LFN counters in extension workflows

commit ab56b64817419cc3845e4d114109dec1d304edee
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 9 14:38:07 2012 -0500

    Improve the PhEDEx subscriber

    The PhEDEx subscriber component now uses the spec information
    to make the subscriptions regardless of where the files ended up.

    The specs are copied to a directory in the JobAccountant and its
    path is stored in the DBSBuffer workflow table under the column "spec".

    The PhEDEx subscriber can operate in 2 modes:

    - Unsafe: Subscribes custodial as Move as soon as files appear in DBSBuffer
    - Safe: Subscribes custodial as Replica first, and when the workflow is complete as Move

    Also implemented a PhEDEx subscriber unit test using the PhEDEx emulator and fixed
    the broken JobAccountant_t test.

    This is a client-only patch, if an agent is updated but the reqmgr doesn't have
    the previous commit then the PhEDEx subscriber won't do anything. But it is stable,
    i.e. it won't crash.

commit 7ebea945487ad0552b553d5326d5200035f9617c
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Oct 8 16:40:33 2012 -0500

    Store phedex subscription information in workloads

    Modify the WMTask and WMWorkload classes to store subscription
    information for each outputModule/dataset in each task.

    This can be specified during assign, the reqmgr web interface now shows
    another set of site lists for subscription configuration. The subscription
    configuration can be totally unrelated to the site whitelists.

    The options that can be configured for a subscription are:

    - Custodial sites
    - Non custodial sites
    - Sites where the request can be auto approved
    - Priority

    The option to customize subscription information per task is not exposed in the ReqMgr
    for now is only intended for the Tier-0 use case.

    This a ReqMgr-only patch

commit 55e2e9bfcf826430a49ad377fde8ba49490f6fd9
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Oct 3 11:23:08 2012 -0500

    Set SplitOnRun back to true in LumiBased

    I learned from Marco Rovere that the DQM step cannot process two runs
    in a file (though the harvesting should on a merged file with multiple runs).
    Therefore it's not a good idea to not split by default on run, reverting
    that change.

commit e77f783e1f612c9c2ce82326f0c64cb646e870c5
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Oct 17 13:54:27 2012 -0500

    Modify LFN counters in extension workflows

    Add the initial_lfn_counter parameter to the splitting of the
    production and merge tasks in MonteCarlo and LHEStepZero workflows
    if the FirstEvent/Lumi is different from 1. The initial_lfn_counter
    is calculated as what the last previous job count should have been
    based on the current events_per_job and FirstEvent.

    Fixes #4151

commit 9f99629620e0212f6efedefa3e89ba741252867c
Merge: 591ecca f0a3622
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 18 11:31:24 2012 -0700

    Merge pull request #4175 from dballesteros7/taskchain-refine

    Make TaskChain more configurable

commit 591ecca9fbe30a755d8151c2046edafd0b897b84
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Thu Oct 18 15:01:20 2012 +0100

    Fix upstream push

commit 3821daecbabd43d475fea09fd18df31973d000aa
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Wed Oct 17 18:50:04 2012 +0200

    Obsolete Alerts couchapp removed, AlertsCollector replication finalized

commit f42d24bedb2f0781c3c6d7f994e42e58aadec5f2
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Tue Oct 2 15:18:40 2012 +0200

    fix file deletions in the CERN rfcp stageout plugin

    The CERN RFCP stageout plugin passes full PFN URLs to stager_rm and nsrm, which is not supported.
    Fix this and also backport the latest change from the PRODAGENT CERN stageout plugin, passing
    the file size and file checksum as xrdcp options so they are checked on the fly.

commit 498859ed1d1b0b98b3fc9918d07788d68f048da8
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Oct 3 12:00:23 2012 -0500

    Start finishing subscriptions from the top

    Before the latest changes the subscriptions were finished from
    the bottom because they were deleted on the spot. However
    now that the deletions only take place at the very end when everything
    is finished we can actually remove the requirement that the child
    subscriptions are finished to mark the parent as finished. This will
    allow us to have a view of the progress of a workflow in terms of its
    finished subscriptions, e.g. a workflow finished the DataProcessing subscription
    and now it's just waiting on the merge. Also the more we can skim from this
    dreaded query the better.

commit f0a3622956698df418715699036cf8527b4798db
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 16 14:30:25 2012 -0500

    Make TaskChain more configurable

    Changes with this patch:

    - Allow different parameters which were constant throughout
    the workflow to be configurable per task, this are usually set
    on the StdBase instance. Add a decorator to modify
    this instance before any task is setup and restore it
    afterwards.

    - Cleanup the unit test for the TaskChain spec and test
    the new functionality.

    - Allow that some tasks' output be only in the unmerged
    area only for the use of later tasks but not propagated to
    DBS.

    - Change the dashboard activity to relval, makes more sense
    than having some under production and others under processing.

    This is a pure ReqMgr patch, no changes needed on the agent.

    Fixes #4138, #4117, #4116, #4112

commit c87a77b6e470dab7f46ec4a68a0e35df56ca3c43
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 15 14:49:23 2012 -0500

    0.9.21

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 0aef6f4ccec6c10ce7ff13146c66e7e64f1df3f2
Merge: e0126d4 acc841b
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 15 14:42:33 2012 -0500

    Merge branch 'master' of github.com:dmwm/WMCore

commit acc841b42b1d6a0ba7f6031a4a6ce86f9e406414
Merge: 61f26c9 43ea26a
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 15 12:38:43 2012 -0700

    Merge pull request #4166 from ticoann/assign_state_change

    change monitor status when assignRequest is called directly

commit 61f26c94aae33f47256c9acd11da1e690e4a4362
Merge: b2b0e1a 143bdcf
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 15 12:37:18 2012 -0700

    Merge pull request #4168 from ticoann/add_lq_warning

    fixes #4089

commit b2b0e1aa6defa6b9e4579526d0ab617e31df99c4
Merge: 00bd5f1 30aa8a0
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 15 12:21:30 2012 -0700

    Merge pull request #4169 from dballesteros7/fix-jobsummary-changestate

    Include the jobType in all jobs in ChangeState

commit 143bdcf1b4c9a256baeee7288eec4d12fcec9c21
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Mon Oct 15 13:28:48 2012 -0500

    change analysis db name

commit 30aa8a0778b9b601fa295b1a1ef91d2583844914
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Oct 12 14:54:46 2012 -0500

    Include the jobType in all jobs in ChangeState

    Any job with a fwjr will require the jobType key
    to allow the jobsummary to be uploaded to WMStats,
    instead of looking for all the places where the job can come.
    Make it central and load it always when missing, it's part
    of a frequent query so it's inexpensive.

commit 43ea26aee8f1985345d9b3c71bd4523da1f9e30e
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Oct 12 14:15:55 2012 -0500

    change monitor status when assignRequest is called directly

commit 51955be3bbf7fc4d6a90785e80ca85bd63deeefd
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Oct 12 14:08:56 2012 -0500

    add local queue pull warning: need to define for different case and add wmbs job id in jobsummary doc (Tier0 ops request)

commit b8a6bd530639125accf901ea9c18a7cdf3ef7afe
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Thu Oct 11 21:02:22 2012 +0200

    When killing a job for timeout, check if a report exist before writing the fake-one

commit 00bd5f1a7c51ace825d5391057d0a529b1b89691
Merge: a2b8e1f 763e3e4
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 11 11:15:44 2012 -0700

    Merge pull request #4161 from mmascher/poller-errors

    Fix alerts not sent in StatusPoller, wrong usage of getattr, and printed...

commit 763e3e4f99df9206ee4740bae267fa26332f0390
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Thu Oct 11 19:11:30 2012 +0200

    Fix alerts not sent in StatusPoller, wrong usage of getattr, and printed the stachtrace of generic Exception

commit e0126d413e69a089b73a85de9f6d454076c894b8
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 16:04:48 2012 -0500

    0.9.20

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit a2b8e1f0fd72c9184c69161d5be3aaf0cc1d03b5
Merge: bfef915 eb70d76
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 15:28:54 2012 -0500

    Merge branch 'pull_monitor4' of https://github.com/ticoann/WMCore

commit bfef915ab6c0c9f4e175b201479ae14610723f74
Merge: 3b6c3bb 3df405c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 15:06:06 2012 -0500

    Merge branch 'combine_cleanup_taskarchiver' of https://github.com/ticoann/WMCore

commit 3b6c3bbbbbfb99f95611a3985b897d581d005a66
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Thu Oct 4 19:51:43 2012 +0200

    Upload the toPublish.json with the user's credentials. Fixes #4135

commit ded33be1b221e5f1d3236cc3f7c6ab69afff1ad5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 10:41:22 2012 -0500

    Revert "Merge pull request #4144 from mmascher/publish-proxy"

    This reverts commit 469f51ca2b54479a96131e02747af56f9f480d6e, reversing
    changes made to 9f194da829987c68f8c40e88798581c78f1c72fb.

commit eb70d76e6bbda24b7da37016df931a5dc6c784a0
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Tue Oct 9 14:30:15 2012 -0500

    Monitor layout change and add support for tier0 and analysis use case

    commit eee10963627df7bcd1769e0e16f94e9807651794
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Tue Oct 9 13:22:28 2012 -0500

        add more data for detail request view

    commit cbd26b3d85dd4f94b984043b9740c200b5741437
    Merge: ce51a80 b93f0a9
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Mon Oct 8 16:21:46 2012 -0500

        Merge branch 'tier0_bug_fix' into pull_monitor

    commit ce51a8049baa8ce036d3aa68721628cd9d2edcfd
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Mon Oct 8 14:56:45 2012 -0500

        Change the layout and add warning mechanizm add support to alanysis use case
        Squashed commit of the following:

        commit b93f0a9f020bb42dff4cce650f88a4523db44a4a
        Author: Seangchan Ryu <sryu@fnal.gov>
        Date:   Mon Oct 8 14:51:06 2012 -0500

            fix the layout and campaing and site table

        commit 9078bedd70087b939bc55cf96e2dc6c0d21b9f18
        Author: Seangchan Ryu <sryu@fnal.gov>
        Date:   Wed Oct 3 14:53:57 2012 -0500

            add user view for analysis monitor

        commit 69c1d77707ec221434603fde650fd2f6a363b857
        Author: Seangchan Ryu <sryu@fnal.gov>
        Date:   Mon Oct 1 15:24:46 2012 -0500

            add GQ pull warning

        commit a8c36d776f47f552b62106ea0b6162ee33a0a96b
        Author: Seangchan Ryu <sryu@fnal.gov>
        Date:   Fri Sep 28 11:29:16 2012 -0500

            major layout change. task level job report - still need to work on GUI

        commit 8fa5f9a305dba74e40a6d2f489af8b38aa658a51
        Author: Seangchan Ryu <sryu@fnal.gov>
        Date:   Fri Sep 21 16:43:30 2012 -0500

            redone layout for monitor work in progress

        commit 0f134001c5c2ea4d6bd64024d6abb6322c8adcee
        Author: Seangchan Ryu <sryu@fnal.gov>
        Date:   Tue Sep 18 14:01:09 2012 -0500

            add status window and get db from the url string

    commit b93f0a9f020bb42dff4cce650f88a4523db44a4a
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Mon Oct 8 14:51:06 2012 -0500

        fix the layout and campaing and site table

    commit 9078bedd70087b939bc55cf96e2dc6c0d21b9f18
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Wed Oct 3 14:53:57 2012 -0500

        add user view for analysis monitor

    commit 69c1d77707ec221434603fde650fd2f6a363b857
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Mon Oct 1 15:24:46 2012 -0500

        add GQ pull warning

    commit a8c36d776f47f552b62106ea0b6162ee33a0a96b
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Sep 28 11:29:16 2012 -0500

        major layout change. task level job report - still need to work on GUI

    commit 8fa5f9a305dba74e40a6d2f489af8b38aa658a51
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Sep 21 16:43:30 2012 -0500

        redone layout for monitor work in progress

    commit 0f134001c5c2ea4d6bd64024d6abb6322c8adcee
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Tue Sep 18 14:01:09 2012 -0500

        add status window and get db from the url string

    import campaign for analysis

commit 19d0e75b1a84ba1f358688a4861f5600884645c6
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 09:45:42 2012 -0500

    Revert "Revert "Merge branch 'master' of github.com:dmwm/WMCore""

    This reverts commit c147f1f916940710bf25c6fe7d5c3bf881fed0ff.

commit c147f1f916940710bf25c6fe7d5c3bf881fed0ff
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 09:44:57 2012 -0500

    Revert "Merge branch 'master' of github.com:dmwm/WMCore"

    This reverts commit 70b35db240423d1c3e0e890ea1015d9ef40748a2, reversing
    changes made to af68aa7062ce2b46fe34c911a3b8e135f577c35f.

commit 70b35db240423d1c3e0e890ea1015d9ef40748a2
Merge: af68aa7 469f51c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 09:40:15 2012 -0500

    Merge branch 'master' of github.com:dmwm/WMCore

commit 469f51ca2b54479a96131e02747af56f9f480d6e
Merge: 9f194da 7fba920
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Oct 10 07:18:10 2012 -0700

    Merge pull request #4144 from mmascher/publish-proxy

    Upload the toPublish.json with the user's credentials. Fixes #4135

commit 9f194da829987c68f8c40e88798581c78f1c72fb
Merge: 553b7c6 d3c3f2e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 9 11:23:31 2012 -0700

    Merge pull request #4145 from mmascher/condor-proxyui

    Take the UI from the conf instead of using the sourced one. Fixes #4141

commit 553b7c672612863d7569a4c7482d6c1396161998
Merge: bddd819 524d1e4
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 9 11:12:15 2012 -0700

    Merge pull request #4011 from mmascher/monitoring-view

    Added status to the requestByDate monitoring view

commit 524d1e48c6c542e1c1fce9087f1c2f8a8ac18fa4
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Fri Aug 17 17:41:12 2012 +0200

    Added status to the requestByDate monitoring view

commit d3c3f2e054fcfa75bab1b8ba269cf025c1228691
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Tue Oct 9 13:40:45 2012 +0200

    Take the UI from the configuration instead of using the sourced one. Fixes #4141

commit 7fba92068737bf27fe9d397dcac64cc1dcb3dc18
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Thu Oct 4 19:51:43 2012 +0200

    Upload the toPublish.json with the user's credentials. Fixes #4135

commit bddd819c330af51f2ce0f5298a5d1c9a2093dcb6
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Oct 5 15:08:06 2012 +0100

    Add ACDC integration test

commit 12aa1dd266cd00c15975a0d4d0e95b3dec384283
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Oct 5 12:39:48 2012 +0100

    Remove debug printout from requestDb

commit 4f2d204efeeb3c2611218dcdfde7cba31ad920c1
Merge: 0e98906 d41aed8
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 4 13:10:03 2012 -0700

    Merge pull request #3958 from dballesteros7/fix-reqmgr-unittest

    Fix reqmgr unittest

commit e1cee43b7c55c1e366cc428d2b8683a1f063c5d8
Author: cinquo <m.cinquilli@gmail.com>
Date:   Thu Oct 4 18:43:21 2012 +0200

    Removing extra slash from analysis spec #4131

commit d41aed8a046e9bdd6d1513555a8d288949dd1b99
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Sep 6 20:05:52 2012 -0500

    Fix RequestManager unittests

    - Add default performance parameters
    - Make sure WMStats is deployed in couch during the tests

commit 0e98906cf5de0b6d6805349df85af3aadd24a2c2
Merge: 18af991 d4f7378
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Oct 4 07:35:54 2012 -0700

    Merge pull request #4130 from samircury/workload-summary-couchapp

    Adding page to visualize requests which are done and summarized

commit d4f73784ac52cfdbf5ffbbcfa1e45a2a5ce0019f
Author: Samir <samir@cern.ch>
Date:   Thu Oct 4 16:15:42 2012 +0200

    Adding page to visualize requests which are done and summarized

commit 3df405cd59ad71c30a5fb44eafe7e72f8a4c7920
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Sep 28 14:28:17 2012 -0500

    combine taskarchiver and cleanup manager to handle the case when reqmgr is not used
    (need to be tested.)

commit e15f8f2f498992044bf032ea57334d87a8e95e6b
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Oct 3 15:41:53 2012 -0500

    Bugfix for validation of well-formed dqmHarvest

    Validation depends on having a self.procScenario in stdBase
    this holds in all cases but ACDC workflows, this bug
    prevents from creating ACDC workflows of requests
    with DQM harvesting included through the webinterface (it is still
    possible through script by passing a dummy 'scenario' parameter).

    This patch changes the validation to check the steps in the harvesting task
    and make sure the scenario was defined, this covers all workflows including
    ACDC.

commit 815a70e9d755fe0f8d734c77587cdf4baaaee427
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 2 22:49:17 2012 -0500

    Support for the new splitting algorithm in reqmgr

    Reqmgr modifications to include the new algorithm

commit 75aa3b4432d3feea59b8de63091cc12deef5f89a
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Sep 25 13:35:15 2012 -0500

    New splitting algorithm

    The purpose of this algorithm is to dynamically split jobs by lumis
    trying to make the jobs have a target number of events which is
    ultimately the unit that determines job running time, disk space used
    and memory constraints.
    However this differentiates from the EventBased algorithm since
    it doesn't split indiviual lumis, the basic unit for the splitting is the
    lumi and every job is guaranteed to have at least one full lumi, this implies
    that the number of events in a job will never be the exact target
    but on average all jobs have about that number.
    Also this includes the option to define a hard limit on the events
    a job can process, if a file has a single lumi and more events
    that such limit that job is created and marked as bad. This means
    that this job will be moved from created to createdfailed by the
    JobCreator.

    The ErrorHandler is now configured to immediatly exhaust createfailed
    jobs. The reason is that the createfailed state didn't make much sense,
    the transitions defined it to go from createfailed to cooloff and then to
    new. But a job that is in createfailed, had to actually be created otherwise
    it wouldn't exist in WMBS. I changed the transitions to allow from created to createfailed and changed the transitions from createdcooloff/paused to new
    for created instead which seems to be the logical step, since jobs in "new" state are not picked up by any piece of code.
    However for now the createcooloff state is unreachable as the ErrorHandler will always take createfailed and send them to exhausted.

    The algorithm takes the following inputs:

    - events_per_job : This is the main parameter and defines the target events we want in each job. The algorithm will split lumis according to average events per lumi in each file
    - max_events_per_lumi: Hard limit that defines jobs to fail when there are too many events in a lumi
    - halt_on_file_boundaries, splitOnRun, etc..: Usual LumiBased algorithm parameters

commit 18af991e089789600cb2851c7150978c5449f05e
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Oct 3 16:52:54 2012 +0100

    Fix string formatting error

commit 7c408bd34512402c9d0db08dc1ada1d206eabaab
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Oct 3 11:30:34 2012 +0100

    Remove hardcoded wq url in integration test

commit 6a3a75e75e7d2234a4e541945fe44890f5a04df9
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Oct 3 11:06:14 2012 +0100

    Fix integration test reqmgr url

commit 3414e74c1da078d2e99a3b1965e6cef6e9e6c854
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Oct 3 10:40:07 2012 +0100

    Make Reqmgr url configurable in integration test

commit c36906abd96916c944fb00151bf1bb8d2059251f
Merge: 821a9b5 bc47a52
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 2 09:02:03 2012 -0700

    Merge pull request #4125 from dballesteros7/fix-taskarchiver-bug

    Fix TaskArchiver not cleaning outputs

commit 821a9b52e1de5df8e35d83f972ae8184f89979af
Merge: bd408c3 ce81e8f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Oct 2 09:01:20 2012 -0700

    Merge pull request #4079 from dballesteros7/reduce-couch-queue

    Reduce default couchdb queue size for ChangeState

commit bc47a527ec63dd341788f91d6aaf0c0da3d73a30
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Oct 2 01:22:49 2012 -0500

    Fix TaskArchiver not cleaning outputs

    In the last task archiver fixes I missed loading a workflow
    when loading the subscription before deleting it which
    made the workflow have empty output maps.

    Modified the unit test to catch this

commit bd408c30810708bd5de726ed27be290b2da0b27a
Merge: 7648a3b d1debc6
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 1 08:01:36 2012 -0700

    Merge pull request #4120 from mmascher/udeepable-error

    Avoid the error: un(deep)copyable object of type <type 'thread.lock'>

commit 7648a3b8c5ef0df3a6610631461684447cd88a51
Merge: 6707191 adfbb1e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Mon Oct 1 07:58:26 2012 -0700

    Merge pull request #4123 from mmascher/fix-inputAsyncstageout

    Fix inputAsyncStageOut view: delete undefined task variable

commit adfbb1e8af2cf93efa23d9b504498f5201c47929
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Mon Oct 1 11:30:26 2012 +0200

    Fix inputAsyncStageOut view: delete undefined task variable

commit 6707191e6d54decb75b877109e268e1fa5df8059
Merge: 9a7007f 65bebd3
Author: Stuart Wakefield <stuart.wakefield@gmail.com>
Date:   Mon Oct 1 02:11:39 2012 -0700

    Merge pull request #4122 from mmascher/log-wg-rm-interface

    Add log instruction in WorkQueueReqMgrInterface

commit d1debc60727c81974c26530c703ce68f5acda587
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Fri Sep 28 11:42:17 2012 +0200

    Avoid the error: un(deep)copyable object of type <type 'thread.lock'>

commit 65bebd31cfc78ac893fb87393d071e53e4d150d8
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Mon Oct 1 01:01:45 2012 +0200

    Add log instruction in WorkQueueReqMgrInterface

commit 9a7007f14220ad3a311e62a2f28d8b2c742c3a91
Merge: 7184fa9 e17adaf
Author: Andrew Melo <andrew.melo@gmail.com>
Date:   Thu Sep 27 14:55:21 2012 -0700

    Merge pull request #4119 from dballesteros7/fix-taskchain-unittest

    Fix TaskChain unit test

commit e17adaff56dd159c1afb571be6854f7b007e3233
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Sep 27 16:48:31 2012 -0500

    Fix TaskChain unit test

    This lines shouldn't have been committed. We need to use
    the DBS emulator to do pileup stuff, however this test doesn't
    check anything related to pileup.

    Fixes #4114

commit c46acea3a7d2b4d1181f362eb2ff40e427c6c133
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Sep 27 14:44:23 2012 -0500

    Merge the Tier1PromptReco and PromptReco specs

    They do the same thing, no need to mantain so much duplicated code.
    I tested this against the T0 unit tests besides the usual WMCore tests.

    Also added a few optional parameters to make it easier to setup
    a testing environment to run the tests which involve skims

    Fixes #3994

commit af68aa7062ce2b46fe34c911a3b8e135f577c35f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 15:45:21 2012 -0500

    0.9.19

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 20e611e391a7f12ba30b21b768861d404b670f48
Merge: eb4df0c 7184fa9
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 08:06:08 2012 -0500

    Merge branch 'master' of github.com:dmwm/WMCore

commit 7184fa9277f6dd59ae3da7a0805173f7adfcfa07
Merge: 857a8cf 772ab97
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 06:04:35 2012 -0700

    Merge pull request #4065 from samircury/t0-backlog-monitoring

    adding view for backlog monitoring in T0

commit 857a8cf1cdef4e0d761624b285113f946f3d1b04
Merge: f9c7c8f 8169065
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 06:03:48 2012 -0700

    Merge pull request #4080 from dballesteros7/default-job-state

    Add the "New" state to the LSF plugin map

commit f9c7c8f43f52f12705b52fc1684d51fd5d216d1e
Merge: c90a391 786269f
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 06:03:34 2012 -0700

    Merge pull request #4100 from sfoulkes/bigger-lfns

    Increase the size of the LFN column in DBSBuffer.  Fixes #4907.

commit c90a391f8a6bdb0789ea2d3cf81206d5f6814a69
Merge: af26045 032d09e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 06:00:06 2012 -0700

    Merge pull request #4101 from HassenRiahi/RemoveTaskParam-1

    Remove unneeded parameters.

commit af26045924e990ee81ed47b64979338d6006c23a
Merge: 2324175 34c004c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 05:59:55 2012 -0700

    Merge pull request #4106 from HassenRiahi/AddParamsToAsyncConfig

    Set the values of ASO parameters in the config.

commit 232417507080031c35318400735fe917dce94aaa
Merge: 3c88c94 1e3de90
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 05:57:19 2012 -0700

    Merge pull request #4109 from dballesteros7/fix-pickle-dump

    Write the configuration as a pickle in binary mode

commit 3c88c944e7dedf22a3f193be6e9b3f0af020a63f
Merge: 42d91f3 75c09cd
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 05:56:46 2012 -0700

    Merge pull request #4068 from hufnagel/promptreco-logcollect

    Tier0 PromptReco does not support LogCollect

commit 42d91f32b0c71f63a907a61ca29b90f7bf4d2fac
Merge: ffd6ed9 d85469c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 27 05:53:50 2012 -0700

    Merge pull request #4070 from stuartw/patch/Testing/integration-testing

    Add first pass at ReqMgr integration tests

commit 75c09cda023bfcd86d76cd9bcc778bec1256a8d5
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Thu Sep 27 14:37:53 2012 +0200

    make logCollect optional for PromptReco

    Changes to the PromptReco Spec to evaluate the DoLogCollect spec parametere and use it to configure with or without logCollect tasks (default is logCollect enabled).

commit ffd6ed9fbb40ebd93a903b29e73b0d46eadf5a64
Merge: 86a6497 5b5a4de
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Sep 26 14:15:12 2012 -0700

    Merge pull request #4073 from dballesteros7/taskarchiver-fixes

    Taskarchiver fixes

commit 86a6497007e6ce3fd213dca13c379ccdc242a29e
Merge: 29a17cf 940ea08
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Sep 26 14:14:24 2012 -0700

    Merge pull request #4105 from hufnagel/alcaharvestfixes

    some fixes for AlcaHarvest

commit 29a17cf2a408bd15297f60b11d57dbb7ebe1e225
Merge: f64f0f6 bae0d9d
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Sep 26 14:14:04 2012 -0700

    Merge pull request #4096 from samircury/pausealgo-couchupdate-fix

    PauseAlgo  - making the state change propagate to couch

commit f64f0f6b6bc4185f881d980fa29792cbd005a600
Merge: afc6e9a 1b78055
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Sep 26 14:13:52 2012 -0700

    Merge pull request #4111 from samircury/paused-jobs-propagate-summary

    Paused jobs propagate summary

commit 1e3de9027715f4544f73d0c5957f84b7140e3afc
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Sep 24 16:42:40 2012 -0500

    Write the configuration as a pickle in binary mode

    Make a suitable PSet that loads the binary pickle and stores it as
    the process for the cmsRun command

    Fixes #4104

commit afc6e9a8e3a5e51f833729b9907f8d47e397380e
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Sep 26 12:48:11 2012 +0100

    Make teams optional for global queue.

    If not given, requests for all teams will be acquired.

commit 8e26fc1b577001746ffddd2d3663b34c20d8dbbe
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Sep 26 11:55:15 2012 +0100

    Fix place where couch conflicts could block request completion.

commit 1b7805520f9937b6ab31acb4729ddb973f2868a8
Author: Samir <samir@cern.ch>
Date:   Wed Sep 26 10:37:36 2012 +0200

    propagating to job summary when failing or resuming jobs

commit 8d70aeba1c789fcba5550d3a0510374cfc76c794
Merge: 857b559 bb69ddc
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Sep 25 08:48:13 2012 -0700

    Merge pull request #3969 from mmascher/lumi-mask

    Put the lumiMask in the spec instead of using ACDC. Fixes #3968

commit 857b5590539dce542c7e14a319018eecc16ca5db
Merge: aae947d 67566b0
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Sep 25 08:46:43 2012 -0700

    Merge pull request #4107 from mmascher/mispelled-running

    Fix simple typo: running was runnig.

commit 67566b0cbd23d9ddf76042c56c752e388cc522d0
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Mon Sep 24 12:28:03 2012 +0200

    Fix simple typo: running was runnig.

commit 34c004c58f94a40fc427405711d3627b9c46f1fc
Author: Hassen Riahi <Hassen.Riahi@cern.ch>
Date:   Sun Sep 23 15:39:55 2012 +0200

    Set the values of ASO parameters in the config.

commit 940ea086f52b0a95ab6f9d75bf89d1e943bd62d7
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Sat Sep 22 10:45:07 2012 +0200

    some fixes for AlcaHarvest

    Remove the validation mode checks as that is all handled in the upload
    code now. Also always write both the regular and validation destDB
    records into the metadata file, the upload component will pick
    the correct one.

commit eb4df0c3a9aedf1c546272c5dfe8654399a8fd63
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Sep 21 11:21:40 2012 -0500

    0.9.18

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit a1cf914ac2ab3dd02a9af0697a144ce7ad2b0d8a
Merge: 84f1de5 aae947d
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Sep 21 11:20:37 2012 -0500

    Merge branch 'master' of github.com:dmwm/WMCore

commit aae947da1b0c40b8fe343a4faf0e8b27b4152e15
Merge: 7dec085 386e2e5
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Sep 21 09:20:09 2012 -0700

    Merge pull request #4103 from ticoann/wmstat_auth_fix

    fix wmstats validation

commit 386e2e5eb7e8f1afb3b0c7765ad84741adfa45ad
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Sep 21 09:56:31 2012 -0500

    fix validation

commit 032d09e6683b63d7cbae500927edfb18bca625b7
Author: Hassen Riahi <Hassen.Riahi@cern.ch>
Date:   Fri Sep 21 00:26:41 2012 +0200

    Remove unneeded parameter.

commit 786269feca740c065ce4cc235bf26b2432b54c5a
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 20 12:55:43 2012 -0500

    Increase the size of the LFN column in DBSBuffer.  Fixes #4907.

commit bae0d9d2fe59b1f609d8c2eb914a2a7fe54ebdea
Author: Samir <samir@cern.ch>
Date:   Wed Sep 19 15:34:08 2012 +0200

    making the state change propagate to couch

commit 84f1de529dce306aa5c79058f5308177bec1dd29
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Sep 18 16:16:45 2012 -0500

    0.9.17

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 7dec08509e3c5b0f315b6fd6e296673b88c21536
Merge: 929c952 fbaef98
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Sep 18 14:04:51 2012 -0700

    Merge pull request #4092 from ticoann/fix_cleanup_mgr

    fixes #4088 (cleanup manager won't crash when local couch fails)

commit 929c9527a0cb3c6dd65edaaae37f2893f3edf875
Merge: 21dbd04 6cf3a48
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Sep 18 14:04:33 2012 -0700

    Merge pull request #4093 from dballesteros7/make-harvesting-optional

    Add the option to disable DQM Harvesting

commit 21dbd04ea3514e147115cd8b9fff79d9064eb1dd
Merge: 1c3240a fbaf16e
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Tue Sep 18 14:04:20 2012 -0700

    Merge pull request #3961 from dballesteros7/retry-manager

    Make the RetryManager more flexible

commit 6cf3a4862c19c449150ff240e4b5799d084c735c
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Sep 17 13:37:08 2012 -0500

    Add the option to disable DQM Harvesting

    People may not be ready for this feature and may want to
    turn it off, this patch gives that option

    Add also the testbed versions of the dqm gui urls

commit fbaef988830e237ae21992091449fe534cae9e00
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Tue Sep 18 10:00:42 2012 -0500

    fixes #4088

commit 1c3240ae96538ef04adbb2cad4334110342763db
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Mon Sep 17 11:27:19 2012 +0100

    Fix failing request timeout using wrong start time.

    Fixes #4087

    Timeout in queueing a request was measured from the time
    the request was first seen in global rather than when it was first seen
    by the current queue.

commit a20f5cab243be0ad8c774721a41c50a0c661b216
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Sun Sep 16 21:58:48 2012 -0500

    0.9.16

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 5b5a4de28764d4c4d2bf8b3a0e134f7b21597e63
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Sep 5 17:21:50 2012 -0500

    Pull lumi information on failed jobs from WMBS

    Instead of using couch to get the lumi information for
    the failed jobs, get that information from WMBS.

    Fixes #4038

commit 3e178f61933c0236b987649254e513ba41a7c870
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Sep 5 12:02:44 2012 -0500

    Make the TaskArchiver more robust

    This commit has one purpose, avoid missing summaries from the
    workflows.

    The changes make use of the previous commit to identify completed
    workflows and delete them only after we know all its subscriptions
    are finished and its summary is safe in couch.

    Now the deletion process is more sporadic but everything goes in one
    go.

    Fixes #4055

commit a627d3c1fc89c23587478c8d3964c3595fc4be82
Merge: 2da8e17 6e2e0da
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Sep 14 14:37:57 2012 -0700

    Merge pull request #4084 from dballesteros7/taskchain-pileup

    Fix the support for PileUp in TaskChain requests

commit 71de139f0086a2afd660e7987f3d70d03b2d63f2
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Aug 30 15:22:30 2012 -0500

    Support for finished subscriptions status

    Adds a new column to WMBS to mark subscriptions as finished,
    adds new DAOs to handle this column and to find
    workflows that have all its subscriptions finished

commit 2da8e17d29e0660403f5d4dc217327314b968a5a
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Sep 14 16:31:36 2012 +0100

    0.9.15

    Signed-off-by: stuartw <stuart.wakefield@gmail.com>

commit 2426056f6b9886ae115b2046fb39bf922e3235ea
Merge: 9eeff19 5b8db66
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Sep 14 08:35:59 2012 -0500

    Merge branch 'master' of github.com:dmwm/WMCore

commit 5b8db66c57aa38ea9e162ef83b2d0fd147bb5c2e
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Sep 14 11:29:22 2012 +0100

    Fix bug where open blocks wouldn't be closed in global

commit d6575e8165e6014d90b663fe011e1e25ba812882
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Sep 14 10:41:08 2012 +0100

    Skip adding new files to open blocks in global queue

commit 9eeff198be5e805cfe2f22f10f135f27b074d5ef
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 16:04:33 2012 -0500

    0.9.14

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>

commit 4d0fae1344c619fb21eaf75f769673074c8f1890
Merge: fea93ed b75b131
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 16:03:16 2012 -0500

    Merge branch 'master' of github.com:dmwm/WMCore

commit b75b1313e24199ac49cf3c7aa87424dace42fd5d
Merge: d0548d5 f38bd65
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 13:56:28 2012 -0700

    Merge pull request #4062 from hufnagel/worker-errors

    cut off error message passed to DAO

commit d0548d5c2129a037d2a8e4603bd99502017a0df6
Merge: 60eacbb 5f07869
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 13:55:33 2012 -0700

    Merge pull request #4082 from ticoann/wmstat_update_fix

    Wmstat update fix when request doesn't exist in monitor but exist in reqmgr

commit 60eacbb72b929c1b7cfbfae195ab11ca6098157a
Merge: 17c349d 600700a
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 13:54:59 2012 -0700

    Merge pull request #4059 from ticoann/refactor_reqmon

    refactoring wmstats monitoring including tier0 support with runs and jobpaused status.

commit 17c349d28fd29caebf3bab04c110ec9526710cc5
Merge: 11db7bc ebfd16c
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 13:53:04 2012 -0700

    Merge pull request #4077 from dballesteros7/fix-rc-unittest

    Fix ResourceControl unit test

commit 11db7bc78fd964c1028dc60cfefe1a5b07064f69
Merge: a1addc3 ca02be4
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 13:52:38 2012 -0700

    Merge pull request #4064 from hufnagel/dbsupload

    fix Tier0 DBS2 upload problems

commit a1addc37f27af97f8db7ad1f1d01d294205bb6c2
Merge: 174b15b 606d879
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 13:52:14 2012 -0700

    Merge pull request #4061 from dballesteros7/paused-jobs-options

    Add options to the paused jobs tool

commit 174b15bb7ac506abd12d1bbaa826c8e9d1359e79
Merge: 9d7c9b6 60284d1
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 11:06:13 2012 -0700

    Merge pull request #4085 from mmascher/fix-jobarchiver-JSM

    Put a transition for killed job in JobArchiverPoller

commit 9d7c9b629582a96e848f47b431a423a1f1e0c9e9
Merge: 44f9f10 75abb02
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 13 10:55:16 2012 -0700

    Merge pull request #4083 from dballesteros7/fix-condor-jdl

    Fix Condor JDL

commit 44f9f10c85b75231c3c700d59b7c8cad5d709861
Merge: 2359ebf d911b69
Author: Stuart Wakefield <stuart.wakefield@gmail.com>
Date:   Thu Sep 13 08:45:23 2012 -0700

    Merge pull request #4081 from ticoann/rm_couchskel

    remove couchskel link from the script. wmagent.spec 1.131 needs to be used

commit 60284d14b0cb27cefa5dc70a7dcdff808a158077
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Thu Sep 13 16:58:06 2012 +0200

    Put a transition for killed job in JobArchiverPoller

commit bb69ddce4b32258a906044a6fca4e1f4de149901
Author: Marco Mascheroni <marco.mascheroni@cern.ch>
Date:   Tue Jul 31 19:15:33 2012 +0200

    Put the lumiMask in the spec instead of using ACDC. Fixes #3968

commit 6e2e0dae944e2d3aa32f50a5c7eb0e016185f044
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Sep 12 16:35:42 2012 -0500

    Fix the support for PileUp in TaskChain requests

    Add the option to specify pileup information for each task individually

commit 5f0786939781f7d0bd274c8ab1a6fd1b9d3fef00
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Sep 12 18:02:26 2012 -0500

    add unit test

commit 25016b7d5a8c7b1b351cc805fd6cfb161b36b78d
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Sep 12 17:09:32 2012 -0500

    fixes when requests need to be updated but not in the wmstats monitor

commit 75abb02784f83e36aef68db31c57a954674684eb
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Sep 12 16:38:01 2012 -0500

    Fix Condor JDL

    Newer condor versions enforce quotes around strings

commit d911b69bfd50a72fe8c667b0078388328e0393e0
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Wed Sep 12 14:30:17 2012 -0500

    remove couchskel link from the script. wmagent.spec 1.131 needs to be used.

commit 2359ebfb91289a46a58ff5c3d43a1f80d8802bd0
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Wed Sep 12 12:23:25 2012 +0100

    Modify build script for wmcore/wmagent rpm merge

commit 8169065ff099f1b4adc2544b8d11ca57e69f0447
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Tue Sep 11 09:50:59 2012 -0500

    Add the "New" state to the LSF plugin map

    Even though the initial state of BossAir can be configured
    as PEND for agents with the LSF plugin configured, if
    we want an agent that uses both condor and LSF plugins that option
    would not work anymore. Therefore the LSF plugin
    should map the New state to Pending as well.

commit ce81e8f8d931f56a96ed3921a9c43711dd40d3dd
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Mon Sep 10 15:35:42 2012 -0500

    Reduce default couchdb queue size for ChangeState

    1000 seems to be too much in the busiest machine (e.g. vocms96)

commit 606d87926198674c8af9f7d21125c471b5c12725
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Sun Sep 2 21:21:21 2012 -0500

    Add options to the paused jobs tool

    Add the option to filter by site and task

commit d85469cf775e44781d62f224927d7350a5e72fd6
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Mon Jul 30 17:11:43 2012 +0100

    Add first pass at ReqMgr integration tests

    Signed-off-by: stuartw <stuart.wakefield@gmail.com>

commit 3a69483240d899275256441936fcbb8be12bdd21
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Mon Sep 10 14:18:42 2012 +0100

    Improve workqueue cancellation error handling

commit 15492908ef58838188d7dd42cffc409764cc8d93
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Fri Sep 7 11:16:37 2012 +0100

    Improve WorkQueue handling of conflicts on cancel.

    For some reason update calls fail sometimes when canceling work. Replace
    update with a full save.

    Fix #3856

commit ebfd16c04e78424e96c34122a72487517437761b
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Sep 7 00:23:54 2012 -0500

    Fix ResourceControl unit test

commit b8e5664e9003ec727052f7685ed7b2cb6f0b7f7f
Merge: b86f3de 7206feb
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Thu Sep 6 13:31:40 2012 -0700

    Merge pull request #4075 from dballesteros7/bugfix-reqmgr-lhe

    Bugfix LHE request web interface

commit 7206feb3371f31e28bdec3be4830f7b495d60cf4
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Thu Sep 6 11:50:05 2012 -0500

    Bugfix LHE request web interface

    Remove the duplicate time per event field

commit fbaf16e75fc1bef7dad362e1617ae7cfc091a091
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Fri Jul 27 01:17:04 2012 -0500

    Make the RetryManager more flexible

    This patch allows the RetryManager to be configured to use
    different retry algorithms for different job types.

    Also do some bugfixes in PauseAlgo.

commit b86f3de9500725defd6ec74155b4bc521d21fd04
Merge: 5756399 9166e3b
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Sep 5 11:29:38 2012 -0700

    Merge pull request #4071 from dballesteros7/add-harvesting-threshold

    Bugfix for harvesting tasktype

commit 5756399b35a54873f0fc2a7a1405a16222f2f725
Merge: 7508b3e 5239d4a
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Wed Sep 5 11:29:16 2012 -0700

    Merge pull request #4072 from hufnagel/siblingcomplete

    optimizing SiblingSubscriptionsComplete DAO

commit 9166e3b17beb7c6daeddd334d2dae022196bc1dc
Author: Diego Ballesteros <dballest@fnal.gov>
Date:   Wed Sep 5 12:09:42 2012 -0500

    Bugfix for harvesting tasktype

    Add it to the defaults in wmagent-resource-control
    Also fix a reference in the Harvest algo unittest

    Fixes #4069

commit 5239d4a54d24ded086da19b836f7c7cf88a26e1d
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Wed Sep 5 18:05:41 2012 +0200

    optimizing SiblingSubscriptionsComplete DAO

    Should optimize the query (it can be an expensive query) by starting
    not will all available files but only the files actually in the
    input fileset to this cleanup subscription. Really suprised this
    actually worked before.

commit 772ab973db90b5b55471d1daa01e30d722baa1cf
Author: Samir <samir@cern.ch>
Date:   Mon Sep 3 18:47:21 2012 +0200

    adding view for backlog monitoring in T0

commit f38bd6538653db98d50d3d94126fcde4dfd35d5d
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Mon Sep 3 16:38:57 2012 +0200

    cut off error message passed to DAO

    Database column for the error message is 1000
    characters, therefor cut off the error message
    passed to the query

commit 7508b3e9fcfe6872d6bef4db17173b614b243e3d
Author: stuartw <stuart.wakefield@gmail.com>
Date:   Mon Sep 3 11:34:49 2012 +0100

    Fix typo in MockPlugin. Fixes 3931.

    Signed-off-by: stuartw <stuart.wakefield@gmail.com>

commit ca02be425c130496acc8f1b63074315472640c61
Author: Dirk Hufnagel <Dirk.Hufnagel@cern.ch>
Date:   Mon Sep 3 10:56:21 2012 +0200

    fix Tier0 DBS2 upload problems

commit cbbf8f9db8186ea2423d67b5b2044fbdd1868dd0
Merge: 27cd23b 931a6f3
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Sat Sep 1 15:46:07 2012 -0700

    Merge pull request #4060 from zdenekmaxa/alertsfwpollerfixes

    Alert emails from address set to noreply. Process pollers fix on disappared child processes.

commit 931a6f3aa43041eb221086c4b9ee280301094325
Author: Zdenek Maxa <zdenek.maxa@hep.caltech.edu>
Date:   Sun Sep 2 00:33:27 2012 +0200

    Alert emails from address set to noreply@cern.ch. Process pollers refreshing list of child processes each poll interval. Alerts no longer generated on disappeared processes (can easily be re-enabled).

commit 600700a2c0060bd2fdb44e4fe33848ccd4cd3b48
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Aug 31 15:11:44 2012 -0500

    cleanup views

commit 720ce377ed320637e92c64333dcfc12aa75b3580
Author: Seangchan Ryu <sryu@fnal.gov>
Date:   Fri Aug 31 15:04:09 2012 -0500

    refactor wmstast add support for tier0 run view implement caching

    commit c3f3aae438edda819078d0dc4c0c1a77b8264349
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Aug 31 14:57:25 2012 -0500

        clean up

    commit ad6d3abcf368f20c8df615ab334d8fde67c1a04e
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Aug 31 14:06:32 2012 -0500

        move back to original

    commit 1ba8cedbf21f9d1b1162d8469b955f1d89f31c1d
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Aug 31 13:34:40 2012 -0500

        rename the files and refactor

    commit c2e59a26f98c1d478dd8a2e3449f3f4e9cb30a98
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Aug 30 11:09:00 2012 -0500

        add categorized view and refactor the codes

    commit 9d6d192349b8a0a735b330e40f96d272c6279af0
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Aug 9 16:52:29 2012 -0500

        refactor request structure to handle tier0 specific overview

    commit 6233b1718b1d6b3b1045d32789a84372d321e689
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Wed Aug 1 14:15:47 2012 -0500

        add jobpaused status in tier0

    commit add74550c0266bf0956ee4febac57e1de9814367
    Merge: 0b7d738 e8568b1
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Tue Jul 31 15:38:32 2012 -0500

        Merge remote-tracking branch 'upstream/master' into add_jobpaused

    commit 0b7d738d549abd4f7a7a722606f349713fc045d0
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jul 27 15:32:50 2012 -0500

        revert the change to make the patch

    commit a52830a2297c65c002b0809147d0daa4d9c2a864
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jul 27 14:37:59 2012 -0500

        revert couchdb change - this already made pull request.

    commit 843c16c3c716a3bffed1f96dbcb566163d636808
    Merge: 04aca5a e263cb2
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jul 27 14:27:14 2012 -0500

        Merge branch 'role_fix' into tier0

    commit 04aca5afadcb03eaf6e4732e13b7cc418bcf4ef5
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Fri Jul 27 14:24:00 2012 -0500

        add active request tier0 support

    commit ff64f0f42b2d5de625c6519615897b6b786d466f
    Merge: bbc8f05 72b3c84
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Jul 26 10:17:32 2012 -0500

        Merge remote-tracking branch 'upstream/master' into tier0

    commit bbc8f05c5b349c5845e0b165c070bd6e18ed4585
    Merge: 2474d2e 87f7135
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Jul 26 10:12:44 2012 -0500

        Merge branch 'master' into tier0

    commit 2474d2e7451ac59d131459b58a57094ada50755a
    Merge: 518be99 b2ba54d
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Mon Jul 23 14:35:30 2012 -0500

        Merge remote-tracking branch 'upstream/master' into monitoring

    commit 518be99219df200eab1c46186a25529bee13874a
    Author: Seangchan Ryu <sryu@fnal.gov>
    Date:   Thu Jul 19 14:50:21 2012 -0500

        update transition changes

commit fea93ed961c5591bc1c422651c0fd2f4279b3b41
Author: Stephen Foulkes <sfoulkes@fnal.gov>
Date:   Fri Aug 31 09:35:38 2012 -0500

    0.9.13

    Signed-off-by: Stephen Foulkes <sfoulkes@fnal.gov>