Squashed 'src/Ganga/' changes from 161626b..c7a261e

c7a261e (ganga@7040)
14c5934 strip the location of the output file
a576199 Fix for bug #90890
7a437ab Needed to handle default, string executable arguments
1eb1d94 (ganga@7016)
63def43 pass more files to the outputsandbox, the files that should be uploaded to lsg storage element
0b683b3 (ganga@7014)
ee0304a changes for uploading of the output files to lcg storage element
293d29e Correctly ship sharedir contents with job to all backends
4a4ea0e Correctly ship sharedir contents with job to all backends
c64609e (ganga@7010)
06d57ae (ganga@7009)
03f6f99 changes for uploading the output  to lcg storage element
563bf6c update the postprocess file on resubmit, there could be some changes in the outputfiles between submit and resubmit
c1f2388 reduce duplicate entries of the outputfiles list
1fcd2b7 fix for automatic output file type detection
ee9fd29 raise config error if filename is defined more than once in output config section
f886074 config default value changes
ea20bf3 remove repeated entries from input and output sandoxes
550f1c0 add non repetead entries for input and output sandbox
872ebb4 (ganga@6999)
0de9e1a (ganga@6998)
55bdda6 (ganga@6996)
452b0a2 kepp the postprocessoutput file always updated
625e3c3 (ganga@6991)
582f13c (ganga@6987)
1ff7fdc (ganga@6986)
392a22e (ganga@6985)
51aebcb (ganga@6984)
280e479 (ganga@6983)
bf4fde8 (ganga@6982)
877046b (ganga@6981)
4c7b417 (ganga@6977)
646864b changed config values
b8973e1 (ganga@6972)
296ff84 (ganga@6971)
7230bb8 (ganga@6970)
888384f changes for retrieving the output from the lcg se
d2df61d Check for existence of shared directories when we copy an application
7279c82 (ganga@6963)
4d0af17 Catching this exception was causing problems elsewhere
813c6c2 Iterating over each prepared application to check that it's sharedir is present is too resource intensive
b27236e changes for uploading the output of CREAM jobs to lsg storage element
7f7823d (ganga@6953)
a066c2d (ganga@6952)
285e01e (ganga@6951)
03e2180 changes for the output files
01adbe8 added get method that in principle should retrieve the output file locally
d444cf4 Reverting changes made in revision 6936 (dynamic checking of ShareDir presence)
04a0dbf (ganga@6938)
242ac72 Switched to dynamic checking for presence of shared directory, rather than loading very job object when we exit ganga (which can be very expensive).
d4a58a5 Removed commented line
d22ed03 set the default objects properties to config values
351a35c instead from configuration , get the lfc host and se from the output object itself
d355331 (ganga@6926)
bec421b setting the locations of the outputfiles
3ec3485 added methods for setting the location of the uploaded output files
eb8ec34 changes for uploading of the output to lcg se
6300af9 (ganga@6922)
5b33627 (ganga@6919)
e3321c3 (ganga@6918)
1ad2cbe (ganga@6916)
b3f67e3 (ganga@6915)
8af5c3c (ganga@6914)
de2e88d source code reorganisation
48e9a21 (ganga@6912)
cd379fe changes for uploading the output to SE
8e1b154 (ganga@6897)
f791c76 Fixes for shareref counter and broken write-protect on protected=1 attributes
df92e24 Tweaks to fix sharedref counting
caa3099 The unprepare method is now required
45dd661 changed config value
bf7f09d (ganga@6889)
f558f3b fixes for post processing of output
0d9bfe5 changes for postprocessing of output
2b67443 Undoing the last commit. You fool
231c4ca Fix for bug #89869
2851c16 changes for postprocessing of output
7d83ef8 We don't need to export the unprepare method for the Executable app
90d614a (ganga@6881)
f918d1a (ganga@6880)
9894260 (ganga@6879)
4c6fe80 (ganga@6878)
9afa458 check for post process output file existance
b921ac9 adding the postprocessing file to the input sandbox
ad72a9f added config section for lcg se output storage
f730438 changes for output stored on lcg se
3ba4e2a check first if this postproces locations file exist
1747f28 (ganga@6856)
eaf6fa2 postprocesslocations file could not exists, so make a check first
ac343f1 (ganga@6839)
e7f7612 Implementing feature request #89441
d6615a4 uncomment and empty the location list in the constructor
a642ed3 uncommenting
16e2352 commented one method
39c66ef commented one method
828dd8b (ganga@6831)
783d322 (ganga@6830)
bb91544 (ganga@6829)
e3cdf97 put in the locations list only diff paths
27a4b98 (ganga@6827)
fd9ee8b (ganga@6826)
ebc89f1 (ganga@6825)
1376365 (ganga@6824)
ef45191 set location of the output files uploaded to mass storage
3774593 (ganga@6822)
58f0063 small fixes
1d2d294 set location of mass storage output file
444c0c7 setting location of output files stored on mass storage
043ecca (ganga@6818)
639aee2 implemented method for local retrieval of outputfiles uploaded to mass storage
fbf28db small fix
eaa4276 (ganga@6815)
474f16f added comments
6c3ca49 remove the postprocesslocations file after job is completed
f67c6e6 added abstract methods for location of the output file
d7ffc16 added method for the location of the mass storage files
9d7cc44 (ganga@6810)
58877ac set locations of the outputfiles
2d1583c small fix
ed8c0c0 pass info in the outputsandbox for output file location
9a0b8fc use glob instead of regex for names of the output files for post processing
c658478 use glob for post processing of output instead of regex
18631a4 use glob instead of regex for output file names
e202239 removed comments
4da1b65 small fix for glob
a972dff check for putputfiles pattern with glob instead of regex
8f2e060 use glob instread of regex for file name patterns for output files
96dd1be removed not needed code
2be0bf6 small fix
e9bcb51 replaced regex string with wild char string for output files
336bfd6 added new method of handling of the output
880c354 imports of file objects, added posibility to instantiate them from the config
b3f255f import file classes
6892589 fixed tabs
1fb9c18 added config options
7ea8435 added config section
09f925d Removed startup message informing users of new XML repository
d0f57ab Enabling prepare() method for Root application
8172290 Profile log messages are too verbose to have at INFO level
fd125cd Slightly improved the tasks.table() formatting
7756320 Committed on behalf of Mark
70c2540 Fix committed on behalf of Mark
dfdc7bc postprocess of output changes
4c4736f some clean up
ed0af47 post processing of output
ded2907 more debug info added
72ef818 debug info added
d5ba1dc more fixes
f6a1300 more fixed for output post processing
02958ae more fixed for output post processing
b168016 more fixes for post processing of output
ca04a5b small fix
a2b4368 add postprocessing output to the downloadable files on the grid
dcb2e61 changes for postprocessing of the output
3991fb1 import subprocess package
5bc070e Fix for bug #89110
f3234a9 forgot to import re
0506bcd add shared area to the report generated from the report() function
4f6f82c changes for postprocessing of the output on the WN, upload to Castor
949957d removed comment
4c50a53 added postprocessing of the output for CREAM on the client, for Castor uploads
3bcb00e added postprocessing of the output on the grid, upload  to Castor mass storage
4ed9b63 removed handling of the output at the client for Local backend
7fa1f3b if there is no CASTOR_HOME env variable set, set the default path to the home directory of the user in Castor
2d269fc more bug fixes
bca9c39 more fixes for mass storage uploads
08e3b9b small fix
6337687 small bug fix
bf8d0bd put the castor upload on the clinet, just for test
39ece71 small fix
43652e6 small config fix
8ac6e57 expect regex instead of exact names for postprocessing of the output
5b7a719 fix for compressing output in the Batch, expect regex instead of exact names
a52de74 file removed
48ca84d (ganga@6680)
500a18f (ganga@6670)
0975cd4 (ganga@6669)
80af4e4 commented one config option
b984f81 test
fae51d6 test
0b683be test
aed557c removed checks for castor upload, if there are error they are written to the stderr of the job
56f0646 changes for Castor uploads from Local backend, write to stderr if there are problems with access to castor and copying the files there
83b8ea2 bug fix
8e8f857 making tests
c27e3ac Feature request #88651; make unprepare_on_copy a configurable option within gangarc
ce27c9e bug fix
30de1c5 code reorganising
91ec564 write to postprocessoutput file instructions for castor uploads, also check if castor uploads are made from lxplus, only from there they can be done
02cbd4f changes for Castor uploads for Local backend
ac19114 added section for mass storage output
127153d mass storage file added
0e1f07a output file for mass storage
01d6dac Implemented j.time.runtime() for batch backends (Bug #86537)
6d28a28 Merging 5.7.0 from Ganga-ConfState branch into Trunk
f984c27 changes to the job script for compressing the output if there were such instructions from the user
75171fd changes for the post processing of the output
e04f2cb added OutputFileItem class for the outputfiles category that is used for the post processing of the Ganga output
ba758f9 added configuration for the Output section - this defines which output files to be processed how
ef6d67a added method for postprocessing output in the Ganga client, for batch submissions it is not possible to postprocess it on the worker node , i.e. __jobscript__ file
02c03dd added postprocess method that will define the way of psotprocessing the output for backends that can not postrocess it on the worker node
d19c00d import the output files
b52ff15 added another class for post processing of the output files
0411ea9 another class for output file post processing
ac1f24a added class for output file that have to go to mass storage
7a33971 added file that will mark output file for compressing
abd9d5c added class that will be base class for outputfile classes which will define how the output should be postprocessed
3db917a Fixed Savannah Bug #87477
20410be Making tasks openable again if in completed status
f1fa8f4 Fix to Patricks ReleaseNote stuff
cfe95ae Added hook for change in master job status
74d674d Adding Patricks ReleaseNote printing
544e97b Improve documentation for box object
b1f7da8 Improve documentation for box object
c9c7cf1 fix the parser on glite-wms-job-list-match output to recognize the CREAM CE
d54906e Committing on behalf of Mark Slater
8c97eeb fixed 'fix' not working properly on master jobs, accept panda reporting of ddm site instead of analy_xyz for output data
88f1ff2 Fixed attribfilter for non name attribs
bb49de4 Fixed Savannah bug #85332 options file check not expanding env vars
3975dd6 fix for bug 77962 and for test case test/GPI/auto_resubmit/auto_resubmit
c7ff7e6 Minor syntax bug
5106d01 Changes to allow more intuitive and efficient use of FileBuffer.
1401fdc (ganga@5954)
b6df083 (ganga@5909)
1a8ade2 Bugfix #83902
208f589 Upgrading stomputil to 2.4
21b4aa7 Adding username/password for connection to the ActiveMQ server. Note that these are not sensitive passwords.
83c56ac Implemented Mark's changes to enable the use of environment variables in .gangarc and .ini files.
9e5a830 Added --no-mon option to disable the monitoring loop at runtime.
8802d83 Removing redundant popen2 import; we now use subprocess.Popen()
05eee60 Put something useful in the job's monitoring link attribute.
9e78135 Added a username/password to the stomp connection code; without this the MSG server gets upset.
e9002c5 fixed bug #81654 by moving cwd from the schema into the python object
0287460 demoted error to debug to remove follow-on errors from partially loaded tasks. fixes #78715
0c8616a - bug fix in using LCGSandboxCache
be8e6e7 locking for jobtree cleanlinks fixed
f21877b Committing files on behalf of Mark Slater. Some updates to Tasks didn't make it into 5.5.26.
31d2aaf Quick fix to print lists of Ganga objects with few items in a more meaningful way.
dccae5c added releaseWriteAccess to Objects and made jobtree release its lock to fix bug #74327
1b5ecff Merge GangaLHCb-Tasks branch 5424:5641 into trunk
871406d Messaging server renamed from gridmsg101 to two aliases (ganga.msg.cern.ch and ganga-atlas.msg.cern.ch)
a9f9dc6 - handling environment variables with "space" characters
507362a - force check the remaining credential lifetime when delegating proxy to CE
c5c5872 - add debug information
8cd8931 - add debug information
fa463e5 - add debug information
c953b41 - optimize the proxy delegation to CREAM CE
b02f168 - optimize the proxy delegation to CREAM CE
0a9eaf8 - optimize the proxy delegation to CREAM CE
458e7cf - optimize the proxy delegation to CREAM CE
36cc802 - add resubmit functions
6872483 - add resubmit functions
e8e3b91 - add resubmit functions
c8a9798 Reverted move from tasks to core. now only in LHCbTasks branch.
c167062 some fixups from the tasks migration
ce144ae missing __init__.py
e1f2436 moves TaskRegistry, Task and Transform into Core without changing any functionality
aa35e26 minor logging fix
52c7143 using bulk resubmission for automatic master job resubmission
aaac7f2 fix auto resubmission
dacbbe8 Bugfix for #77962; an unnecessary call to increment() was causing j.resubmit() to increment the submission counter by 2.
0efaa1b reverted accidentally commited local changes
b303864 fixed auto-resubmission algorithm: for plain jobs it takes into account MaxNumResubmits, for master jobs it takes into account MaxNumResubmits FOR INDIVIDUAL SUBJOBS
9e2beca more user-friendly checks for validity of arguments passed to runMonitoring()
2562db5 Fix for Bug #76973; Job templates had 'new' state and could be submitted
6273b65 add fix for bug: https://savannah.cern.ch/bugs/index.php?77180
582c584 remove DQ2SandboxCache.py from LCG backend - move it to GangaAtlas
d2d8a7a Added bulk monitoring of all jobs from a single remote machine (Bugs 76808, 76295, 76294)
29638d6 fixed: SmartMerger now inherits from IMerger
a84a38c added IMerger.set_outputdir_for_automerge flag as a fix for previous bugfix 76445
fff43e0 additional check hook for backends to restrict when auto resubmit happens
96c7930 fixed the flawed rasubmit condition (which otherwise would allow to auto resubmit an empty list of failed jobs)
d83c4e6 small modifs to auto_resubmit trigger condition + a basic test
a33796c implementation of auto_resubmit() from M.Slater with some modifications from me
97ee64a fix for bug #76733 - set ganga version before saving the user config where ConfigError can occur if some config section is not found
21f2d03 leave the output directory to the implementation (fix for http://savannah.cern.ch/bugs/?76445)
f159e22 bugfix https://savannah.cern.ch/bugs/index.php?76520
ae741ad use globus-url-copy (instead of uberftp) to download output sandboxes
706cd87 (ganga@5205)
46221c1 (ganga@5204)
6d5612e added util module
add14ad changed name of the LCGAthenaUtil module with AthenaUtil
aadb0ba bug fix
3688d4d bug fix
5f47c01 bug fix
9c11cb9 bug fix
732aac0 bug fix
07de065 fixed bug in the module for backends other than LCG and CREAM
68cf5e0 bug fix
fcfe83b bug fix
e6f7cc5 completed and failed method made to return compelted and failed status of the job
4f2ab18 add one more module in getSandboxModules in case of Batch submission
ed95c62 moved import tarfile block a few lines above due to import error
b857b5b JobTree will now handle job lists and slices, as requsted in Bug #74531
695e6dc fix for backend which use default implemenation of IBackend.master_submit() but which do not support keep_going flag
b274644 fixed way of retrieving of panda monitoring link
a202cbe added util module for Panda
539bbb2 set mon links for dashboard and panda
d6374dc removed simplejson
c7a4e88 updated monitoring link
7f98812 mon links is a list of tuples now
ec601d3 mon links is list of tuples now
138c667 made mon links list of tuples
5090b07 python 2.6 not compatibility error fix
5223e22 delay the sys.path.insert call in job wrapper - fix for different import behavior of PYTHON 2.6
b1f7e7d fixed odd import problem for python 2.6
d1826a6 fixed odd import problem for python 2.6
e034aa2 support for keep_going=True with j.submit()
4b3327b show an error to the user if problem occured while preparing the error report
6ccfdcc improved version of backend option for j.resubmit()
df8d16b implement bulk killing for a master job
af05e35 added a possibility to change backend with j.resubmit()
4b6f8c6 fixed typo
a3466a5 bug #74402: forcing into completed from failed does not work
1100a6a bug #73328: Make Job.submit args configurable
a95ea57 fixes
8f2e1a5 file added
a3b74bf file added
2bc7df0 file added
15f8c32 removed method
87740da set default middleware flavor from EDG to GLITE
7709824 fixes
4399989 fixes
201af52 fixes
e98a8b0 fixes
857aad5 fixes
547f9c0 fixes
3dbe788 fixes
0f0f284 fixes
6923b18 fix
65274f9 fixes
e61e105 fixes
1fb13fc fixes
d7f5b68 tests
5651c54 tests
9b9e461 added parameter to a function
d23f4ff pass parameter to a function
54ec707 file added
a01daa8 file added
5cb8c27 file added
1501ec8 file removed
02158d3 file removed
4c685a5 fixes
f4d816f fixes for generic backend MS
0fe99b3 tests
7151f27 tests
d5a231d tests
9cfe29a tests
a593f66 tests
a5e4295 tests
43d9606 added file for test
1faf6b8 added file for test
c5b7ef3 fixes
64ba8b6 bug fix
0984a4f test
f036fd5 test
81a8bcb test
11736cc test
d9ee3a4 added for test
8bd6cf0 this module should handle monitoring of all backends for Athena application
c603ec2 this module should handle monitoring of all backends
03bdba2 include in the message that is sent the runtime packages so that we can get the experiment
3a10c26 place each mon link in a div, so that in the UI they apper one under another in the links column
aa38bc1 bug fix for #74231
6230632 some warnings are made infos because they are not that important
cf97300 make heartbeat frenquence configurable
313c903 bug fix for bug #73416 - job slicing was wrongly documentated
d8e4667 Added online help to jobSlice GPI function
bcd0313 JobInfo property renamed from links to monitoring_links
1bfc814 JobInfo property renamed from links to monitoring_links
618cc90 JobInfo property renamed from links to monitoring_links
0878368 fixed bug when there are no compelted jobs - the JSON creation for the chart succedded jobs in time failed
e867c53 added links property to the JobInfo schema that will be list of monitoring links
0bd1db6 overrided submit method and added two monitoring links - this is for test
643bd47 added monitoring links - extracted from job.info object, added test JSON
5eecf2d upload the report if it is less then 100 MB, otherwise print message that it is too big and it was not uploaded
38c37d2 print to the user the most important info that is collected in the tarball
6b74178 get the submission queue from configuration
c0b896b added spyware infroamtion of submit and resubmit of jobs
eb294d1 added config setting for the name of the queue where job submission mesasges are sent
8419ebc added function for sending messages to msg queues when job is submitted
402dfd5 cleanup of timestamps and updateStatus
d89edc7 forgot to make jobSlice function use an ordered dict instead of an regular dict
b094668 Added jobSlice function that makes a slice from a job list
a75d1bf added methods for returning another google charts
fcf7597 added colors in the returned json for pie charts for statuses, the colors of the pie charts should be the same as the status column colors
c4d2bd7 load all jobs at begining, not only for the last month, capture the messages that are coming from the http server and log them as ganga debug messages
bae5acf refactoring and added functionality for returning json prepared for drawing pie charts for jobs and subjobs for status, backend and application
4f1ae96 added support for numeric arguments in executable application, fix for bug #71343
8504c17 get fully qualified id instead of id, bug fix for bug #70758
3bfc5d9 get fully qualified id instead of id, bug fix for bug #70758
ba68474 fixed bug #69216: .ganga.log surprising behaviour
0be0dc3 fix: bug #68896: ganga - glite-wms-job-list-match fails - reports full output is in file ... - but file does not exist
3123f93 patched by S.Schmitt: bug #70712: SGE() should have information about job's worker node
b56a0da gather info for the pie charts for jobs/subjobs statuses,application and backend  that will be displayed, return this info as a JSON
7ed381b refactoring and preparing data for pie charts that will be shown in ganga --webgui
b9a5b83 print of the jobs added in the tarball
7c4b971 fix for bug #67690 - include Ganga. at the begining of the logger name so that it can inherit from Ganga logger
4781b4e bug #70796: Message after force_status not illustrative - My status completed completed
e30a5b1 path to ipython hsitory file is different now -> savannah bug #71912
6332d0b Fixed typos in psychiatrist
6a629bc add debug message to check how many output downloading agents are still alive in background
9c29e24 bug #71563: DeprecationWarnings in Core when run with python 2.6
09af31a bug #71912: Use custom ipython directory to save ganga history and configuration
abedc15 fix: bug #71512: BackendError is not propagated to the user
26b34fc fixed bug 44116: app.postprocess() may now raise PostprocessStatusUpdate('failed') exception
b288537 added extended attributes to usage monitoring spyware and more configuration flexibility
445beaf change config of DashboardMS: use production server gridmsg101 and rename topic names to better describe what they are
7a458a5 1) extra check (via postBootstrapHook()) for outdated config files for Atlas monitoring dashboard. 2) added loading of DashboardMS config section with other core plugins
a19cf53 fixing bug #70849 - when removing a template it was saying removing a job ..
5725cf6 fix bug #53939 , first set the id of the subjob and then append to self.subjobs
cb5be46 reverse the order of arguments subjobs and sum_outputdir, fix for bugs #71333 and #71376
7048773 fix the bug in getting md5 checksum
658bc17 some fixes for the report function to handle also subjob as argument
d9cda6a fix import typo
acfd4d0 replaced dummy user in the list with the user that has started ganga
d51d41b remove license note
9e1e307  - add additional module "Compatibility.py" to handle python 2.6 compatibility issues  - replace "md5" with "hashlib" if python version larger than 2.5
daa1922 remove useless imports
03542a3 added gangarc file - the default configuration in the report
e13cc39 Fixed typo (bug #70750)
a90fc3c fixed removing incomplete jobs
d8fcd1a  - fill CE attribute value for all subjobs  - add automatic CE selection if the getce method in the requirement is provided
9c8e347 Job object was not recognised
cc4e22e set time period to lastMonth by default
01040c0 fixed returned link to return the local user, made jobs that have TestSubmitter backend for example to show in the page -> previously we catched an Error and skipped this jobs
d36ea66 fixed returned path to page, added exception handling for attributeError and registryError caused by testSubmitter backend
eb7b768 bugfix #69715: traceback "raise Error" in WNSandbox.py
ba9a3b2 bugfix #69715: traceback "raise Error" in WNSandbox.py
b0e4ac3 untabify
a288a9c CREAM handler added
2d4b29a returned link to the page from where the jobs can be monitored, fixed bug when exiting the session whit ctrl+D
c240551 added a new hook postprocess_failed() which is only called from completing->failed, this avoids recursion with exisiting postprocess() hook and will allow to separate the two functionalities more cleanly
b5d78ac web-based monitoring GUI
72d9c76 bug #68547: A function to monitor some jobs by hand?
20209c5 bug #53751: Release reported wrong when checking out from SVN
f696a02 bug #55503: Jobs should not be able to be removed in completing status; bug #55952: postprocess_hook never called for failed jobs; bug #64514: ApplicationConfigurationError triggers WARNING instead of ERROR -level messages; bug #67876: Re-running the merger should be more straightforward
694b023 bug #52815: Color customisation for jobs display
c809b2f enabling CREAM backend in LCG module
4d4428b purge the jobs on CREAM CE if output are successfully fetched locally
b389260 full implementation of GridftpSandboxCache
32c3beb add support for parallel job submission
d66199b fix md5sum calculation
656f0f8 add sandbox cache module for gsiftp protocol
f615f9c patch
a453ab7 patch
29fdfe9 patch
4c7ffc2 patch
6fa1560 patch
7a26de4 patch
9ff9f16 patch
7b5a031 add proxy delegation for CREAM jobs
ada8273 patch
14eff83 patch
5d82feb patch
2d01219 patch
9647525 handling oversized inputsandbox
546c113 update schema
f256d80 introduce CREAM CE direct job submssion (preliminary test only)
e0d38ab introduce config.LCG.StatusPollingTimeout to avoid blocking monitoring thread if glite-wms-job-status call is stuck.  The default timeout is set to 5 minutes.
cdf6944 disable the out-dated Octopus monitoring service
51eb88b fixed bug #68800
c3faa48 fixed bug #68535 make check for the type of the parameter passed to the report function
9465255 Backend now caches passwords for keys Setup now only opens the transport on startup
7f3265e Fixed bug #68317
7d6c08f reverted to an older (patched) version of ipython
296bad6 Added polling framework to Registry/Repository
db281ef - remove unused import: tempfile
f77c5aa Only kill other session locks if monitoring loop is active
638dc97 Added check for monitoring_component.enabled in Tasks loop and Session cleanup loop
8f77c56 Reformat taskname attribute of messages. Update task_meta and job_meta message.
64464ef - apply timeout feature in calling glite-wms-job-submit command - submission timeout is configurable via config.LCG.SubmissionTimeout.
3706283 bug #67938: LCG backend blocking during submission ?
667b8b3 fixed: [bug #68006] Submit counter broken for subjobs
234a657 Use MSGUtil in UsageMonitoring as intended instead of MSGMS.
85eb964 rollback to previous rev 3446
7212b94 applying new user dataset naming convention for dataset created by DQ2SandboxCache
a0a16c3 ignore stdout/err from sourcing dq2 client setup script
f74a0a8 Update stomputil to version 2.3.
8691cc9 ignore stdout/err while sourcing the dq2 setup script
3eeb6d0 Fix bug #67799: MSG traceback during job submission (Check for undefined inputdata / outputdata)
4318527 Fixed bug - was loading from backup instead of properly failing like it should be
6fe51a4 full_print function was not recognized - added import for that
8f1b627 do not get the whole IPython history, just the last 20 commands
2f94a7a user config was not getting the changed values, but the default ones
2fa10a6 fixed upload server path
0ec6b45 store the archive in tmp folder of the mashine, delete the tmp archive always - even on error occured, also delete the temp upload file that is internally created when uploading to a django server
439969c syntax fix
34ac535 fixed bug #66804: subjob resubmitting error: cannot compare datetime.datetime to lis
6a9ed8b filename fix
3f1cfa4 integrated feedback report into GPI
461efc0 some work for Ivan
5d37cef Separate out meta-data utility methods. Code formatting.
18a6927 changed server upload path
1936258 upload error report file on a server
89569f1 Move logger creation into MSGUtil
208a69c Upgrade to stomputil 2.2
5dda699 * fix bug #64041 - subjobs can now also be deleted in the repository
9e36f28 report returns the physical path, not uploading to a server
551f05a fixed bug #64512
656a26f Add more debug logging in shutdown method
8fe9ae8 * try loading from backup file 'data~' if loading from 'data' fails * added IncompleteObject for cases where one wants to reload or remove objects where load failed
4f4829b * make 'select' available to all RegistrySlices * add incomplete_ids() to RegistrySliceProxy
4b459b3 * fixed jobtree.getjobs() * display now uses job slice display
61325ec added uploading of the error report to a server, fixed name of the archive
0a5c158 fixed logger messaged: job id -> fqid
5ab7847 fix in the heartbeat
5bf10d7 test cleanup
e10a097 Second part of fix for bug #65797
a62c66f Fixed bug #65797
f0f4b02 added heartbeat to fix an AFS problem: 'touch' is not propagated correctly anymore across multiple AFS clients at CERN
ce0c0bc Pass DashboardMS logger to Publisher
34e76f9 ipython setup: sys.path->PYTHONPATH
e30e5ae bugfix 55568
be3526a conditional transition message from 5.4 -> 5.5, will be disabled after some time
b52eba9 move to ipython 0.8.4
34004b8 Fix bug #65444: config['MonitoringServices']['Athena/LCG']=None breaks job submission
e19b42b converted tabs to spaces
1e640d0 this is my first commit blah...
fe22593 Added GangaValueError for proper error reporting if wrong constructor argument
e6ee405 Make Job(JobTemplate()) and JobTemplate(Job()) work (allow down-copy and up-copy in class hierarchy)
0c64b14 Changed TypeError to TypeMismatchError (GangaError)
60cb27d Introduced __construct__ function to GangaObject. Constructor positional args are passed to this, by default this implements a copy constructor If a GangaObjects want to implement a constructor, it should implement this function, check if the args match and if not call the superclass __construct__
fc16e11 make a new configuration variable: IgnoreGliteScriptHeader to turn on/off the trick for 32/64 bit library compatibility issue.
a1c2388 minor fix
fa8d142 print JDL when no matched resource.
2a4c000 fix 32/64 bit issue with glite-wms-job-status and glite-wms-job-logging-info (script-based) commands
a442de7 bugfix #31690: Empty the outputdir of the subjob just before resubmitting it
5af4117 bugfix #63157
0509e67 bugfix: #31690: Empty the outputdir of the subjob just before resubmitting it
98837d9 bug #55568, ganga -o option does not accept =
79a56fd bugfix - bugfix #64081
81875dc bugfix #59165
d331c58 Add a global flush in Batch job wrapper scripts (in some situations the status of a Batch job was not updated until the statusfile was closed at the end of the job's execution). The flushing function looks now as follows:
14f80b7 Fix bug #62543: Exception in thread GANGA_Update_Thread_shutdown (Avoid accessing globals between sleep and exit tests in shutdown and stomputil daemon threads)
1917fb8 Stopped continuous retries with lots of jobs (bug #58527)
69f6604 change logic of activating monitoring plugins: use all plugins which match all patterns defined in the config, rather a first pattern only
678248c Fix bug #64080
45b16da Fix bug #64080 Check for incomplete XML file
7a098f7 change the default dq2 client setup path
aa06ca7 Fixed wrong behaviour for 'subjobs' fields without Ganga objects Fixed AnaTaskSplitterJob losing information
8b2d56f remove lcg-infosite query when initializing Grid object, call it only when LFC_HOST is needed and not presented in user's shell environment
9474ac5 update lcg-cr command (replace -T to -D in SE type specification)
8460928 - skip calling lcg-infosites when LFC_HOST is already presented in user's shell environment - reduce the number of credential.voms query in job submission routine - remove all glite command log after the system call is finished
b1d3187 add debug messages
70ee4bc Fixed subjobs not being marked as "clean" after flushing
f339f63 (ganga@3250)
013ee84 to be replaced by the XML branch verison 5.5.0
cc6e2c6 Add pypath and pyldpath in front of dq2-get calls
a5ba350 Split lcg-getturls call in chunks of 50 for DPM
001cbea Remove dq2clients requirments
10a224c Remove sqlite download
e5e025d Update to 0.1.32, user10
5e13299 Change order of glite UI and athena setup
99ab211 Don't use local cmt_config. Get version dependent string from AthenaUtils.
f029422 fix typo
af5a83b use the libds cloud if libds supplied
127c464 disable outputdata.location. Refer user to DaTRI.
53e044d Fix 4 digit release brokering for Athena and AthenaMC
f01fa76 Minor bug fixes for stupid errors introduced previously
0d5906d Automatically updated template for 5.5.0
ec70faf Added PACKAGES-5.5.0
535edfa Added ReleaseNotes-5.5.0
f4549b8 Fix AtlasProduction tag problem
933d8eb Automatically updated template for 5.5.0
abe940a Automatically updated template for 5.5.0
f18bd6d Automatically updated template for 5.5.0
10b3ba5 Added PACKAGES-5.5.0
be2715e Added PACKAGES-5.5.0
da0f8bd Added ReleaseNotes-5.5.0
ffd7d75 Fix PhysicsAnalysis submission tag
85059f4 make accessmode unhidden to work with GR
3600e00 fix for https://savannah.cern.ch/bugs/?62031 add hidden accessmode for FILE_STAGER
c9df114 * Added GRL support to AMIDataset * Removed bad combinations of grid/local options * Made 'ALL' the default for the cloud in LCG * Fixed TAG bug - now TAG files are downloaded on the WN
e710eb7 Merge AMIDataset to trunk
5a20a41 Automatically updated template for 5.5.0
fbf2f81 Automatically updated template for 5.5.0
f45c54c Automatically updated template for 5.5.0
72ab3c6 Added PACKAGES-5.5.0
07e024c Added ReleaseNotes-5.5.0
5f96c59 Added PACKAGES-5.5.0
aa39022 Automatically updated template for 5.5.0
ba89ca3 Automatically updated template for 5.5.0
f851865 Automatically updated template for 5.5.0
ca516f2 Added PACKAGES-5.5.0
bf778bb Added ReleaseNotes-5.5.0
2410325 Update externals
b7733c3 Change log message
8e34d8a Modified JEM startup banner
db2e877 Code documentation
7ee2e21 Update to panda-client 0.2.12
c9b328a Fixes for stomp data transfer, edited default configuration values, bumped referenced JEM version
12f2651 * Add doc strings to critical property methods
d3d3c67 Bumped referenced JEM version to 0.2.7. This version includes the new StompValve to use stomp (ActiveMQ/JMS, gridmsg-Server) for data transfer.
7a09b24 * set stomp.py default logging level to CRITICAL
7bc8aa5 * Remove debug print for batch shutdown
8eece0c * Set STOMP publisher thread as non-critical
a8e770e * Add non-critical flag to GangaThread. * Prevent Ganga shutdown hanging for non-critical threads.
8b9f287 Add AtlasLCGRequirements.list_release_blacklist() and black-list release warning in AthenaLCGRTHandler.py
8792585 (ganga@3131)
a118fb2 move pandatools.Client import into functions that use it. This prevents ganga startup blockage if the panda server is down.
b15f8b3 Add clean_duplicates_in_dataset routine to DQ2OutputDataset
bad8337 Automatically updated template for 5.5.0
4c013e1 Automatically updated template for 5.5.0
80f7742 Automatically updated template for 5.5.0
dd937ae Added PACKAGES-5.5.0
31804b3 Added ReleaseNotes-5.5.0
9d323d5 Automatically updated template for 5.4.5
d0553ca Added PACKAGES-5.4.5
e0993c7 Added ReleaseNotes-5.4.5
d09b38b Fix 32/64 bit setup and pybin problem for slc5
d812bd4 Fix 32/64 bit setup and pybin problem for slc5
7dcb129 send cmtConfig with jobs
27ccbb1 Automatically updated template for 5.4.5
18b14a6 Added PACKAGES-5.4.5
1bf80d6 Added ReleaseNotes-5.4.5
ec52bc0 Automatically updated template for 5.4.4
3920b62 Automatically updated template for 5.4.4
f917a58 Automatically updated template for 5.4.4
a19785d Added PACKAGES-5.4.4
d0a9e75 Added ReleaseNotes-5.4.4
6770cac Add missing DQDatasetExistsException exception
e1b8eca add overall dataset size to info log
2d01f36 handle the unlimited filesize case
673a8a9 Changed splitting algorithm to better respect the filesize constraint:
ed95a5e Add option for WLCG formatter to include microseconds in date format
0c84036 Fix cmtconfig SL5 problem
86dcd9b Fix cmtconfig SL5 problem
69002a5 Fix return value of DQ2Dataset.get_content(size=True)
d0c9805 Automatically updated template for 5.5.0
bd4d27d Automatically updated template for 5.5.0
723b6e8 Automatically updated template for 5.5.0
e0cab42 Added PACKAGES-5.5.0
89f7370 Added ReleaseNotes-5.5.0
9ec251f removed debug pritnout
eab1616 Automatically updated template for 5.4.4
513f38f Automatically updated template for 5.4.4
e9ea7c3 Automatically updated template for 5.4.4
e3e6ee7 Added PACKAGES-5.4.4
d989e66 Added ReleaseNotes-5.4.4
ba25a0b looking for all input files instead of files with 'root' suffix in the given local directory (e.g. the data09 AOD doesn't have .root extension)
b2b45c0 bug fix in generating the input.py
c0aa4af (ganga@3050)
b8b496b added MSG-based usage monitoring
bbfcae4 Minor fix creating temp dir
6dc3daa Change preJobO code to always prepend
5068dad Automatically updated template for 5.4.4
6992a9d Automatically updated template for 5.4.4
2c2fdaf Added PACKAGES-5.4.4
ca885ba Added ReleaseNotes-5.4.4
6f6ee07 bump external JEM version
36c41c4 Added PACKAGES-5.4.4
be6f9f7 Added ReleaseNotes-5.4.4
e23ae8f (ganga@3029)
4d2665c (ganga@3027)
3ee3428 (ganga@3026)
ba919c4 (ganga@3025)
05423c4 Fix DPM workflow
deed29c * Local, Batch backends: Enable job statistics in Athena.stats field * Local, Batch backends: Fix DQ2 output dataset creation workflow * Local, Batch backends: Fix setting of config.DQ2.DQ2_LOCAL_SITE_ID * Updates for InputFilePeeker/AutoConfiguration in   Athena/LCG/Local/Batch:   - search jobOptions if InputFilePeeker/AutoConfiguration is configured   - use preJobO.py and prepend it to jobOptions   - automatically works for DQ2_LOCAL, FILE_STAGER and DQ2_COPY   - Work-around for rfio at DPM sites: uses short TURL since Athena     bug (http://savannah.cern.ch/bugs/?59253) is not solved yet.
7b0b0de add more sophisticated event loop specification in FileStager_jobOption.py for different use cases
c0e3b1e merge the latest change in GangaAtlas-AMAAthena-new-workflow
86b7469 Remove creation of datasets and set DQ2_LOCAL_SITE_ID
ddd8b68 add subroutine for download DBRelease tarball
59e2b0d Added 'tail -f'-like live peeking to GangaJEM. Usage: j.livePeek(). This still is an experimental feature.
34b7fb7 Fixed: JEM IMonitoringService callbacks are run even if JEM is globally disabled; this may lead to debug-noise like stacktraces.
06782dc Automatically updated template for 5.5.0
823472d Added PACKAGES-5.5.0
5333546 Added ReleaseNotes-5.5.0
f4b4f93 (ganga@3004)
992cd58 Fix to support inputFilePeeker
e4d7419 Removed obsolete status vars
fc11be8 bugfix #58196
0b2ada3 (ganga@2999)
c8a48e9 (ganga@2998)
582c085 remove debug codes
86632e9 Automatically updated template for 5.5.0
d2b2006 Automatically updated template for 5.5.0
67c106b Automatically updated template for 5.5.0
58cd89c Added PACKAGES-5.5.0
909d27b Added ReleaseNotes-5.5.0
ea96188 Bumped referred external JEM version. Minor debug output addition.
1da240f Added PACKAGES-5.5.0
c38b4af Added ReleaseNotes-5.5.0
2c43e17 new workflow for setting/preparing AMAAthena application:  - specify framework-level AMA job option files by setting flags, e.g. j.application.job_option_flags = ['DATA', 'TRIG']  - convert AMA configuration into user-level job options on the client side
51573c0 add necessary changes to adopt the revised StagerDataset
e65d96a rewriting the StagerDataset workflow:  - using the same code made for DQ2Dataset(type='FILE_STAGER')  - adding the workaround to work with inputFilePeeker
c2b39e0 include AMAAthenaTask application in the new DQ2OutputDataset workflow
077262e adopts the common subroutines: filestager_setup and frontier_setup
e3b7ce7 introduce two common subroutines: filestager_setup and frontier_setup
3b831a4 bugfix in the function for making external system call
41e6f84 Automatically updated template for 5.4.3
160f4e3 Added PACKAGES-5.4.3
667190e Another fix for the TAG stuff. This time, it'll hopefully work!
9ea5039 Automatically updated template for 5.4.3
2db9027 Added PACKAGES-5.4.3
5339477 Added ReleaseNotes-5.4.3
c9aa356 Added PACKAGES-5.4.3
d04c7cb Added ReleaseNotes-5.4.3
a2b5777 Added PACKAGES-5.4.3
cc3b1df Added ReleaseNotes-5.4.3
4a9ff3c show transExitCode
1effc5c Minor fix to stop crash for non-DQ2Datasets
4619dc0 enhance the code for generating FileStager job options
691f8c2 Automatically updated template for 5.4.3
66524e0 Added PACKAGES-5.4.3
c549587 Added ReleaseNotes-5.4.3
eb1bf3d Formatting fix
cac3bc4 remove unused function
d40ed82 Fix to properly use setup the Executable path on the remote machine
85e6e8f Minor fix to stop ALL option selecting T0
04cfaa1 Additional changes for new TAG prepare method
8bf6a59 (ganga@2890)
cc0d0f7 Add AMAAthena to DQ2OutputDataset fill and fix outputsandbox problem in fill
edf5764 Fix bug #59849
b5d6e83 Fix for inputdata and StagerDataset
3090ab3 Fix for older lcg-getturl versions and add fail-over for empty usedProtocol
ac82a2f Remove dataset creation in AthenaLCGRTHandler.py
100f754 Further hot fixes for JEMs splitjob capability
dde7f91 Increased JobExecutionMonitorMS error verbosity for debugging purposes.
a856b4c workaround for */* issue. changed default listener PID to zero.
278bcc5 Automatically updated template for 5.4.2
7d7438c Added PACKAGES-5.4.2
b13c446 Added ReleaseNotes-5.4.2
c757e74 - Fixed handling of user-defined DBReleases stored in lfc with new path - Fixed removal of old files in srm when using resubmit()
6db8cf5 (ganga@2859)
7ce9498 Fix namespace issue in JEMs service handler for splitjobs.
0dc4a26 BugFix for latest ROOT versions from Manosh
84dcd45 Automatically updated template for 5.4.2
31fdc70 Added PACKAGES-5.4.2
73c12db Added ReleaseNotes-5.4.2
a736b5d fix for ipython 0.8.4+
086d56d fixed root arg bug
48d524b Added AthenaTask to Athena name at all occurrences
e52577d fixed dirac root macro args bug
43cafa2 Caching module from Manoj Jha
d0d6208 introduce new objects for running AMAAthena analysis with GangaTask
1650343 small fix for catching the exitcode correctly from athena process
1a38fbd Merge GangaAtlas-21-32-dq2 branch to trunk
720da9a Revert rev 2743 - all code is on GangaAtlas-21-32-dq2 branch
2feba82 Revert rev 2743 - all code is on GangaAtlas-21-32-dq2 branch
ca860cb Added PACKAGES-5.4.2
1a438f6 Added ReleaseNotes-5.4.2
f906f44 New libdcap.so with improved read-ahead buffer tuning
2bc0a94 Merge GangaAtlas-athena-autoconf to trunk
9a39f92 fixed ipython args
66f9f27 fixed monitoring warnings
26ff290 Fixed empty dataset bug
00853cc * Remove unused variable
fd30985 * Documentation * Add second task-meta message on complete to provide additional data * Do not send job-processing-attributes message if job failed
ba345f6 (ganga@2817)
e8d800d Bumped referenced external JEM version to 0.2.5.3
e54a15d use extOutFile to specify AMA specific summary tarball
93ee891 Updated installation section of LHCb manual
a760e2a * Add job meta message
c8ebc72 * Add job processing attributes message
73bf8d9 * Update task meta message
2e9e557 * Add task meta message
7ce206d * Use singleton publisher. * Pull out LCGMS superclass for LCGAthenaMS. * Separate meta-data builders out into utility modules.
500d285 Update to maintain unchanged behaviour following removal of overloading of complete callback in IMonitoringService.
54103c0 IMonitoringService: Remove overloading of complete callback. Add fail and kill callbacks.
8825ff2 D.Tuckett:IMonitoringService: Remove overloading of complete callback. Add fail and kill callbacks.
3868fc7 D.Tuckett: IMonitoringService: Remove overloading of complete callback. Add fail and kill callbacks
a7d0003 putting GangaSAGA on TRUNK for further development by Ole
85639a2 Begun working on https://svn.grid.uni-wuppertal.de/trac/JEM/ticket/20 - this is mostly needed for a nicer batch run possibility in GangaJEM.
6ef2ddc Made JEM work area path-independent (JEM ticket #18)
4d01001 Automatically updated template for 5.4.1
7dcd156 Added PACKAGES-5.4.1
dd0bf3c Added ReleaseNotes-5.4.1
8d8f198 Revert to module level singleton publisher to avoid creating excessive threads
512f54e Fixed extend method in LHCbDataset
b67d364 Automatically updated template for 5.4.1
31e365b Added PACKAGES-5.4.1
860dc2f Added ReleaseNotes-5.4.1
43408b1 Minor JEM stuff (updated website link, bumped referred JEM version)
e0fe607 Automatically updated template for 5.4.1
72ace37 Added PACKAGES-5.4.1
657822d Added ReleaseNotes-5.4.1
ee6558d Fixed bug in data extraction from opts file
1f437c7 New extended dq2tools
e30c34b Automatically updated template for 5.4.1
c4e8c94 Major rewrite of data handling of Athena/DQ2/LCG/Local * Clean-up and new workflows in ganga-stange-in-out-dq2.py   - New host detection algorithm based on DQ2_LOCAL_SITE_ID   - Move all SE protocol detection to ToA.py file   - Improve TURL generation algorithm and use bulk methods     and more retries   - Register output files already on worker node in DQ2 * Changes in DQ2OutputDataset   - change DQ2OutputDataset in Athena/DQ2/LCG/Local workflow     to register files already on worker node in DQ2   - Output datasets are now based on sites and dataset names     get space token name attached.   - If a master job is split among several sites there are output     dataset set per site created. These output datasets are attached     to a container at the end of the master job   - output datasets are frozen at the end of the master job   - No emptry dataset is created in AthenaLCGRTHandler or     AthenaLocalRTHandler   - config.DQ2Dataset.DQ2_LOCAL_SITE_ID should be set for a Local     or Batch backend job
9040060 Added PACKAGES-5.4.1
54ba21c Added ReleaseNotes-5.4.1
ac0b816 additional checks when caching proxy information
7d218b8 Dashboard monitoring services based on MSG. N.B. This code is under development and should not generally be used or relied upon.
2bc75b5 avoid downloading output sandbox of removed and killed jobs.
eab9fd3 load panda Client late to speedup start
fb08c94 fix
52b2ac0 delat panda client loading
529f363 unique name for test
901ddfa load panda Client.py in the functions (faster startup)
274764f fill cloud if None from PsubUtils
fe5d2e2 add usage note inline for online help
fccfb5f restrict "DQ2" mode (copying file remotely with lcg-cp) to be used with local storage
f798454 check proxy validity before updating information cache
4adf9ff Automatically updated template for 5.4.1
a1117ef Automatically updated template for 5.4.1
3b1cd9f Automatically updated template for 5.4.1
34906a5 Added PACKAGES-5.4.1
4bb5703 Added ReleaseNotes-5.4.1
c96af7d (ganga@2715)
ea83e4b (ganga@2714)
e9d30d5 add xrootd support
81add5e get default cloud from cert
46508f4 add support for StagerDataset
cb06a8c Added caching of credential information
e8f7a67 remove StagerDataset/StagerJobSplitter from AMAAthena plugin
d331276 introduce StagerDataset and it's corresponding splitter, StagerJobSplitter.
d03c042 Automatically updated template for 5.5.0
6717577 Added PACKAGES-5.5.0
a65fd6e Added ReleaseNotes-5.5.0
06f0468 Added PACKAGES-5.5.0
6cba314 Added ReleaseNotes-5.5.0
217902f Make MSGMS default configuration publish to /queue/ganga.status instead of /topic/ganga.status
49368b6 (ganga@2692)
8d5326f Remove Athena MSGMS
781558d fixes to resubmit function
b845762 Minor cloud=ALL bug fix for non-DQ2OutputDataset jobs
5462cab Use IMonitoringService support for automatically copying configuration dictionary to worker node. Extract reusable code to MSGUtil.
21ccdf7 (ganga@2672)
80de690 (ganga@2671)
09b111f (ganga@2670)
e339e68 [by David Tuckett]
6818c29 fixed cmt bug
6e3b675 Fix for bug #58122: rename instance variable "complete" to "_complete" to avoid shadowing new method "complete" in IMonitoringService.
e18b909 Automatically updated template for 5.5.0-beta
1d94136 Automatically updated template for 5.5.0-beta
93fa987 Added PACKAGES-5.5.0-beta
dbf80f6 Added ReleaseNotes-5.5.0-beta
4c2f856 Server can now handle 5000 jobs
a6a52cd indentation fix
8e0da33 AthenaMCRTHandler.py: a tabulation error in a previous bug fix was allowing only one output dataset to be created and therefore owned by the user. This is now fixed
0f3d699 Bug fix for new method validate() of AthenaMCDatasets, allowing it to work with datasets generated by jobs submitted to the Panda backend
b1e0ba1 Fixed some problems with nonexisting paths, and updated the JEM external tarball
77be226 add error code for the failure in packing output summary directory of AMA
f35278d Some quick fixes in sandbox preparation etc.
269f7cc Automatically updated template for 5.4.0
7d404eb Added PACKAGES-5.4.0
422416b Added ReleaseNotes-5.4.0
5b2e5a2 Removed the ban on t1-repro queues, per Andrejs request.
8a96498 Fixed problems with multiple lfc path conventions. I imported the dq2 version of lfn_to_native as dq2_lfn_to_native, kept the version in LFCTools.py as lfn_to_native, modified behaviour of the part of the code that adds the lfc path. Problem: Submission is not slower, as we check the lfc path for every file...
6182db6 support configurable serverMaxJobs. Default is 2000.
d918bba Fix for bug 57480, change tar option order
ddff704 enhance the event number parsing
9d919b7 add alternative way to get number of processed events: parse message from AMAEventCounter
2494a58 fix message parser to get number of staged files to WN
b4754b3 remove useless imports of python modules
8ca56db enhance the handler for uploading AMAAthena output files to a grid storage  - tar up the "summary" directory in which AMA stores ntuples and histograms  - specify the summary tarball in DQ2OutputDataset
4b136ab add new AMAAthena runtime handler for Panda backend
ad1f809 Automatically updated template for 5.4.0
b1e6b35 Added PACKAGES-5.4.0
b8dc2d3 Added ReleaseNotes-5.4.0
da1df40 Fix for groupname problem
a03dfde fix for https://savannah.cern.ch/bugs/?57112
67bccdd Added new TagPrepare application
5a5d5b5 (ganga@2596)
221144f Automatically updated template for 5.4.0
8a8d9b8 Added PACKAGES-5.4.0
93dc006 Added ReleaseNotes-5.4.0
5cc153b SLC5 DIRAC edits.
6e6ceeb revert previous change, which breaks gangarobot
2626268 fix the requirement of the pyqt version
98310e9 set DQ2OutputDataset datasetname to not copyable: https://savannah.cern.ch/bugs/?55749
1771f84 change the dependency to SLC5 external libraries
2791689 Change Milano access algorithm
3936128 (ganga@2571)
3d1bcaa (ganga@2569)
e88d0a5 introduce the AMA specific getstats script for collecting application.stats
250489a introduce the AMA specific getstats script for collecting application.stats
f80a821 updated to stomputil 2.0
6f4cd93 Enable MSGMS for Athena/LCG in Atlas.ini
1c21c83 Update to panda-client 0.1.94
c1aabc4 Update to panda-client 0.1.94 and DQ2Clients 0.1.31
4339cab add codes in job wrapper to collect simular runtime status in Athena module
42c4788 - Added auto-generation of giises.txt in work dir - Fixed some SLOW id-info from proxy, stored in AthenaNGRTHandler directly
8653b3a * Added warning that files_per_job is changed if initializeFromDatasets is used
8ab5db1 add more output files
7a196bc include env, various athena files
f32ccd5 should be working
4befd61 MSGMS for Athena. Still not functional.
21fdf93 Added support for the new method of dealing with TAGs
c6890e9 Added support for the new method of dealing with TAGs
a5c9dc2 Added the ability to supply TAG info for the dataset
40578e1 Added the ability to split TagPrepare jobs and TAG info provided
2835e5c Added the TagPrepare application to allow more flexible TAG usage
169030b Change in output dataset management: dataset freezing is no longer done automatically. Users have to run job.outputdata.validate() to do so. New application method: prepare() to append user area to the job. Added support for transforms inline option (new member transflags) and a new application method: diagostic() to identify jobs and subjobs in completed state but with some errors. AthenaMCPandaRTHandler: bug fix to prevent Client.addDataset() to be called once per submitted subjob (which causes a DQ2DatasetExists exception to be raised at the second subjob).
f4518dc changes for new Athena interface
aa04f25 changes for new Athena interface
9f4a02e Changed pyqt version
ac4924f adde external dependencies for GangaJEM
8638223 Added PACKAGES-5.4.0
47f8566 Added PACKAGES-5.4.0
aa9002a Added ReleaseNotes-5.4.0
55d8166 Added PACKAGES-5.4.0
1b14f6b Added ReleaseNotes-5.4.0
cdf2db3 Modified __init__.py to prevent loading of Panda RT Handler if there is something wrong with the Panda module (Panda.lib.Panda classes failed imports)
442062f GANGA_NEVER_REEXEC is needed to prevent rexec if Ganga runs within another Ganga session (e.g. testing framework). Note that this requirement is exactly opposite to bug #40110: "GANGA_INTERNAL_PROCREEXEC should be unset after use". Both use-cases are valid and need be supported.
a77f685 migration asserts relaxed for LHCb use-case (Mike)
1a11986 rm outdated test
c2bbd39 rm some files
3e14434 Merge LHCbSetupProject changes back onto trunk
0ccebc8 Automatically updated template for 5.3.6
6b046f6 Automatically updated template for 5.3.6
c3ebf28 Automatically updated template for 5.3.6
11844bd Added PACKAGES-5.3.6
9f4eb92 Added ReleaseNotes-5.3.6
a4161d7 changed name for old cvs-based release tool
a370a32 changed name for old cvs-based release tool
5c22528 remove DQ2_DOWNLOAD
d121950 disable DQ2_DOWNLOAD
32b53ba  - set default LCG env. setup to /afs/cern.ch/sw/ganga/install/config/grid_env_auto.sh  - fix bug in composing JDL
1cb5c75 * Update to work with stomputil version 2.0. * Remove unused files.
b5dc1a3 Small fixes for GangaJEM, split job handling, default configuration creation, simple debug statistics
d4efc3f change help link
c663124 fix the rfcp command issue with the TURL returned from CERN castor
420fd1e another way to invoke https://savannah.cern.ch/bugs/index.php?53939
6685eab Worked on the split-job capability of GangaJEM
f0a59b4 Automatically updated template for 5.3.6
b4ce43e Automatically updated template for 5.3.6
6dbab99 Automatically updated template for 5.3.6
3ac59b1 Added PACKAGES-5.3.6
7fa392f Added ReleaseNotes-5.3.6
8519ae1 Removed some obsolete warning messages. Fixed JobExecutionMonitor not being copy()-able while the watcher thread is running.
cda0a93 Bumped version of external JEM library reference to the version that'll be used in the pre (0.2.5)
1c27838 Merged GangaJEM plugin from Ganga-JEM-5.3.x-branch to trunk
687945d added comments
72c4ba6 merged from Ganga-JEM-5.3.x-branch
900e412 Automatically updated template for 5.3.6
5b2e6ae Added PACKAGES-5.3.6
d5fcc28 Added ReleaseNotes-5.3.6
82c2919 Added PACKAGES-5.3.6
4d25634 Added ReleaseNotes-5.3.6
c54932c Minor fix to sort out the environment for TAG retry
adea8c6 Minor fix to sort out the environment for TAG retry
6fd4f81 removed printout
6080fe4 Automatically updated template for 5.3.6
48fda65 Automatically updated template for 5.3.6
79d7275 Automatically updated template for 5.3.6
08b42ec Added PACKAGES-5.3.6
f1627c7 Added ReleaseNotes-5.3.6
b46f005 Minor fix to stop US/NG being selected when specifying cloud==ALL
41b56a4 panda-client 0.1.91
c6df254 DQ2Clients 0.1.29
fc693d3 fixed slc5 compiler to be 43 (NOT 34)
12c6c18 platform string for SLC5
c70673d added test for https://savannah.cern.ch/bugs/index.php?53939
36166fc MERGE GangaAtlas-21-18-access 2245:2380 into trunk
b98a5d0 Add extra ATLAS local setup for Frontier/Squid
986c813 removed typo
99611f2 Automatically updated template for 5.4.0-beta
5d9a393 Added PACKAGES-5.4.0-beta
9859bd9 Added ReleaseNotes-5.4.0-beta
20c9c11 move some Panda ptions to backend.requirements
de4aa1a Automatically updated template for 5.4.0-beta
e8a6348 Automatically updated template for 5.4.0-beta
450e8f3 Automatically updated template for 5.4.0-beta
795b856 Added PACKAGES-5.4.0-beta
846a5f3 Added PACKAGES-5.4.0-beta
f1ad18f Added PACKAGES-5.4.0-beta
463dae8 Added PACKAGES-5.4.0-beta
69f34a5 Added ReleaseNotes-5.4.0-beta
cfd5c76 AnaTransform fixes for incomplete datasets
bd46173 bug #55267: The output filename that is not present is now named in the stderr
98800df Added PACKAGES-5.4.0-beta
0477815 Added PACKAGES-5.4.0-beta
caf4689 Added ReleaseNotes-5.4.0-beta
6e651aa Extract dirac minor status from jobs
9b07b2b Fix bug 55633
5457e6d allow empty input dataset.
93fde0e Minor fix for TAG usage with multiple datasets in one job
69c5fd2 Fix setup of Athena.atlas_environment
8071705 Automatically updated template for 5.3.5
aaaeab0 Added PACKAGES-5.3.5
d7411a6 Added ReleaseNotes-5.3.5
e25ea81 prepend username to outds
badbf6d https://savannah.cern.ch/bugs/?55411
b0b57f2 Changed indentation from 3 to 4 spaces (no other changes)
0af2fd5 Automatically updated template for 5.3.4
0fc8f12 Automatically updated template for 5.3.4
3231f2a Automatically updated template for 5.3.4
06ef79c Added PACKAGES-5.3.4
b910e6d Added ReleaseNotes-5.3.4
15e1dd4 Updated uuid messages for master jobs / sub jobs
2b6c14c Upgrade to panda-client 0.1.87
ce3a158 latest panda client
cf5b234 Creating output datasets from AthenaMCPandaRTHandler.py before job submission to make sure that the user owns his output datasets
bc2f5be create output and lib datasets before submission.
870f837 (ganga@2296)
0fde5d5 (ganga@2294)
0a220ed added replicate ability + a few others
0db26d1 added replicateFile
5552e6a Fixed typo (thanks, Johannes Ebke).
e93969d Added protection for odd DQ2 inconsistency
6276605 SVN commit test
d3ff51e Fixing bug in setup-release.sh preventing kit setup on remote grid nodes.
f681e51 Small fixes
d2c5e7e rm'ing backendapp...matrix stuff
b3c442c Adding dedicated run-time handler for Local and batch backends, which are now no longer convered by AthenaMCLCGRTHandler.py. Modified job scripts and AthenaMCDatasets.py to allow for potential DQ2 support for Local and batch backends (this requires a full DQ2 installation available locally)
f5d31fb Added protection around lfc import, for systems with lfc present but dysfunctional binaries supplied.
9b6576e add workaround for compilation on SL5/64-bit machine   - search for gcc34 and g++34 and use them instead of the default ones
ce36dc1 (ganga@2255)
f2a266d Dan: added SetVerCommand
bb9a904 corrected wrong global function export
eca14ca set Ganga-SVN tag
140bfc9 Added PACKAGES-5.3.3
4e11a92 Added ReleaseNotes-5.3.3
99c824f (ganga@2231)
7c1d51a Changed logic for updating job status to final value
b9fa390 Minor fix for checking TAG files have downloaded properly
82e62f1 Fix missing % problem
2295d8f Fix dcap/gsidcap and QMUl problem
18899ba Automatically updated template for 5.3.3
1b58427 Added PACKAGES-5.3.3
b2bf2ca Added PACKAGES-5.3.3
0a97877 Added ReleaseNotes-5.3.3
0986ca2 set $Name to Ganga-SVN globally
4626594 Updated master/subjob uuid for split jobs
27b80cd uuid fix for subjobs
dd57c19 Updated version, same as Ganga-MSGMS-1-0 tag
717df06 handle gridJobId as list
a5455eb handle outMeta and outMS
198fca4 removed debugging statements
b900218 enhance match method  - if __jdlfile__ is presented, no need to emulate the job preparation  - restricted the method to jobs in one of the states: new, submitted, failed, completed
751456a  - remove the change for deprecating config.LCG.VirtualOrganisation  - remove the change for setting middleware='GLITE' as default
2841a41 introduce j.backend.match() method for job list match
492ad49 (ganga@2184)
897fb1a AthenaMC.py + wrapper.sh: restored handling of trf error codes. The job will return as 'completed', but AthenaMC.postprocess() will scan the logfiles and issue a WARNING if a non-zero transform code is found
65512c0 Added failovers for TAG workflow
67b123a AthenaMCDatasets.py: bug fix to support backend.requirements.excluded_sites. wrapper.sh: disabling trf error code return as it causes more troubles than necessary (CPU and storage waste coming from perfectly healthy output files lost by stupid trf errors)
e257ace Fail over for new lcg-gt syntax
7929be9 Fail over for new lcg-gt syntax
46fbac5 Fix access for StoRM at Milano and clean-up some special SE access cases, use "file" protocol in FILE_STAGER mode for SToRM SE.
5f3fa66 Add additional protection against empty output_location
b873eb2 Fix bug 53707, filter lcg-cr against malformed output
ccb123c stage-in.sh: using dq2-get without -s flag first as it will look for local instances of input dataset first, AthenaMCDatasets.py: minor bug fixes and verbosity changes
bfeab4e Fixed a one-liner for accesing datasets that are not containers but also not _tid sets (i.e. real data...)
32aa3d4 test.py
4f76d35 none
9892763 Added PACKAGES-95.3.2
5f02ae7 Added ReleaseNotes-95.3.2
5e941db final destination
8af7a76 updated SVN tools, reasonably tested with 5.3.1 and 95.3.1 releases
053976d Second step when the jobs are finished
aad0a06 massive changes version2
367d112 *** empty log message ***
c78d75b TestPythonOptionsParser.py
0a36a12 Gauss-Job.py
bae46f0 *** empty log message ***
731c709 "remove config['Gridway']
25c9dcf Forces the use of the GridCommandSet
3d418c6 Updated GangaCron for Gaudi v23r3p1 and tutorial v8r2
8bf8edd Automatically updated template for 5.3.2
87e11fd Added PACKAGES-5.3.2
05173a8 Added ReleaseNotes-5.3.2
718dc09 removed the OSX fix (2 ganga processes would be run at the SAME time with very wierd side-effects)
d56e0e8 added stomputil external package (for MSG monitoring)
5c9b2b6 PythonOptionsParser.py
a1c326f export TUTDIR to GPI
96c80b1 Allowed the safe use of cloud=='ALL'
51970f7 Improved cloud <-> site info retrieval
b1e9426 Short term fix for bug #53613
3ffcc81 DiracScript.py
179967e Removing default value provided to config[DQ2][usertag] as this has the potential to create wrongly named datasets. Adding extra try cases in functions to catch these wrong defaults at job submission time
8b239b2 LHCbDataset.py
56d6a1b Fixed the defaults for GangaRobot to use the repository in ~/gangadir_robot instead of ~/gangadir
60d81d5 * put path into variable * print test success * change verbosity to INFO to avoid Athena crash
05ef4ec GangaKISTI first commit
0f3dbf0 GangaKISTI first commit
479dd0c Added excluded_sites to PandaRequirements. Use like this to avoid matching to sites:    j.backend.requirements.excluded_sites = ['ANALY_A','ANALY_B']
50d6608 Call gridProxy in master_prepare
db86b90 Call gridProxy in master_prepare
f6589b3 Small fix
cde4fc4 Critical patch in AthenaMCDatasets.getDQ2Location(), preventing jobs to write into production space tokens if no specific output site is specified via app.se_name or backend.requirements.sites
81da08c DiracServer.py
60d69e9 Fix: input/outputsandbox not settable
ff9cea0 Fix #47680, restore orginal settings of numfiles and  numsubjobs for every subdataset
8f3b6a0 DiracServer.py
0e858c0 *** empty log message ***
a52ab5e DiracUtils.py
c39da3d __init__.py
856adac Further fixes for the AthenaMC tests
8e147b1  * enable local protocols of dCache: dcap, gsidcap  * add option to enforce the usage of lcg-cp:    - by setting Athena.atlas_environment += ['FILE_STAGER_IOMODE=remote']
e80c348 Added args to GaudiPython
679dad7 Fixed numfiles bug (51299)
93119ad fix for https://savannah.cern.ch/bugs/?53470
70093c8 Removed length restriction and changed doc
17cc47a * change default middleware to "GLITE" * remove VirtualOrganisation from config.LCG and rely on the credential package for voms proxy operations
73dcb47 AthenaMCLCGRTHandler.py, bug fix: app.se_name is restored as output data location handler. Also enabled support of requirements.excludedSites
8e09339 workaround for wierd OSX execv behaviour (from Ole Weidner)
07baa07 Fix for bug in retry of dataset consistency check (52066)
aa00217 Fix test reporting logic and decrease timeout to 3h
6aad21e Fix #44427, protection against listCEsInCloudByDomain
133a5bc Update all tests to 15.1.0, add LSF test and some clean-up
82f12ff Fix dataset naming problem
f278c90 Fix bug #53251: lcg_util problem
15adbfb Fix #53251: add missin get_lcg_util call
8eeef65 Fix #53251, short_filename as string
0cd4ef5 Fix #53251: missing protection for RECEXTYPE, also improve X509
fd23566 bugfix for https://savannah.cern.ch/bugs/?50530  - remove deprecated and redundent import statement from code
cfa94d2 bugfix for https://savannah.cern.ch/bugs/?50048  - RetryCount and ShallowRetryCount to 0 is now included in JDL
d6ceb4c Added PACKAGES-5.3.1
7a1f3a6 Added ReleaseNotes-5.3.1
3465ab0 Enable group dataset names in LCG/Athena, #53155
92fb1ce .
f4fd8ac add resource match-making as an option before doing real job submission to WMS.  - this option can be activated by setting config.LCG.MatchBeforeSubmit = True
2cd989d added uuid into JobInfo object (j.info.uuid)
1936c91 Added Fluka and some other major updates
595a3ca AthenaMCDatasets.py: forcing exact matching of input dataset first in order to cut down DQ2 query time, and therefore job submission time. Bug fix: new containers for output datasets can now be created at master job completion time instead of just trashing the master job completion
8fde4e6 change pandamon url
f830a60 Added PACKAGES-5.3.1
b83da6e Added ReleaseNotes-5.3.1
3f2d975 TestFrancesc.py
03c1078 DiracScript.py
6820d93 AthenaMC.py: adding a warning in the event jobs.force_status(completed) is called while the subjobs are still completing as we cannot prevent this command to freeze forever without breaking the normal job completion. wrapper.sh: addressing Savannah bug #51792.  AthenaMCLCGRTHandler.py: adding default value for backup output location. Value is CERN-PROD_SCRATCHDISK
ee66c48 Initial commit of ArgTransform - a taskification of ArgSplitter
b908ea1 Better protection in master jobs stats collection
c52672d Add --prepare_old option
36c68dd Increase timeout to 4h
60f013a Remove unneeded files
a6f2a0c Remove from release
f766155 DiracServer.py
d5c51c3 TestDaVinci.py
2bc6826 Disable sorting of job builder list view.
766c9d3 Fix symlink problem is staging out logfiles
cd221b5 Automatically updated template for 5.3.0
85959cd Added PACKAGES-5.3.0
ff0130d Added ReleaseNotes-5.3.0
3f375fb fixes a problem when TMPDIR is set.
ff7f8b6 Small fix
56f49d7 Initial import
d37057b Add TNTJobsSplitter in splitting
67b7571 AthenaMC.py: fixing propagation of 'none' from outputdata.outrootfiles to final fileprefixes, allowing users to effectively disable output types in all modes
f29fd35 stage-out: fixed a bug in guid extraction by forcing the scan to match to ID= instead of ID.
3b9a4fd Revisiting dataset registration implementation to allow job.resubmit() to work without troubles. Also put stage-in back after athena setup, in order to avoid downloading DBrelease tarball if there is already a local setup available. Paving the way for support for perf/phys/trigger group production managers
7d7d6b4 Automatically updated template for 5.3.0
5477e46 Added PACKAGES-5.3.0
e5c5d54 Added ReleaseNotes-5.3.0
808ce6a Remove getAgg call
99d3b6f Revert QMUL changes
c0614e0 Fix IFAE
34ff210 Fix
ac99ea9 Fix for QMUL
792dd5f Adds an accidently deleted file and improves the caching proxy slightly so that it refreshes its environment if the grid proxy is renewed.
ee9aeb5 Fix for dq2-get enviroment problem
bf2c26a Fix for IFAE scratchdisk token
a44e2b2 Add file protocol in general and QMUL
ce01729 Automatically updated template for 5.3.0
023e445 Added ReleaseNotes-5.3.0
2322ee2 print first 4KB of error message from shell command (rather than only 255 bytes)
56198ba Added PACKAGES-5.3.0
2f59fec Added ReleaseNotes-5.3.0
ffe4f56 fixed a problem of the unregistered handler workaround (KeyError on logging shutdown) reported by J.Ebke
f785e28 fxed previous fix
3b7854c resort to using os.system('tar xfzv') for python < 2.5 -> tarfile module in v 2.4 is broken for some files
3657b98 fixed tabs
5a4aa92 Fixes to gsidcap, DN username extraction etc.
a71a9a9 Changed kill command to do bulk kill of subjobs
7ab68f6 Removed.
cbec406 Changed to using wrapper-athena-ng.sh
c7c1f3c Re-importing a hopefully executable wrapper...
11c3d1f Executable now?
2ab9029 Removing, to re-enter after changing to executable.
8ce2f45 Trying to make this wrapper executable...
43905ba USERDISK->SCRATCHDISK
7b58bfa add support for two JDL attributes: DataRequirements & DataAccessProtocol
603276a Added direct gsidcap access functionality
c36e850 Adding new series of tests for AthenaMC with the Panda backend
25c769e localxml.ini
149ca6d LHCb.ini
fbd2d54 Removing extra printout from AthenaMCDatasets.fill()
58c7248 Also added transform_script to MCTasks test
a90ed35 Added transform_script fields (now required) to AthenaMC tests
7f397fc Increase timeout to 10800s
5beda99 USERDISK->SCRATCHDISK
b78cd1c Added better support for RecEx* analyses
5263550 Fixed cleanup being applied to task and not to test
e263c38 Panda RT Handler: adding .tgz extension to output logfiles from the runAthena job
dc8abf3 DiracServer.py
cc87830 Automatically updated template for 5.3.0
35cbada Added PACKAGES-5.3.0
a4a85fe Added ReleaseNotes-5.3.0
be2dbab panda-client 0.1.71 trust the information system (jobs won't submit if athena release not installed).
085017a panda-client 0.1.71
403e784 Remove libgfal
aa910c5 Major update to TnT module
a470b46 updates the version numbers
7687ec6 DiracCommands.py
40bf064 more fixes
a476c24 Bookkeeping.py
a8a67e2 Adds a versioning mechanism to the DSTMerger
04be3e1 avoid operations on the log files from the glite commands if the log files do not exist/produced
3d5a194 fixed 'remove jobs' for task - slices do not work yet
2cbd421 *** empty log message ***
fa5aa5c Fixed initial bugs
19ae4b3 Added all Atlas tests to gpip
19de1f9 Committed initial Atlas GPIP test containing three AnaTask tests
a2b6efd TestDiracSplitter.py
38ed6f9 BenderExample.py
4d98e92 Replaced references to USERDISK by SCRATCHDISK, removed default for evgen transform_script, replaced by documented exception thrown if not set by the user
ddae34e Updating evgen test with new space token for output destination
d0f8c7c Fix AthenaMCSplitterJob to use the use_partition_numbers flag
d91d612 Stopped multiple data copying for DQ2_COPY mode (bug 51512)
1228d17 DST/
8326fc6 TestDSTMerger.py
f4f646f Internals/
e374f08 BOOT.py
546e37c BOOT.py
6457e71 Bookkeeping.py
66141ef Dirac/
c3fc19e __init__.py
d6f7004 TestDirac.py
fc32e52 Dirac/
7360eec BackendApplicationMatrix.gpim
2f570bd CVS
d8948d6 testRootVersionsError.gpi
a6ac7b1 Savannah40219.gpi
6ce99cc Really fix getJobs()
6b117ca Fixed getJobs; added try-finally blocks to check() so that the task is always in the correct status
0c031f7 Small change
4a7ec3d added backend/cloud detection and initializeFromDatasets to AnaTasks
9cc52d1 Add getJobs() to Task and Transform and getPartitionJobs() to Transform, returning the job slice corresponding to that task/transform
0b2466a If a non-taskified Application is set in a transform, it is tried to copy all values to a taskified version of the same application.
55fd4b6 Added functionality to use a user-speficied database release, as set in j.application.atlas_dbrelease. Same syntax as on lcg.
7dc9d7a Adds a binding for dst files to the DSTMerger
6dd81fc Automatically updated template for 5.2.4
5164f4b Added PACKAGES-5.2.4
30d4d27 Added ReleaseNotes-5.2.4
619d286 adds a check so that the output is not stored if the output could not be parsed
a3498dd fixed bug #51592: Add self to logger
1c95ab7 pycrypto and paramiko python 2.5 hack in syspath
a530de1 Check for NULL return string of Panda job Id and suggest to shorten dataset name
8dd4c67 Fixed issues with the Remote backend and python 2.5
4ebc996 Changed print to error in TaskList.py
7a2b4d4 Automatically updated template for 5.2.4
567736b Added PACKAGES-5.2.4
634b412 Added ReleaseNotes-5.2.4
53ca7f8 Increase time-outs
8750caa Fix bug #51300
13a884d bug #51369: the new prepare method generates bad requirements file
f43518a bugfix: https://savannah.cern.ch/bugs/?50589
c1d3c6f Merge from Ganga-MSG-branch
ff077ca fixed bug #51051: Failure to load XML repo (https://savannah.cern.ch/bugs/?51051)
86186cd fixed bug #50351: Logging messages aren't displayed while waiting for commands to finish
2c7427a removed obsolete variable
4c3085b Added proper backend treatment of raw input_sandbox and output_sandbox entries; fixed handling of the case where we get a master jid back but all subjobs have empty jid. Will happen e.g. when a site does not have the right release installed.
780f345 set default site to CERN-PROD_SCRATCHDISK
8e124a1 fix Ganga to work with newer versions of ipython (-noautocall option was removed in newer ipython versions)
ecd00bc force to submitted (because jobs can go from running to activated)
dc34a5a added CPC program summary
6d2d116 backend.CE doesn't exist
7be0580 close ~/.ganga.log if redirected (to avoid expired AFS token in $HOME for long running Ganga processes) - merged from the Ganga_MSG_branch
44ee233 set MaxFileSizePandaDQ2JobSplitter to 10GB
7a775e4 bugfix for https://savannah.cern.ch/bugs/?51298
56627a1 Automatically updated template for 5.2.4
6f6b2cf Added PACKAGES-5.2.4
4663f41 Added ReleaseNotes-5.2.4
2f78c19 Add exception for listMetaReplica
ed9d850 Aquire locks for dq2 calls in dq2_list_locations_siteindex
e18ff32 Update DQ2Client 0.1.28
4a0ab7f Update to panda-client 0.1.64
a709a97 Added AthenaMCTaskSplitterJob to the possible splitters
296554d Adding asymptotic limit to proxy lifetime estimate to allow big jobs to complete.
067dd09 Adds the LHCbCachingProxy fix to the default configoration as well as updating the versions of Root used in lhcb.
bc53799 Fix minor bug: if transforms complete while task is paused, they will now be "completed" if the task is run()
165f45b Changed local dq2 site discovery, relies more on environment Works too if the site still has _USERDISK set and is not in TiersOfATLAS
42ff273 Re-fixed a bug for treating ATLAS_PRODUCTION in rel. 14-series
ca3d3b9 uncomments the exception file, so we fall back on lhcb-proxy-info if something goes wrong.
8cac8ba Bookkeeping.py
3d29841 DiracCommands.py
e1563a4 Protection for fileinput.input
175a7af Protection for fileinput.input
82f351d Protection for itotalevents
c92e7d1 Protection for listMetaDataReplica
fb143bf DATASETLOCATION protection
83df51f Removed a remaining reference to USERDISK
5121564 limit submit to 2000 subjobs
c66d6dd Panda server has a per call limit of 2500 subjobs per getFullJobStatus.
9987a96 getFullJobStatus
846580e fill libds if buildjob succeeds.
471af17 fix the broken alder32 checksum calculation - read file data piece-by-piece
bcd6246 disable the usage of dccp protocol in FileStager
9e873f2 Cache proxy identity for subjobs
9521697 processingType=ganga
9c5db25 get sqlite files to local working directory
94d4a63 add a function to get SQLite files to local working directory
3ef3815 * reduced usage of _name * moved adding handlers to the initialization of TaskList, since not all handlers are added before tasks loading * Added .jobid.date to dataset name for AnaTask if datasetname is not set automatically (necessary at least for Panda)
4ff44bd Skip splitter check for dbrelease if AnaTask is used (splitter already used internally)
e5d60ff Increased speed of loop testing for db_release presence
f1527b0 Fixed no AOD files in recon by adding "force AOD" jobOption
414bbb3 Changed AnaTask test to different dataset
f8c7c9b Outputdata now filled even if enableDownloadLogs disabled
5f8bd93 Added gziping of athena log files, settable log file names through environment variables etc. Also fixed the propagation of atlas_production (again). Note: This update looses us live log file peeking of athena jobs. Must look into this later.
16b113e Increase timeouts to 3600s
b3bf9b7 Fix totalevents problem
4e339d9 New placement of Athena.prepare() in help text
6b52477 Add appendTransform for convenience
ceafd3c Make task list wider (jobs also uses more space)
a064296 fixes a typo which meant that the wrong doc string was being set
c3b69c4 Add check for atlas_run_dir
05a5f96 Changed WARNING to INFO if pausing completed transform
5fa0dd1 *** empty log message ***
0f05ef2 Fixed typos
05f9681 Changed dataset to merged
8de0127 Fix so that pause() works with the new task status update
5a843fe * Added state running/pause to Task to show that some transforms are paused * Set Task state dependent on transform states
6bdac8e Added general test for the Tasks functions.
c4d79de finishing touches - last(?) round of corrections
6471b5f Automatically updated template for 5.2.3
8292bb5 Added PACKAGES-5.2.3
31acf0d Added ReleaseNotes-5.2.3
e4d4a5b Fixed some hardcoding for rel15
dc9f218 Remove libgfal.so
bd43515 Attempt to improve input data downloading by clearing up dead copies from failed attempts. Also, reorder list of input sites by forcing local site first.
281bcc5 More robust code for SL5 and minor fixes for lcg-cr
03a70ed initial import
188b486 Fixed a problem with previous upload
64a3a0a Changed Scratch disk to cambridge scratch disk
cd68b7d Added preliminary AnaTask_Panda test and MCTask_LCG test
a1de903 remove task at the end
6cf9773 Protection against task pausing
918b05d Allow AnaTasks splitter on Panda
6f76d2f Changed AthenaMC test to use the dryrun facility
b32ab06 Switched off random failing in wrapper.sh on DRYRUN to make GangaTests repeatable
f13cff0 Added use_partition_numbers to AthenaMCInputDatasets to make Tasks safe and still accommodate loose matching.
1f9d658 Added test for AnaTask, the test took 15min for me.
d829f8b added filter property for GangaObjects
445ffb2 Fix for new lcg-cr timeout syntax
4de9119 DQ2Dataset->DQ2dataset
ca8e9a2 debug directory (https://savannah.cern.ch/bugs/?50305)
54591c9 Fix AnaTransform to work for NG and Panda
f55513c AthenaMCDatasets.py: adding proxy lifetime check in outputdata.fill() to force the job to abort output dataset registration if the proxy is about to be expired. wrapper.sh: now catching the exit status code of stage-in.sh to abort job execution if any single input data file is missing (Savannah bug 50622). stage-out.sh: removing -t timeout flag in lcg-cr as it has been made obsolete in the recent version of lcg-utils
5303ae1 debug directory (https://savannah.cern.ch/bugs/?50305)
4e8d719 Add feature #50558 and remove getJobObject() dependency for self.names, self.exclude_names
04a04ce Add CPU speed Architecure to stats
37da777 adds some extra checks that the tempory directories from the outputsandbox download are correct
c3280c7 Bug fix: extFile -> extOutFile in AthenaMCPandaRTHandler.py, improving logging messages in AthenaMCDatasets.py
db9fa83 startime -> starttime
a24db67 Adds the initial pass at an LHCb caching proxy script which is stand alone.
fa2078b setPath PANDA_SYS
582ec9e applied comments from the reviewers applied corrections from Ulrik and Mike
7f2cbdd Revisited output site allocation mecanism in LCG (RTHandler), added some info level messages to help passing submission time...(MCDatasets and RTHandler)
4797aed Added PACKAGES-5.2.2
09d9396 Added ReleaseNotes-5.2.2
7382f47 revert fix (bad software practices)
41af397 Fixed TAG/TNT usuage
e1ff4fb export PANDA_SYS
d60f4bf added option 'append_to_user_area'
c2c33a9 bug fix in wrapper.sh and stage-in.sh for input file downloading : clashing of environment variables now sorted
a02b175 Added PACKAGES-5.2.1
9c5e2e2 Added ReleaseNotes-5.2.1
4125218 Timeout to 2700s
5136f83 srm url corrected for lfc registration
09d6fce CMTscript.py
ee78036 AthenaMCLCGRTHandler.py: fixing bug preventing job submission to sites where the input data is located
02a5d3a Added PACKAGES-5.2.1
2948e71 Added ReleaseNotes-5.2.1
a8a61da Increase timeout to 2700s
3211574 Bug fix for cavern/minbias file handling
1970b9a Added PACKAGES-5.2.1
39dbf2d Added ReleaseNotes-5.2.1
c174445 concat all ErrorDiags for backend reason
27f0576 Correct Panda retrieve
e760eb4 Correct logic for enableDownloadLogs
dc7c7b3 Add config[enableDownloadLogs] and remove build job output download
a7ce4a8 Change Panda retrieve logic
ad9061b use spacetoken in srm url
d4a33dc Fixing https://savannah.cern.ch/bugs/?46482 in wrapper.sh by moving the call to stage-in.sh before setup-release.sh
b3ea6cb Change to user09
97f0203 export prepare_old method
1b34ae0 DB Release treatment: now parsing properly DBRelease=DBRelease-x.y.z.tar.gz in extraArgs
acbd3e3 Splitters.py
08dd900 added modules for pdf processing
3574c84 added imense scripts
3bab00a added imense scripts
579d066 use panda-client 0.1.53
2f9e85a fix bugs in falling back to lcg-cp protocol when local protocol fails
b7c99c9 Fix for https://savannah.cern.ch/bugs/index.php?50086
b3ba2c4 Added GangaPanda
ebde05d use additional packages such as GangaPanda for generation of config templates etc...
59b92e6 Bug fix in stage-in.sh: prevent dq2-get from downloading the whole dataset, overriding input list of files when the input dataset has more than one source. Plus some debug lines in AthenaMCDatasets.py
4d38e4f fix escape order to be symmetric with unescape.
cd29bc4 fix a problem of  in the strings... use repr() rather than custom quoting...
9501d56 Fixed fix...
0543727 Adding protection to master job completion in AthenaMC.postprocess(), forcing the thread to wait for subjobs in completing state to finish before running outputdata.fill()
a3b90c7 Added PACKAGES-5.2.0
7509cb5 Added ReleaseNotes-5.2.0
7f27f23 Added log file downloading and outputdata filling (DQ2Dataset) to Panda backend Not tested AthenaMCDataset yet!
5fac676 Fix #49823, proper setup for local DB
faad5c9 Expected number of files fixed in error message
a1f2658 Fixed bug #46825
97f1189 bugfix: https://savannah.cern.ch/bugs/?49806
42b54a9 Changed misleading wording of WARNING
9de5b62 Removed spurrious warnings
83641c3 Change schema version of Athena class
5d48a7e unique outds for tests
8cdc358 TestFrancesc/TestFrancesc.py
535e200 Added close statements for tempfiles
de2fbbd fall back to lcgcp if trials with local protocol were all failed
47f6b4e resolve the transfer protocol based on the SE name associated with the DQ2 site id instead of using close SE
966a21c Added PACKAGES-5.2.0
694c103 Added ReleaseNotes-5.2.0
e099cb3 Small fix
c7636b2 simplified handling of logging filters
8b15655 simplified logging filters
38fc55f Removed trailing newline from DQ2OutputDataset.output
3ea8d3d Added the option of skipping PoolFileCatalog generation for local backends
36faf06 Fix if numfiles is empty string
1d16a50 Removed black magic since the core bug for multiple inheritance is now fixed
617c5ea Fix bug #49365
18b19ca Fixed ARA again Fixed libds support changes to ARA test case new libds testcase
3f218c1 Change prepare calls
4dd8491 * Fix for bug #49337: PYTHONPATH cmt setup problem in 15.0.0   should be solved now. * New: prepare() has been rewritten   - The functionality should be the same   - New CMT detection and tarfile creation for Panda   - New Athena.athena_compile variable   - prepare(athena_compile=True/False) is obsolete * renamed original method prepare() to prepare_old()
0fa0666 don't overwrite outputdata for --panda
3784c9e CMTscript.py
13db663 loggers created in interactive session are now mapped to Ganga.GPI loggers created in other packages (not starting with "Ganga.") are now correctly using the log file
d9c3911 Added ARA test case
6415acb Do not uploadSources if libds is set
16f90b6 Added user-settable libds support, and fixed submission without local athena setup
82874a2 Securing dq2.freezeDataset() statement in AthenaMCDatasets.py fill() function
a3e0302 fix #29745: use __mro__ rather than first-generation of base classes
b59afab Fixed dbrelease not available (necessary for tasks.overview())
9611936 extra file sanitation
b63661b Fixed copy() function of task; added "print" on exceptions
cd72461 Exclude ganga lfc lib
8ba8b5e #49529: provide name for GangaThread
7aca461 removed __init__ function , moved initialization of private members to master_configure()
b1081ae fixed GPIExport of functions added NG RT Handler (not functional yet) added pi example
597eaae Fixed multiple resubmission
95654d0 Adds a section on the Dirac.reset command
360a7e4 Added __init__ function to avoid global scope of lists
17fcfad Added protection and error message when dq2client_version is not set nor read from configuration
ad8d6a3 Restoring dq2client_version in Atlas.ini, as it is needed by AthenaMC
9eb123c bug fix for evgen job option file handling and missing transforms flags in panda
b288cf6 TestGPUserDLLs.gpi
9b44d3b ResubmitGaudiDirac.gpi
9ec760e TestDaVinci.py
269093b PythonOptionsParser.py
01d661e change stats. schema 2.0
b524855 obsolete test
7bd7207 TestSplitters.py
e438fce *** empty log message ***
d300c02 Error codes in the Panda object
dbae87b percentcpu is an int
e02747d collect stats for Panda backend
cb6aa56 Use PandaRequirements
d06cd98 - Move requirements to PandaRequirements - Store the Panda JobSpec in a backend.jobSpec dictionary - Added backend.get_stats()
d34ca30 Fixed bug to allow ArgSplitter to work (added check of wether application was really athena in one crucial location
07e3ed9 Changed because of changed behaviour of AthenaMCDatasets
a318007 fixed integer conversion
b0508ad fix the // issue for queriesto the bookkeeping
63cef9f Fixed not printing logger from Thread (again)
a3419bd Added PACKAGES-5.2.0
1ba4219 Added ReleaseNotes-5.2.0
a0ead0f Rename prepare->prepare_old and panda_prepare->prepare
56aed9b Redistribution of code between AthenaMC.py and the RTHandlers. The application handles the preparation of input/output data through AthenaMCDatasets functions, while the RTHandlers just deal with backend specific issues and formatting, reading all the data they need from AthenaMC members.
7016864 Ulrik: Several updates to the Batch backends. The bugs and features requests numbered 35994, 36651, 45219, 47814 and 49078 have been addressed. The most important change is that batch jobs killed by the batch system itself (like if they overrun their CPU limit) are now correctly registered in Ganga as failed.
0cd5117 Ulrik: Several updates to the Batch backends. The bugs and features requests numbered 35994, 36651, 45219, 47814 and 49078 have been addressed. The most important change is that batch jobs killed by the batch system itself (like if they overrun their CPU limit) are now correctly registered in Ganga as failed.
98c493e GaudiUtils.py
acd736e add processingType
136e5b6 Revmoved Panoramix
4a60041 Revmoved Panoramix
eefd7b3 Fixed test
8afb6ac Savannah 48760: Adds a reset method to the Dirac backend which clears out the jobs output and resets its status to the monitorable state 'submitted'.
d55710d Savannah 49245: Adds a cache to the gridProxy validity check in Dirac.
d90e557 Added PACKAGES-5.1.10
afbfe77 Added ReleaseNotes-5.1.10
4d161a0 handle multi datasets in subjob
0f1e5cc fix
8409586 typo
b457091 prepend username to datasetname if not set
176c0e2 uses the version of setenvProject on the path
1867437 don't use user_area_path
cadbb67 removed dependency on the LHCBSCRIPTS environment variable
2955546 Patching logic hole in output storage token allocation which was causing output data to be written to a forbidden production space token when the input dataset was located in a production space token
e3de2ac runPandaBrokerage works for no inputdata
ca0b8a2 Removed setting the inputGenerator file from tasks, this is set in AthenaMC automatically
ba979c5 Added safeguard so that manually resubmitting split jobs that failed on submission works
4b7c116 handle empty inputsandbox
02891ff remove pandatools from pythonpath
33ac520 use pandatools
24a1ba3 Introduce panda_prepare, dependent now on panda-client external
2cb4aeb massive Panda changes:   Many Panda options moved to Athena.py.   Athena RT handler now uses prepare from Athena.py   Added Executable RT handler. Not working for me yet.   Added test cases for Athena and Executable
3529059 Small fix
06270f7 Fix for #48964, athena v15 setup
aa790c4 Adding new Run Time Handler for the Panda backend. Simplified AthenaMC interface: production_name is now inactive, and process_name can be used to fill in for run_number as well. Finally, an attempt to extract process_name for input job options and/or dataset has been implemented, reducing the total number of mandatory fields. Native backend member to select a given storage can be used to replace se_name as well. Arguments in extraIncArgs can be put in extraArgs as well, using the dedicated key word . Added length check on output dataset names.
0e339fd Automatically updated template for 5.1.10
fcd5e73 Added PACKAGES-5.1.10
65d9ccf Added ReleaseNotes-5.1.10
49bd1cf warn if BASH_ENV script is missing on worker node
2b86804 Fixes the getOutputData method so is doen't produce an error incorrectly. Test now passes.
af73022 Further fixed copying of tasks
e6b9744 Fixed not being able to modify copied transforms
51ad733 Added input/outputsandbox to Transform.py
58febe8 Fixed logging from thread not being visible
c80c5ca fixed wrong bugfix (#47065)
8a7ed4c AthenaMCLCGRTHandler.py: bugfix for min bias and cavern files counting
7e11381 fixed initializeFromGenerator
97054b2 First upload of Imense/ILexR Spidering plugin
55a2702 Removed Euler
742d5a6 Return the list of LFNs prefixed with LFN (uppercase) instead of lfn (lowercase). Seems as if the dirac tool expect this :()
8330427 Adding new user function, outputdata.recover(). This function runs fill() over all subjobs, then the master job itself and allow to register output files of failed subjobs or subjobs stuck in the completing state. It also allows to freeze the output datasets if the master job is declared as failed
a90fe7f Replace USERDISK with SCRATCHDISK token
4a58aad synchronize the data iterm insertion
5ebd1b1 improve logging information handling
5ee09a6 fix race condition issue: https://savannah.cern.ch/bugs/?48435
e8654a0 fix bugs in adopting GangaThread module
a1632a0 set default monitoring service to the analysis Dashboard
7641f0e Fix for Bug #48578
aefff80 Fixed outputdata copy bug.
9357e38 Automatically updated template for 5.1.9
ba0aa8b Added PACKAGES-5.1.9
3045ffc Added ReleaseNotes-5.1.9
f4af08f check value returned by voname()
7fca2f4 Fixed bug resolving LFN's for split jobs.
48dcf51 Automatically updated template for 5.1.9
d94a6e9 Added PACKAGES-5.1.9
9c25116 Added ReleaseNotes-5.1.9
2e3512a import os
91d00c6 small fix
2b7f6a5 Fixed startup bug due to missing GPI.config
070a492 in getTidDatasetnames use listDatasetsInContainer
41bc3fe add import statement
30fbb27 Patch for CNAF
e0ef8d9 added config parameter: forced_shutdown_first_prompt_time = 5 seconds
bbc380b Initial import
70ce7a7 * Made GangaThreads quit faster by adding checkpoints * prevented multiple ganga sessions from modifying the same tasks file * Used fsync to make Tasks repository safer * Changed TaskList.py and BOOT.py to use time-based locking * Created new Save-Thread "GangaTasksRepository" to make time-based locking work
5c52bcc update the LCG test case
925126b Thread->GangaThread
04abddc Thread->GangaThread. Need to check 'shouldStop' in while loop.
3b6fafa Automatically updated template for 5.1.9
c2fbb4f removed StuckThread (debugging)
c9d8caf Automatically updated template for 5.1.9
3d2067c Added PACKAGES-5.1.9
b8b88ac Added ReleaseNotes-5.1.9
2a076c5 Small fix
696f687 Remove dq2 calls
0d0ec94 check VO name and provide identity info
3f0e17c *** empty log message ***
c467c6a Ugh!
7a5478a I can't get rid of these
4c1ddc6 This thing won't die!
3381d34 Added first summary line to the docstring of browseBK
33a50af fixed a bug which caused datasets created with browseBK() to be interpreted as PFNs not LFNs. File returned from the bookkeeping interface are now explicitly prefixed with 'lfn:/'
d2e4ca1 Update to 300 sec update
cb557b5 Update to DQ2Clients 0.1.27
0e209ba hacked get_cmd_prefix_hack
168036a removed CE specification
15d7e9e adds __copy__ and __deepcopy__ to GangaList at the object level (only)
c232fd8 make the backup of the essential env. variables (PATH, LD_LIBRARY_PATH, PYTHONPATH) globally available
10e2b5a enhancement for file stager wrapper  - slim the wrapper log  - introduce the detection and the usage of a local protocol
c78516c Adds an args option to the RootMerger
7c71843 remove the codes for excluding lib64/python in make_filestager_option routine  - it solves the problem on some sites (e.g. TRIUMF) in loading the lfc python library
102d612 Added getenv method
b3bf2b6 csh alias fix
55fb1aa Fixing critical bug implemented in revision 1.28 of AthenaMCLCGRTHandler.py that caused all jobs to fail with lcg error code=2. Also improved support of DBRelease, forcing the job to look for preinstalled, local version of the requested DBRelease in the local software area. Local installation is used if available, otherwise the tarball is downloaded from local SE.
7c3e92f TestGaudiLSFRunTimeHandler.py
523aaa2 TestGaudiPythonLSFRunTimeHandler.py
b395ad7 New test for scaling issue
2bff8b8 Sped it up a bit
31bb0f4 Updated
6d362d6 Fixed the scaling issue.
a72a340 merging bug fixes from branch Ganga-LCG-old-MTRunner to trunk
6decd77 adopting GangaThread in Ganga.Core
24b85c2 remove codes as they are integrated into Ganga.Core
2df0178 remove codes as they are migrated into Ganga.Core
8fcc976 fix the broken AMA_FLAG_LIST while setting multiple flag/value pairs
a29ad8b changed semantics of BOOT.py -> don't import the names into GPI namespace automatically...
023b4d4 added GangaThread base class for uniform handling of exit from backgorund threads (the same mechanism is used now for LCG download threads and monitoring worker threads)
0bd7f4b Adding support for AtlasTier0 production caches (to be tested), sort input sites by decreasing dataset shares and allow aggregation of output data files to the site specified in app.se_name by subscribing the frozen datasets at the completion of the master job
a77bb2e bug #40460: Little documentation for setting resource requirements for batch jobs
9af1fb0 bug #40333: Enhance peek functionality to look inside tar files
0e144c2 reverted the previous commit (mistake)
0e42edc fixed: bug #47065: Syntax error in Ganga/Runtime/Repository_runtime.py
f0680dd Force update of outputdata.output_dataset, disable partition number matching if inputdata.redefine_partitions is set to a list of lfns, added support for ATLASSCRATCHDISK space token
907180e deal with monitoring of empty queues
18169a6 modified behaviour of kill() method
d70cf36 Adds the GangaNG tests to the release
5005d2b Adds a testing config for the localxml repository
94a61d4 Automatically updated template for 5.1.8
4662c78 Added PACKAGES-5.1.8
55ecd20 Added ReleaseNotes-5.1.8
43d612f Fixed conflict w/ DiracSplitter
b9d4c8b Updated
79db29f Fixed logging problem with missing job IDs. Again.
1f60448 This test is outdated.
91de3d6 Updated
83e7aab Creating this subdir
9897929 moving to subdirs
25b7063 Adding
f579a2c first version
aaef889 Major arch. change. Gaudi/GaudiPython now inherit from common parent class and share run-time handlers.
ad0dca7 Minor changes to handle new Gaudi/GaudiPython arch.
09f5a9a No longer need this
749ad0f Separated the classes and moved in some utils from Lib/Gaudi
61a9bc4 print statements removed
bf3f133 athena_dbrelease option
4b177d9 removes test for outputdata, as jobs can also get outputdata from options files
b0569b4 adds benchmark test
e0bf278 https://savannah.cern.ch/bugs/index.php?47473 dbRelease option is now deprecated in Panda backend.
6c63dcd https://savannah.cern.ch/bugs/?46836
2c50f84 First upload of basic NA48 module to run Compact and Nasim
a938529 Add size parameter to get_contents
cb7fcce Increase timeouts
829dca4 Change dq2clients config
2df2d74 Change retcode
941ff21 Testing; no changes
5058883 Resume, resubmit, ngclean
7d63fb8 retcode=0 in get_pybin
4f06521 patching stage-in.sh to simplify treatment of input data downloaded with dq2
797d29b Fix bug #47262, athena wrapper should stop now if there is a compilation problem
b5ce962 Fix feature request #47506
b7e7267 Added support for storing log files in DQ2
404eaac Fix typo in AthenaMCLCGRTHandler.py
e19ed9a Fix bug #47643
66e1485 implemented checks on output dataset name length, removed -t -l transform flags in template mode
b03e7d8 Fix if dataset only matches a container
b575605 Fix for bug #47364.
afad228 bug fix for: https://savannah.cern.ch/bugs/?46915
037f930 Fix #47520: TypeError when no file size information available in DDM
08d779e Change uuid generation
d10bd0f Small updates
b7b95f4 Added DQ2 tracer reporting
cad521c Re-Added black magic since the core patch for inheritance from grandparents is not yet in (bug #29745)
a74e29c Updates the GangaList to deal with nested lists more carefully
364f691 Added small section to documentation on use of ~/.ganga.py file for execution of commands at startup.
4387aba AthenaMCLCGRTHandler.py: solved Savannah bug # 47417, added DQ2 tag in job requirements list. AthenaMCDatasets.py: reintroduced output containers and job's subdatasets, since AthenaMCInputDatasets.prep_data can handle them now. Jobs' subdatasets are frozen and associated to their container upon master job's completion. Also fixed potential abuse of dataset naming convention
8becc11 Add fail over for several output files produced by athena and remove DISK appending for Tier1 site IDs
274a5e2 Adds localxml.ini back into the repository types to be used
f7b6e17 Set empty string for job ID for jobs that failed to submit
0ae6af0 Sort DQ2Dataset by lfn/names
0ffb209 Automatically updated template for 5.1.8
25f6025 Added PACKAGES-5.1.8
2851be6 Added ReleaseNotes-5.1.8
4c5c57d Input datasets: now performs a loose match from input dataset name and collect all matching datasets in order to catch all registered input datasets from a container, including files registered in opened (unfrozen) tid subdatasets. Strict input dataset matching is performed on DBRelease datasets only. Automatic input/output dataset naming: removed substrings datafiles and logfiles from paths and dataset names.
0311ae2 introduce and adopt the basic class for Ganga multi-thread handler
f9a9075 remove old MTRunner
dfd203e updated outdated testcases (while trying to make them pass with XML repository)
53176b7 extra comments, remove tabs
e95dbd1 to ease debugging repository problems, make a dump into a tarfile if there are unhandled exception when scanning the repository
095b806 when removing jobs which are in the "incomplete" or "unknown" status, do not trigger callbacks on application and backend -> they may be missing!
38f8cb2 set parent correctly for GangaList items (in __setstate__)
b1e2e2f fixed evaluation of File objects (by value), the implementation uses Utility.Config.config_scope
98c0517 metadata selection not implemented warning adjust the name of masterJob argument to match AMGA implementation
cd85eed Automatically updated template for 5.1.7
7b8ac75 Added ReleaseNotes-5.1.7
8f1655b removed obsoleted test case and fixed an outdated one
4fc975e introduce log_level attribute in AMAAthena, default level 'INFO'
5e08bc2 fix broken requirement statements on cputime specification
656d400 Automatically updated template for 5.1.7
402b3b5 Added PACKAGES-5.1.7
510e293 Added ReleaseNotes-5.1.7
b048216 Savannah bug #46592: restablishing automatic splitting of input datasets with missing files
e2f04d5 New numfiles3 stats number
f679b6a Correct md5sum length
d9d9552 Corrections for Network stats
70999f8 use daemonic worker threads
6d991dd do not copy grid proxy when running with Local backend
2657cb1 MD5 Length set to 32 instead of 36
6d66cdb bug #47015: adding debugging flag to dq2-get in stage-in.sh
9bd2fed Automatically updated template for 5.1.7
6eeef99 Added PACKAGES-5.1.7
15329bf Added ReleaseNotes-5.1.7
dac19a2 Fix for network
3a572da Fix for network
ee23cec Fix for network
7d2819b Fix for no replica datasets
c0cb2f0 Change to error message in case in subjobs are produced
8430225 Procetion against foul dataset containers
9f3a085 add a temporary workaround for an issue in FileStager-00-00-28: copy command arguments wrongly passed to the copy command  - workaround: always using the default arguments in the copy command wrapper
852985e collect all root files in summary/ directory in output sandbox
2ea5ec5 return exit code from athena_compile routine
10010db Update network algorithm
98d0ea4 fix the ama_athena wrappers to generate proper extra job option files for loading AMAAthena Driver
2660634 add 'driver_flags' attribute in AMAAthena for setting AMA driver flags
4770c62 Added support for ssh keys
fbd20f4 Added capability to move files to local disk from symlinks, also added banned reprod queues to xrsl
0a46108 Added capability to move files to local disk from symlinks
3e843fb Fix container submission problem
1ab6ef7 fixed xml catalog call to not happen when backend=dirac
1f77f03 updates tests - at lease OutputData passes
b526ec7 Add network traffic information to Athena.stats
31f7e0a fixes typo in name
865c9d4 Add --client-id=ganga tp dq2-get calls
d458b01 Add proxy.identity(safe=True)
5f61865 add proper DATASETLOCATION for  FILE_STAGER mode
731308f improved warning message if no subjobs have been created
f24ac5a bug #46700: Propages the job name to Dirac.
afa9729 Added NGStatTools.py
005c96f Fixed guid handling
94f24b5 Fixed guid handling
ad2c031 Added NGStatTools
8846838 Fixed issue with returning an LHCbDataset object instead of an LHCbDataset Proxy object
f287b42 added pdflatex flag for arxiv submission
77ae273 Fixed handling of master package for Gaudi jobs
84ddded *** empty log message ***
ca68397 schema changes
4b85392 list files in sandbox cache in color representing different status of the files  - expose list_cached_files() to user interface
5a0fef8 change basedir of DQ2SandboxCache from users to userxx where xx represents the last two digits of year
a26c5c3 Start of implementation for Savannah #46952 - checks Dirac services using dirac.ping....
9b895ef use 0.1.25
b55f9b3 source cleanup (unneeded comments removed)
4e51af3 include the sites advertising 0 MaxCPUTime and MaxCPUWallTime
b2048f2 Added NGInputData
489b718 Small updates
df8d276 Initial import
3ec3909 Initial import
e23e159 fix the issue that reading sample files from a local directory in StagerDataset was broken in current releases
a04af2b Added timeout functionality to ngsub
1169ed4 Initial import of NGInputData
9f304fb Automatically updated template for 5.1.7
f668b3b Added PACKAGES-5.1.7
725129d Added ReleaseNotes-5.1.7
f257f65 thread->threading
df802c0 CHanged from thread to threading to enable Ganga messages from thread
b3d4f7c Replaced many Exceptions with ApplicationConfigurationErrors Suggested at the Developer Days
8ff151b Removed NG_DQ2.py
42f2b62 Removed AthenaNGRTHandler_DQ2.py
d7e3975 Removed reference to outputdata.use_dataset
22a9d33 protection added to cover cases where extractFileNumber() returns an empty string
cca84ab added latex compatibility (so one can use pdflatex or latex) + eps figures for that purpose
cfb609d Using permanent dataset names for simul and recon jobs
9ce77d4 Added a bookkeeping_command
30aba2e removed print statements
823eafb add the import for the Bookkeeping
d515d2c Added Utilityfunction to start the Bookkeeping
af1fe89 New Bookkeeping GUI main code
b081684 readInputData:         Better documentation including examples
c468184 promoted use of new script adler32.py in stage-out script
6b101b3 Fixed crash if task is paused due to too many failing jobs
8c474af *** empty log message ***
1e7ef49 indentation fix in AthenaMCLCGRTHandler.py and added adler32.py
66249c7 Migration to use of dq2-get for input data and strict job to data policy. Added migration classes in AthenaMCDatasets.py to allow display of old jobs. Bug fix for collection of adler32 checksum numbers of output files and improved debug output from stage-out script.
bb7def5 corrections from Glenn
c43797e Updating input dataset names used in simul and recon jobs, clean up unused lines
9b24164 Forgot to add random_seeds to MCTransforms
87a2f5a Fixed typeinfo of schema
8f4700e in DQ2OuputDataset, actual_output is named "output"
bcef0df Fixed bug not setting site for split AnaTask jobs
31fea4b fixed lib tests
64fec94 GaudiUtils.py
f4de1b5 GaudiPython.py
7626fe8 *** empty log message ***
7019053 Added setFailed() to Transform GPI
aea5b28 Avoid crashes if empty datasets are in containers
fef9b99 Adds depth to the MockDataset
a30351d Automatically updated template for 5.1.6
8858504 Added PACKAGES-5.1.6
537f3c0 GridSandboxCache enhancement:  - put cached file information in job repository (instead of __iocache__ file)  - add and expose method: list_cached_files()
4dfa64a Improve DQ2Clients setup
4d53000 fixed frontmatter and HC affiliation (NIKHEF)
358f276 Automatically updated template for 5.1.6
d24592a Added PACKAGES-5.1.6
fcf6249 Added ReleaseNotes-5.1.6
0d7a157 Change to adler32
60285c8 Remove DQ2OutputDataset.use_datasetname
0c68f81 Adds depth propagation to the splitters
da00ccb add AllowZippedISB=false to glite JDL  - workaround for WMS bug: https://savannah.cern.ch/bugs/index.php?32345
1123d0b Changed to using gserver for CVS Changed some platform strings from slc3_gcc323 to slc4_ia32_gcc34
e09a49d Added PACKAGES-5.1.6
207a0e0 Added ReleaseNotes-5.1.6
f47039e added depth to catalog
58b2275 enhancement for bug: https://savannah.cern.ch/bugs/?43502
6297f7a Roll back creation of output containers and job subdatasets, added protection against cross-cloud input data replication, support for adler32 and use of lcg tools built-in timeout mecanism on top of existing timeout mecanism
840ae47 updated plugin diagram
2544157 promoted "interfacing" to a separate secition added glue paragraph for ARC updated plugin diagram
63cdac8 added section on GUI minor updates from Karl
b46db2b *** empty log message ***
a3e0a5e LHCb statistics updates DIANE updates contributor's list update
52bc33e bug fix: https://savannah.cern.ch/bugs/?45071
797f900 Changes the default value of depth to 1
f1a2a54 Change dq2info.tar.gz to DQ2Clients setup
25cb4a0 Correct python setup for dbrelease
fc195a9 Correction for dbrelease dataset handling
95f1dc1 Changed CVS repository from kserver to gserver.
449e52b Savannah 46559 - Adds support for Ancestors to the LHCbDataset and the various Dirac handlers. Also fixes a few problems where inputdata was broken for Root and Executable jobs.
3c65271 Increase time-out
9385788 Savanna 46526 - Adds support for merging textfiles which are gziped.
de1f614 	minor bugfix for setParameter
4f5b229 Added PACKAGES-5.1.5
99afe89 Added ReleaseNotes-5.1.5
c782f78 Added PACKAGES-5.1.5
0ce0fe3 Added ReleaseNotes-5.1.5
902b6a5 fixed: bug #43249: jobs.remove(10) works, removes all jobs
381bc98 Added the readInputData method to the application. This is an exported method visible at the GPI level.
a33b6dd added testecase for bug #40399: Problems with GenericSplitter
e1548f3 fixed: bug #44934: Didn't create .gangarc on first usage
4b6d5ac Small fix for odd dq2-get issue
4c101af bugfix: bug #45679: j.application.transition_update("removed") is not called on j.remove()
e0aa35c Correct middleware
77027e1 Revert FileStager inclusion
95b23f5 Improve TRF support
453aa7f Improve TRF support - enable ATLASLocalDataset
0a30a5f Add FileStager files
dd6479d revert to Ganga-Credentials-20-00
e079ee2 safe option for GridProxy.identity()
e27e076 *** empty log message ***
6f40732 corrected front matter (split title over 2 pages) implemeneted a large number of Karl's suggestions for the text content corrected et. al. in references etc.
e6f208f modified BOOT.py
695d831 Bugfix for overwriting extraArgs!
60b4e88 Merge from 1.7.2.1
a1e6858 updated monitoring figure
1855b60 new monitoring figure
39ee44c Automatically updated template for 5.1.5
548e990 Added PACKAGES-5.1.5
86d9a04 Added ReleaseNotes-5.1.5
e8a7e3a added AMAAthena following a recommendation of Hurng-Chun
d9ddcbd Small fix
6cfe03c Remove apostrophe from DN
fb5e05e Fix input.txt and dbsetup
87021d2 extFile option for additional files to ship to worker node
cd38e8e fix for ares
1694c1e Small fix
9affedf Add #46268 - create input.txt
f8bf160 Removed unnecessary print statement in Download class
4eed43f fix for https://savannah.cern.ch/bugs/?46304
0f860b9 #4620, Add dbrelease support
9928bbf Fix to DQ2OutputDataset.retrieve() method
e0e7d7a Fix to make sure the merger works on DQ2OutputDataset
0804ef7 Small fix to make sure the DATASETLOCATION env variable is set properly when split
6c6d2db Update to DQ2Clients 0.1.24
10c496a use panda-client 0.1.6 use AthenaUtils to extract run config and detect athena env
b32813e Added ability for user to see gaudirun.py output if an error occurs
1781478 introduce the copy command wrapper for FileStager: fs-copy.py
2c2a074 dump checksum information from LFC for all sample files in pickle format: lfc_checksum.pickle
2000a11 Fixed a typo
823e777 added coverage for gpi
c117310 Support for TRFs in the Athena application
95a85e4 added triggerConfig to simulation jobs
3baa1c2 environment setup on execution node
2268222 Added replacement for genCatalog
dd8214e Removed a spurious print statement and added a few utils
8bd850e modification for handling stdout/err in different ways  - add config.LCG.JobLogHandler, default value is 'WMS', meaning that stdout/err    will be shipped back to user via WMS's output sandbox mechanism  - set config.LCG.JobLogHandler to other values will remove stdout/err from WMS's output sandbox    and the application can pick it up accordingly to handle stdout/err in different ways    (e.g. store it in a DQ2 dataset)
9b484db Adding Bender tests
fc6ec8e Adding Bender support
badac31 version2 of images
a61f956 version2
126d11c added changes from Bjorn, new diagrams and a few minor corrections
40a3a86 Adds a method for getting the outputdata lfns and updates one of the tests for it
31004ad propagating changes from AthenaMC classes
e8eca22 moved this test to the GPIP file
b3370b0 Savannah 46184 - Allows the downloading of the outputsandbox on demand.
7bbd113 handle TAGDATASETNAME list
950cab2 Updates the status mapping so that Completed is treated as running. This is because the data transfers are not finished until Dirac is in the Done state.
d33a397 AthenaMCLCGRTHandler.py: bugfix for template mode, removed unwanted default parameters breaking non-evgen transforms. AthenaMCDatasets.py: full support for dataset containers, in both reading and writing modes
d4308bd Automatically updated template for 5.1.4
4dc114e Added PACKAGES-5.1.4
6bef3d0 Added ReleaseNotes-5.1.4
faa86c5 typo
95a6084 Fixed adding further files to AthenaMC, e.g cavern/pileup samples Fix by Frederic Brochu
88ceb75 Automatically updated template for 5.1.4
695c817 Added PACKAGES-5.1.4
72f0636 Added ReleaseNotes-5.1.4
0b94d46 Upgraded to lfc binary 1.7.0 to be compatible with ARC 0.6.5
9127cbd small typos fixed
47789d6 Automatically updated template for 5.1.4
22a3a6d Added PACKAGES-5.1.4
2e6ccf4 Added ReleaseNotes-5.1.4
4a61545 Added support for systems where /bin/sh is not bash
1c74d87 Add ServiceMgr for DATA
406400a using athena utility scripts
376bd9e  - add switch to allow external lcg-cp wrapper script  - enforce downloaded files to be stored in the current working directory
57de80b fixed a bug in one of the tests
7df35b0 fixed a bug one of the tests
979d297 Added dryrun functionality to AthenaMC
84f098c Fix for wrong subjob number for incomplete datasets
d6d0f64 fix for glite perusable function
a89bc3b integrated fixes from Will, Frederic, Johannes and Bjorn
a2ba755 Made Athena also accept an AnaTaskSplitterJob
0d02c17 make redefined_partitions safe, so that they can be used before get_dataset
ac19242 Fixes TestLFNDownload.gpi so that arbitrary LFNs can also be downloaded. Test now passes.
93b1eed small fix for dry-run mode
51acec3 killing partially submitted bulk jobs on WMS immediately if the whole job submission is not done properly
83ce1c3 Upped to ARC 0.6.5
6735c0d Removed Debug
806c91c small fix in fqn()
145257a added check
b643f97 Fix for firstevent in AthenaMC
de20d59 Some NG statistics patches
900629a Big Tasks update: * Transforms and Tasks now use _getParent() * tasks handling is more similar to jobs now * Task initializeFromX work now, and input datasets are displayed in overview() * AnaTask initial working commit * moving setup to check, which can be manually called * several smaller fixes * help update
4e1181a * partition_number=None made evgen fail, fixed * getPartitionList now works without job object, then returns an open range starting at the first partition
3abb50b Add --splitfilesize option
638fb5e fix glite-wms-job-submit command-line option composition for multi-thread job submission. bug #45897
ca1279d config['AllowedSitesNGDQ2JobSplitter']
60fe8a0 Adds an explicit test that SetupProject Dirac returned as expected.
627b7a6 Change algorithm for filesize retrieval and return
6aab036 Add 'filesize' attribute, which allow spitting on dataset size Add config['MaxFileSizePandaDQ2JobSplitter']=5000 Panda splitting ist done on dataset size now
7bf93da Added support for giis list, fixed bug in assigning subjob IDs when some jobs failed to submit
8e7e364 Added giis list file
34c9449 Introduce NG splitting based on filesize - maximum is MaxFileSizeNGDQ2JobSplitter
612f8f2 Introduce get_contents(filesize=True) - returns the dataset filesize
9e7258c added mailspam
f730f7b sites takes priority over cloud
4d796a2 Restructure athena_setup
347a865 Patches for MuonCalibStream dataaccess fror FILE_STAGER and DQ2_COPY mode
e57fcb8 Adds an info log on the dirac version
15f9819 Put patch at correct place
893e22b Improve athena package setup and tarball packing
8b0db0f Fix typo TNTJobSplitter
f532258 Bugfix for http://savannah.cern.ch/bugs/?45593
af0260a Added PACKAGES-5.1.3
86ccddd Added ReleaseNotes-5.1.3
efcaca6 fixed references
748c243 Kuba's updates
d02057b use a net alias for the monalisa server
0db68c7 applied comments
92e3982 Added PACKAGES-5.1.3
55c38ec Added ReleaseNotes-5.1.3
55be01c stdout.txt -> stdout
853310e Fixed bad lfc registration on NG backend
d70e3e4 Fixed bad lfc registration on NG backend
f40e55f applied changes
6d59cb0 fixed minor bug - forgot to import something
3455d73 Automatically updated template for 5.1.3
70feecd Added PACKAGES-5.1.3
ed4b4ca Added ReleaseNotes-5.1.3
7d473ce override theApp.EvtMax in the FileStager specific input.py
0b9b66d dbRelease option
71ec99a Small fix
6c275f2 typelist fix for GUI
53df66a adds missing import
800bb03 fix application.mode unset
145ce2e quick fix for startup problem
4aa37db Changes for improved data handling and more splitting capabilities, goes along with new Tasks version from Johannes Ebke
22887a2 Changes for new Tasks package
d62e61b Remove old Tasks package, Initial import of new version from Johannes Ebke
c0af9cf internal code factorization
c8649e8 include the monitoring service
f10b9ac fix sample file name of the FileStager
7fb262e run time exception fix. failed to find module
6c3cff9 PythonOptionsParser.py
3807775 full DaVinci tests
16a422a adding gpip test
7f028b4 moved into gpip test
82b1945 moved to gpip test
3e0e444 moved these into gpip test
ebab478 unit testing for Lib/Dirac
67a2a05 Moved all DIRAC script-like commands into DiracUtils.py. Also, did some work on bringing things into python std.
ac78843 Code updates for NIKHEF/SARA
6e164f1 avoid having two same downloading tasks in the task pool
c136aa7 clean up logging messages
1a4d35e Add Athena.user_area_path: configure path where to put user_area     file which is created during call of prepare()     Changed default to $TMPDIR or /tmp/$USER     If Athena.user_area_path='workspace' old default of ganga workspace     is used
87c3c71 allow to set the max. node number of a glite bulk job
2d32e1a last commit was incorrect file
71602e5 Updates for sandboxes. Will only work in the latest Dirac client
ed2b1e0 Fix for the master job numfiles2
0c5b539 Change in the .gz reading and fixes for numfiles for the FILE_STAGER
cef17f5 moved future import to 1st line
3173b5a fixed the gaudirun.py + stdout issue
f85d0cc removes dependency on version of root no longer on the Grid
16c0b9d use sampleFile argument instead of sampleList in the FileStager input.py
dfb9ec7 add support for FILE_STAGER mode remove support for DQ2_COPY mode
8699e12 enhance the ama specific athena job wrapper  - introduce exit code  - add compatible timing report to adopt the state collection feature in Ganga/Athena module
20d1751 set default max_event to -1 indicating nolimit
4357e14 Add OUTPUT_FILE_NUMBER reading for Tasks package
7f2890c stdout.txt gzipped
740d2ed Make logfile reading more memory friendly
feb408a Improves the overlarge sandbox handling to make use of the Dirac OutputSandboxLFN parameter and do the untarring.
be6aaf3 make the number of output downloader threads configurable
db00b82 Committed Hurng-Chun's changes for implementing a file access mode using the FileStager
8424591 Fixed a buggy equality
1b2deb7 Small fix
d705a2b Add try/except protection
0f06391 Introduce Athena.collect_stats switch and add master job statistics collection
2fd3d30 Enable DQ2JobsSplitter for NG
6f8ddff dataset name list fix
b9d405e Fixed bug in the schema relating to 'project'
97a090f Changes the SetupProject calls to use LHCBSCRIPTS
ba2be1e introduce histogram chart
c02302e Take lfc, srm info from ToA, allow for writing to remote storage, add timing info for HammerCloud
eaad489 Moved these to Gaudi unit testing
2f7cf43   - Use search/replace in TURL list for file protocol     sites IFIC, LIP-LISBON   - Patch for RALPP, IFAE and PIC localsiteid   - rfio/DPM fail-over improved   - print warning if lcg-gt times-out   - Add extra hosts to defaultSE for GRIF-LAL, GRIF-SACLAY
e1e805b Moved this test to LHCbDataset unit testing
eac5d8e Moved this test to LHCbDataset unit testing
803a5ac Removed some unnecessary print statements
f4ded48 show the test failure with exception traceback on stdout
f583279 tip from Andrei on multiple lfns
cd284c8 Adding unit testing for LHCbDataset
71e25e9 Bringing code upto Python standards
da94d2c savannah 44923: use the dirac job parameters to construct the lfn rather than guessing
42aa5f3 Added stuff about controlling output destination
290325f Another place where [] is the default value
1886b8e Small fix
7934398 Introduce use_blacklist switch and correctly apply GangaRobot blacklist
fc41e98 Revert NIKHEF change
76ddf89 Patch for NIKHEF due to unexpected ToA entry for the Topology
facbab7 fixed small bug in matching a description to the wrong field
5801dc5 updated fort DIRAC 3
2064466 Updated GangaRobot examples for Ganga 5.1 and Dirac 3
8223dc9 Fixes for NIKHEF/SARA
a7f0eca Fix for SARA
ba21c06 Better protection against wrong LFC output, correct IFIC and LIP_LISBON protocols, improve IFAE and PIC localsiteid, set GLASGOW to single TURL mode
899b914 remove useless file
1b64a97 Automatically updated template for 5.1.2
4ce02f6 Added PACKAGES-5.1.2
4b1a717 Added ReleaseNotes-5.1.2
6afa589 use Batch rather than explicitly LSF
68f6005 extra exception output if cannot load the plugins...
e078bee improve get_file_location routine
4652877 include ama_athena_utility.py in the input sandbox
8bdf390 enhance ama athena job wrappers
2aa7ddd load SimpleStagerBroker module
5446bef introduce new class for data-oriented brokering based on file locations
b9c03a8 introduce dm_util.py containing utility subroutines for DM
57b5dbf introduce shell subroutines specific for AMAAthena
4790cfc bug #44393: wrong redirection of stdout/stderr in Shell
9a9ec83 fix the confusing Grid Proxy validity warning and the warning will be printed when the lifetime is shorter than 1 hour
9aafc9d Fix timezone
6310c4f fixed bug in python script written to run jobs
d1d7b02 fix for savannah 44414 - recursive listing fails if called more than once due to the reference being kept between calls.
84330ad Change to dq2client_version = 0.1.22 and add it to Atlas.ini
f852aa1 Athena.stats updates for GangaNG
66e5578 Added -v to time
30494e0 Removed superflous useridARC check
911a213 Small change for DQ2_COPY workflow
5934a6e Add dq2clients requirements for DQ2_COPY mode to jdl
f644156 small fix for join() method
2a4fdc3 remove the uesless import: threading override Thread.join() to see if it helps in shuting down the daemon-like MTRunner
b4d06a8 Small fix
22b9234 removed localxml.ini and remote.ini from the test configurations
6d43bf3 introduce configurable number of worker threads
5e9ada0 Added time information to athena run
305ff71 GPIP (parallel) tests from Mason
b5e1143 GPIP (parallel) tests from Mason
6a95caf PYTF version 1.5: GPIP (parallel) tests from Mason
f47fbfe GPIP (parallel) tests from Mason untabified test.py
bbe4e95 GPIP (parallel) tests from Mason untabified the driver.py file
b78b3a9 Fix logic of siteindex_update
e568d0b Small fix
9cdf91c Add numfiles2
1e8e1d7 Add numfiles2
8e38007 bug fix: #106226  - wrong type setting for 'software' and 'others' attributes
1c5ae9e introducing "SubmissionThread" configuration variable for setting the concurrent number of job submission threads
19554d3 Add addition time parameters for Athena.stats
daf8905 *** empty log message ***
5f02fcb bugfix to allow production with locally modified production tarball. Also prevent creation of illegal dataset names in DQ2 by removing single quotes in user names and cropping long dataset names.
feaf83f Small fix
55d6a30 Factorize and extend job statistics
f728ba7 Update of timeout
1f7f9a7 First Version: athena HelloWorldOptions.py in 14.2.20 on LCG backend
97cd110 First Version: athena HelloWorldOptions.py in 14.2.20 on Local backend
234da8d Fix #44426 - DCACHE_RA_BUFFER should be an str for Local/Batch
9d7ecca print the "waiting for output downloading processes to be finished" warning message when needed (the downloader has been running for longer than 10 sec. when closing the Ganga session)
1b57a9a bug #43917: Implement Batch backend as alias to default backend at a given site
6b5cd09 bug #44259: GangaLHCb tests fail due to gridProxy check
6b7f140 Automatically updated template for 5.1.1
706d3ce Added PACKAGES-5.1.1
e64dd4a bug #44334: Ganga/Utility/Shell.py does not save environ
0210fbf #bug #44256: Job in state "incomplete" is impossible to remove
6b25a5c Fix bug #44372, change md5sum calculation procedure
fa548ed Automatically updated template for 5.1.1
b448ebb Added PACKAGES-5.1.1
d792b94 Added ReleaseNotes-5.1.1
1bd73da switched back to setenvProject.sh
069c359 Removes print statement
88cde8d Savannah 44261: Deprecates DiracTopDir in favour of DiracVersion. If DiracTopDir is set, but DiracVersion is not set then the current behaviour will be kept. If DiracVersion is set, it will override the value set in DiracTopDir. The default for both is to take the latest.
b629557 Automatically updated template for 5.1.1
e2a57f5 Added PACKAGES-5.1.1
3454edd Added ReleaseNotes-5.1.1
a83f72a Automatically updated template for 5.1.1
96caf9e *** empty log message ***
38b5dac first version
d9706c6 Added PACKAGES-5.1.1
1ec8282 Added ReleaseNotes-5.1.1
10c07d1 *** empty log message ***
2b53049 first version
756dd6f try
dd4d7df Fix bug #43702 - add exception and non-zero-return code
bc2e218 Fix bug #44213, input.txt is now written in DQ2_COPY mode
44f9c4c Improves the error reporting for monitoring
5be2d06 New version with official patch against RA_BUFFER problems
5bc1e94 Add -d option to dq2-get
c082c1f test for the grid proxy problems I've been seeing in the GangaLHCb tests
c95fcd2 Add DQ2JobSplitter.update_siteindex and small fixes
a6d1a82 fixed bug #44181
c864f61 Small fix
f4d03b0 Change single DATASETNAME env to DQ2_LOCAL and DQ2_COPY
c22f8fd Change log level of dataset registration
7da1719 Change splitting on dataset level only
17dd16b Add retry loop for dq2-get
ee5abd5 Add MaxJobsAthenaSplitterJobLCG again
26dde0e Make DCACHE_RA_BUFFER configurable
4ff9475 Fix bug #43946, DQ2_COPY mode for multiple datasets
bfff4f7 Add  Add check if input files has copied in DQ2_COPY
05f2bbe Add Athena.stats
48625da * AthenaLCGRTHandler.py: Fix bug #43946, DQ2_COPY mode for   multiple datasets * AthenaLCGRTHandler.py: Add warning message for failed   submission in dataset splitting mode
9246dd7 added property for defining Grid storage
e437009 joining the MTRunner thread to synchronize with the shutdown service
ae1d139 supStream support: suppress some output streams. e.g., ['ESD','TAG'] improved logging messages
1838ce9 update master job's status at the end of the master_updateMonitorInformation() in any case
633e605 update the master job's status after the downloading process is finished  - avoid leaving the master job's status not updated in the repository
d0f038e remove too frequent logging message
4a34270 Moved these files to options dir.
d8cd8d0 Adding unit-testing for the Gaudi directory.
0265906 Adding unit-testing for Gaudi.py
0959e3c Adding unit-testing for DSTMerger.py
64ac9ef Removed GaudiVersions test as what used to be in GaudiVersions.py is now fully tested via the unit-testing done on GaudiUtils.py.
1c447d9 Minor edits due to the move of DiracSplitter to the Lib/Dirac dir.
a5762ad Removed some hard-coding of platforms which caused the tests to fail.
8c48f86 Added addional config option to control output destinations.
bd8197b Modified all source files to bring them (mostly) inline w/ Python standards. Also, added functionality to give user full control over destination of output files.
c36fab8 moved DSTMerger.opts and GaudiPythonExample.py into the options subdir (to separate them from the source code)
4352164 Contents of env.py and GaudiVersions.py have been moved to GaudiUtils.py
9d2ffd0 removing this files - all tests should be under GangaLHCb/test/
66a0ecc Moved DiracSplitter into the Dirac directory. Also, made a few small changes to the GaudiDiracRunTimeHandler to work w/ the new Gaudi outputsandbox/outputdata format.
e6989db AthenaMCLCGRTHandler.py: Fixing condition trigger false warning in input data splitting
20d931c small correction
e63caa7 Fix for bug 42661: User packages not being recognised in Athena Root dir
1607c91 uertag in dataset name and outpu_lfn
f0ea778 fix wrong error message: savannah 43844
fbb2822 Updating default output storage location to CERN-PROD_USERDISK as CERNCAF is long obsolete. Relaxing constraint on job matching to let jobs running at CERN as CERN Tier 2 facility is not properly tagged in LCG BDII
48ccd82 modified cleanup
f2ecb52 Disable credential creation/renewal when Internal Services has been disabled.
a612653 Added mechanism to differentiate exitcodes. Fix for bug #40111.
1d0cfdc modified LD_LIBRARY_PATH
e01c6e1 Automatically updated template for 5.1.0
6247201 Added PACKAGES-5.1.0
f557aee Added ReleaseNotes-5.1.0
bb1bbbb file name fix
dafadeb reduce the task queue checking duration to once per 0.5 sec.
983b553 expand $VAR and '~' when setting path-like options
3be4184 added j.submit(keep_on_fail=False) option (request #43143)
31e20c7 fixed bug #40271
ea976d0 fixed Shell in case the setup script produces garbage on stdout (now discarded to /dev/null)
e43a92a introduce the exitcode in the athena_setup routine and stop the main wrapper when the exitcode returned is not 0
1d449b8 Automatically updated template for 5.1.0
288c5f4 Added PACKAGES-5.1.0
3db1f3f Added ReleaseNotes-5.1.0
fe02284 add workaround for the gLite WMS issue:  http://savannah.cern.ch/bugs/?43632
ccf1ec8 use the type system completely
dbc6cbc remove useless routines
da478b7 update the code annotation
8784c05 fix the bug in passing LFC_HOST to the job wrapper while using LCGSandboxCache
ac16078 Updates the lhcb workbook for Dirac 3
d616312 fix the bug triggering the annoying warning message after subjob resubmission
4111aa9 Automatically updated template for 5.1.0
1fad9c5 Added PACKAGES-5.1.0
ae8d927 Added ReleaseNotes-5.1.0
85ead93 Added PACKAGES-5.0.11
ad24185 Added ReleaseNotes-5.0.11
1cf59cb enhance the internal setup for the SandboxCache
97fcfe8 remove the LFC module python path coming from the external LFC library for GangaNG
b649779 updated config for release
5d2b850 removed spurious print statement
e43b81c Adds a warning if the sandbox was oversized.
a042cc4 Increases the timeout on this test
45eb9ad Added PACKAGES-5.0.11
646f7a0 Added ReleaseNotes-5.0.11
c516a59 fixed typo 'destionationSE'
cb00dbd AthenaMCDatasets: addding exception handling of incomplete records of input datasets in the DQ2 database (datasets with no known locations)
ce959ce Updated AthenaMC.gpi input datasets. The previously used datasets have been removed from DQ2 database, causing the tests to crash
3a224ec Removes some useless test cases and fixes some which didn't make any sense
87aa9ad Fixes the test after the changes to getpack. I wonder if this breaks ganga somehow?
f2f4e5f Automatically updated template for 5.0.11
e02e228 print added
5c5748f Added PACKAGES-5.0.11
966846e Added ReleaseNotes-5.0.11
a4fb8cb Upgrade to new DQ2Clients 0.1.22
3bf273d Fix bug #42944
9739405 old file name convention compatibility
779fb27 Fix for bug 43312. Test case exists and passes
6eba016 test case for bug 43312
2242317 Copies the diracMain into the inputdir
594c7b5 Updates the URL for the monitoring page
cc5c189 stops failed jobs from downloading their sandboxes
d76cd1e Fixes a bug in the error msg formatting that caused an exception
78b3870 Dirac 3 merged to HEAD. Need to perform lots of testing
7295451 Correct setting of max_event
71c226a Introduce splitting per dataset
7b47461 stop logging stuff
80574ac request #41137: Please add a nice level option to the Local backend
ab8bfa2 bugfix #40932: Ganga incompatible with shell functions (using os.environ directly instead of printenv)
b661363 test obsolete
1bc2db5 test obsolete
812ac0c disabled for LocalXML
fc78798 fixed wrong asserts
302fb21 fixed the rollback to new state problem of split jobs: reset the subjob counter to the original value so that j.subjobs[0].id == 0 (Ganga/test/GPI/CrashMultipleSubmitSubjobs testcase)
b2b7af8 move installation of  the shutdown manager for atexit handlers to bootstrap.py
5f693e4 install the shutdown manager for atexit handlers before loading system plugins (e.g. LCG download thread registers the atexit handler using a tuple (priority,handler))
3b63173 Added dependency on LCG/external Python2.5, pyqt (3.17.3_python2.5) and sip (4.7.7_python2.5).
c0b7b13 Automatically updated template for 5.0.10
380258d Added PACKAGES-5.0.10
f403f7c handle max_event as an int
63b9f13 Added PACKAGES-5.0.10
97cb8aa Added ReleaseNotes-5.0.10
9306ff3 Removed debugging message
19130e3 Fix for maxevt
87e10f5 maxevt is now int
aac55d2 min. limit for sandbox/groupArea upload 5M
613fede Removes an incorrect test and adds small fix to make another pass.
7d3fb4a Changes default to be [] for gList.
486ee45 Changes the Robot testing to use a different gangadi and different user name such that it can be run at the same time as the Ganga core testing.
b49e545 potential fix to the dataset thing.
0036823 most stupid bug in the world (I really wonder how it ever worked)
9453589 comment out prints
cce07cb Automatically updated template for 5.0.10
9cae01c Added PACKAGES-5.0.10
20ef14d Automatically updated template for 5.0.10
f113438 Added PACKAGES-5.0.10
fdf3667 Added ReleaseNotes-5.0.10
cf8fd61 Added ARA support, setup of local databases
caf6241 Add AtlasTier0
468d8b4 creativ bug fixing
16ae788 Change defvalue of partition_number
d93c3f8 add lib64/python removal
f40013f Add dcap config for SFU, TRIUMF, ALBERTA
5500b18 Change python32 workflow
8370f64 Remove special MPPMU setup
d1a7fc1 Fix HelloWorld job for Local/Batch backend
3162771 add AtlasPoint1 and AtlasTier0 projects
0fd5adc fix typo: self.nrjob -> nrjob
3d2fc8e add 'AtlasTier0','AtlasP1HLT' to alternateCMTextractor
85a345d removed dependency on external pyqt.
35fd6f4 Remove PandaJobSplitter (implemented in DQ2JobSplitter) Add brokerage if not using splitter
d048375 add runPandaBrokerage and queueToAllowedSites functions
046ff3b Use DQ2JobSplitter for Panda backend
8cc0968 Support Panda backend; performs brokering and handles incomplete datasets.
5b3fef8 Add _append_files routine and add libdcap.so
0e8c709 Set read ahead buffer for dcap access to DCACHE_RA_BUFFER=32768
be3cb3b athena-utility.sh: remove duplicated ganga-stage-in-out-dq2.py statement that could cause false double output file registration
25c98ec DQ2JobSplitter creates the correct number of subjobs for complete datasets (Bugs #41726 and #42856) Creates at max MaxJobsDQ2JobSplitter per site for incomplete datasets
976be6e Athena.max_events has to been an integer now (Bug #42613)
b1604b5 Bug fix for inputdata.inputfiles type, improving timeout evaluation for stage-out, allowing support for stage-in files from SRMV2 space tokens, extending splitter to allow partial resubmission of master jobs (all failed subjobs at once instead of one by one.)
801e72f Modified Gaudi application and runtime handlers so that they now place all .py files under InstallArea/python into the job input sandbox. Previously only *_merged_confDb.py, *_confDb.py, *Conf.py and _init_.py were copied (Configuration.py was missing).
a52cfa5 fixed symlink creation
afd8cf1 removed dependency on Python 2.3
3c1ac7d bug fix
50ff8df lfc updates
b9fcc8d - Added test for GaudiPython on DIRAC. - Removed some duplicate tests.
4e5bc56 Bug in translation of dataset object for GaudiPython jobs running on DIRAC.
8cea8b7 cmt setup, adler checksum
31376cf LFC tools for lfc registration
f6b0d6b Transition to LFC
a9307e6 Update ARC client 0.6.4
e28f1a8 uegede: separate TEST REPORT step with automatic download of testing from external sites (Imperial)
62dead9 Automatically updated template for 5.0.9
ac2bf6c fix the symlink to coverage reports
16f4d86 Added a new action to the TestRobot which is responsible for creating an archive .tgz file with all the test results.
575dc58 PandaJobSplitter now handles the brokerage and splitting of complete/incomplete datasets.
f922beb support dataset files in local directories
d8b39a1 Added PACKAGES-5.0.9
ee4684a Added ReleaseNotes-5.0.9
de44e20 added typelist information for submit_options
0101a6e avoid doing glite-wms-job-cancel on jobs which is in a final state  - glite bulk job status is now correctly stored as master job's status
8e4493c Increase maximum number of subjobs
81ca60a Another fix for bug #41726
a61fbe4 add extOutFile
26777f9 update to Ganga 5
b84107f update to Ganga 5
b013f2a First import to new Ganga repository
3562f2b Now give more advice about what to check if there is an error when eval()'ing the output from gaudirun.py.
3779bea Can now pick up multiple ntuple output files.
27b1a40 Small fix
620abc7 avoid downloading again the output of the jobs failed in the previous downloading task
617a145 Fix _USERDISK problem
481fc5b Remove forcing of outputlocation cloud and allow number_of_files again
f1bd9de Added PACKAGES-5.0.9
9d69bdf Added ReleaseNotes-5.0.9
7f8b734 deprecated slc3 platform and replaced with slc4_ia32_gcc34
1bd61ce bugfix #41372: added backend.remove() method to support job removal on the Remote backend
c5b756a updated configuration URL for monalisa usage monitoring server
5026497 deprecated slc3 plaftorm and replaced with slc4_ia32_gcc34
ba3cf8b Traceback information added if eval() of options fails.
3564a89 Bugfix. Removed [0] since the microDST name is not returned in a list, but just as a string.
9f7f434 get_output_file now detects if the user wants to write a microDST file and adds it to the job outputsandbox.
e1c130a initial import of PandaPilot application.
41e8d17 Added PACKAGES-5.0.9
5ef03c5 Added ReleaseNotes-5.0.9
1eaf21a Small fix
1184967 fine tune the typelist attribute in the schema
66af64c Small bug fix for local dataset merging
46d8a0e Small fix
e15ddf1 Add failover mode
71c79d0 Add failover
fb7ff82 Add DQ2Dataset.failover
27e0aa6 stage-out.sh: prevent script from creating incomplete output_data lines which are screwing up completion stage (jobs remain forever in completing state if the output_data lines are not complete)
4a8a66d change support line to new DA help forum
48fe681 Update to use dq2-get for tagdatasets
0dfd2cc Fix workflow for dq2-get
e236f8f New DQ2_LOCAL_SITE_ID detection
d620ee3 small fix for dq2client version requirement
981d8cd make dq2client_version configurable through AtlasLCGRequirements
c58ef7a Switch off RA_BUFFER
79373ce replacing dq2_get with dq2-get
9e75183 fix the type checking issue
0c632da fix the type checking issue
60f42cb Fix workflow for DQ2_COPY mode
f09e4d7 Fixed type checking for NG.py and NGRequirements.py
06d3a04 Small fix
0c96cc3 Change dq2info.tar.gz unpacking
6e0acf4 fix for type checking
7b8d302 Fix DPM problem and add python32bin and set RA_BUFFER=32768
1a89e87 New version
bd4e0f0 Add typelist=['type(None)','str']
a3cf127 introduce prepare_athena
aeca4d4 remove dq2_get and use dq2-get instead in DQ2_COPY mode
7318652 Better protection for TiersOfATLAS.getSiteProperty(sitename,'srm') calls
7e43228 Fix submission problem with DQ2JobSplitter
b7c68e7 Add typelist=['type(None)','int']
79f4c97 Remove obsolte DQ2Output class
871aa57 More bug fixing on space token treatment
e70c14f Updates the schema definitions for Ganga 5.0.9-pre. Just adds a few type(None)s to the allowed types.
19d9c69 fixing AthenaMCOutputDatasets startup warnings
ceaee49 - Bug fixes to the test Robot - Fixed bug in initialisation of TestRobot package which prevented a Ganga   session from starting the TestRobot.
2311a8d Added PACKAGES-5.0.9
9af8acd Added ReleaseNotes-5.0.9
e6fdbb4 Combined the functionality in lcg and local scripts. Created new versions to aid rollback.
3be74df Added resubmit and kill methods
e6c302f Print warning if srm entry is empty in TiersOfATLAS
6bf462d Fix bux 42044 - drop used guids in DQ2JobsSplitter
677d81f Disallow submission if number_of_files for incomplete dataset is used in splitting mode
070eab8 include AMAAthena module
22304a1 update ama specific athena wrapper script
ba478be Bug fix for DBRelease mecanism. User can now use any kind of DBRelease available as DQ2 dataset by setting application.extraArgs='DBRelease=X.Y.Z'. The job will then look for a matching DQ2 dataset and add it to the list of input datasets to be handled by the jobs
024c69f fix the status update logic
25e506e commit the LCGOutputDownloader.py
32328a9 cache the logging information coming out from the LCGOutputDownloader threads
a16b2c6 turn off debug message of the MTRunner objects try to avoid the race condition amoung concurrent threads
9555134 improving job submission/output fetching performance
dd1a8ee fixed bug 41726
55c5bd7 improve the MTRunner:  - allow dynamically inserting data to be processed  - allow daemon-mode MTRunner listening on the data queue
ea3bc73 athena-lcg.sh: tentative fix for database lock by making copies of the DB flat files on the worker node. AthenaMC: bug fix to open support to LOCALGROUPDISK space token and better general handling of space tokens for writing
855a20e add SandboxCache implementation based on ATLAS DQ2Client
801a1c9 improve sandbox cache handler and adopt it in the LCG backend
73818ab disabled a complex condition
90aa27e Automatically updated template for 5.0.8
45c5570 Added PACKAGES-5.0.8
2f223f9 Added ReleaseNotes-5.0.8
a8da3bd Automatically updated template for 5.0.8
713990d Added PACKAGES-5.0.8
2aa9944 Added ReleaseNotes-5.0.8
061eac2 use DQ2 config usertag
1f54026 temporarily disabled running Root tests on LCG and LSF (due to timeouts), a proper solution is needed for the future (either splitting the submit and verify phases, or to make a test backend that woudl exercise the rt-handlers or to move it to a generic robot).
7e06c59 Removed AthenaMCDatasets config block from Atlas.ini and modified AthenaMCDatasets.py to use DQ2 config block instead
4fd7989 adds returns to the example jobs
f63cf2f Add usertag config option and change DQ2OutputDataset name
ca503a2 initial implementation for extensible GridSandboxCache
e2f3ba6 Added PACKAGES-5.0.8
94fe23e Added ReleaseNotes-5.0.8
21e2c54 Fix srmv1 site problem
f142ca8 Added AthenaMCDatasets config member to upload userXX project tag from Atlas.ini. This tag is used in building output dataset names
3d3eaad fix for bug #40334
3f7a80b setup for loading pre-deployed dq2client on the WN
721d225 fixed: j.submit() and j.resubmit() do not return the status but raise exceptions on error
88d1124 fixed typeinfo
98c420f bug #40696: Exception raised during resubmit() should be propagated to caller
a674b3e bugfix #40220: Ensure that default values satisfy the declared types in the schema
061a61a #38646 bugfix: accept jobs.select(xrange(..))
5e88e7a Automatically updated template for 5.0.7
0a5e462 Added ReleaseNotes-5.0.7
72c4b9a remove the fixpython magic function which has been merged into GangaAtlas package
a8ae0b7 Another fix for PIC
94b0fe8 Automatically updated template for 5.0.7
524f331 Added PACKAGES-5.0.7
1c9d0cc Added ReleaseNotes-5.0.7
97b0e38 less spammy status changes. (Only updateStatus when panda status has changed).
6c416cb don't marked completed when build job finishes!
403f6b6 Automatically updated template for 5.0.7
586fffc Added PACKAGES-5.0.7
929d21a Added ReleaseNotes-5.0.7
dc6c545 XML differenciater added (from Ulrik)
136ddd3 A few more files missed in the first update
452c287 Fix a few bugs in test case comparisons and in the Test instance of the Robot
baa3ea5 Automatically updated template for 5.0.7
9cd88b9 Added PACKAGES-5.0.7
52e4cde fix bug in update
81b86c3 removed file again as it will cause trouble with the way tester works at CERN.
02862bd Removed file as it will cause trouble at CERN in current way test framework works.
d3553fd Automatically updated template for 5.0.7
43bf1f2 Added PACKAGES-5.0.7
141c0f7 Added ReleaseNotes-5.0.7
a8c289f Updated file to write to correct directory.
e177f6d Added file for testing at Imperial
7e6b735 Added Imperial.ini file for the testing to happen at Imperial.
ab59315 Update of default INI file with more sensible parameters.
2059fa5 Forgot to add Framework/xmldifferencer.py
232e291 New features: - Remote testing of Ganga thorugh the use of the GangaRobot - Ability to create test reports showing differences between releases.
863b3d9 Automatically updated template for 5.0.7
16fbffb Added PACKAGES-5.0.7
04ebd66 Added ReleaseNotes-5.0.7
9bdf44a Fix alternateName and PIC problem
15feff5 removed 'completing' state
2e7719f added unknown, starting panda statuses
c1e6a44 Automatically updated template for 5.0.7
a79b64c Added PACKAGES-5.0.7
eaca2c3 Added ReleaseNotes-5.0.7
5b2392f fix the type-checking issue when setting up CE attribute
56f9b50 Add fixpython magic function
2960f4b Added PACKAGES-5.0.7
c23cccf Added ReleaseNotes-5.0.7
36e4845 removed panda client
1c2dccd removed dq2_get
c690bc3 removed config extractor
dc17a9b verify outDS starts with user08.DN.ganga. rewrite cloud in case user specified a 'site'
4e7e430 Add machine and time info print outs
c908037 Small fix
1de65fa Add machine and time info print outs
dcd3af6 Small fix
f371b0e Add cleaner error message
93ae0b3 Remove tabs
1e31350 Add glite UI setup
2de882c Remove gsidcap for UVIC
f9074a0 Use external PandaTools Added cloud Removed useless dq2_get and getQueue EXPERIMENTAL: Added resubmission:     job(x).resubmit() will resubmit the _failed_ subjobs to Panda. Removed useless gridshell Cleaned up status update function
346d3e0 Use external PandaTools Change outDS to user08... Add brokering with PandaTools runBrokerage to find best site in cloud
477fcd8 Make PandaTools an external dep
a47cf62 add a magic function for fixing python conflict between ATLAS and LCG
1e61e50 resolve the TMPDIR issue on LSF machines at CERN  - take WORKDIR instead of TMPDIR for job's scratch dir.
8aaa597 Savannah 40910 - Also problem with datasets in options files
3c751d2 Convert to unix and remove tabs
e678a1d Update to 0.1.20 DQ2Clients
de0a292 bug fix #40268
884eed4 Add gsidcap access rule for UVIC
c3f6a53 Athena: * Fix SE type detection problems for space tokens at DPM sites * Change default DQ2 stage-out to use _USERDISK token * Change default DQ2Dataset default values of   DQ2_BACKUP_OUTPUT_LOCATIONS * Disable functionality of DQ2Dataset.match_ce_all and   DQ2Dataset.min_num_files and print out warning.   These are obsolete and DQ2JobSplitter should be used instead. * Add option config['DQ2']['USE_STAGEOUT_SUBSCRIPTION'] to allow DQ2   subscription to final output SE destinations instead of "remote lcg-cr"   Will be enabled in future version if DQ2 site services are ready for this ============================================= A few changes to enforce the computing model: ============================================= * DQ2OutputDataset: j.backend.location is verified to be in the same cloud   as j.backend.requirements.cloud during job submission to LCG * Add AtlasLCGRequirements.cloud option.   Use: T0, IT, ES, FR, UK, DE, NL, TW, CA, (US, NG) or        CERN, ITALYSITES, SPAINSITES, FRANCESITES, UKSITES, FZKSITES,        NLSITES, TAIWANSITES, CANADASITES, (USASITES, NDGF)
87f1b4f updated athenamc with proper types for j.application.firstevent and j.application.number_events_job
64f4a79 Gave a type and a default value to app.number_events_job
65cf290 set default DQ2_LOCAL_SITE_ID to CERN-PROD_DATADISK
ac4de17 introduce AMAAthena module into GangaAtlas package
c45ff22 Savannah #40903 - Update to use same mechanism as Ulrik with the Gaudi apps. Tests still pass.
9e4e807 Savannah #40903 - Submission of unsupported architestures for root jobs. Now the architecture is taken from the ROOT config section of the gangarc file. Also adds test.
7e47a2d Test for Savannah 40910. Waiting for options file from Mitesh to verify that results are not stupid.
089fe7b Savannah 40527: Updates the default root version and architecture to solve a failing test. Should only really affect the generic tests.
1162039 fix for Savannah 40219 - Manages cache updating better.
e242d8d Modified failing test cases.
5d7e336 Added PACKAGES-5.0.6
c3449db Added ReleaseNotes-5.0.6
e7ce695 Fix _color problem
8ac0fc2 fixed problems found in 5.0.6-pre:
5746d57 Automatically updated template for 5.0.6
c54d1a7 Added PACKAGES-5.0.6
79775d9 Added ReleaseNotes-5.0.6
db4f508 removed python 2.3 dependency (not needed anymore)
95cebac New features: ============= The Gaudi and GaudiPython applications have a new attribute called 'setupProjectOptions'. It contains extra options to be passed onto the SetupProject command used for configuring the environment. As an example setting it to '--dev' will give access to the DEV area. For full documentation of the available options see https://twiki.cern.ch/twiki/bin/view/LHCb/SetupProject. The 'lhcb_release_area' attribute has been taken away as it was not useful.
76593bd Removed python dependency
8ca0a26 remove python2.3 dependency
c38f4a6 Bug fix to avoid problems in tcsh
56eb9cf Work around for #30882
8d4585e Fix bug #40269, ATLAS_PRODUCTION environment variable is now handled properly in Local/Batch backend
ed4fe19 Fix bug #39010, Add ATLASLocalDataset support for list of datasets
9383a7f Introduced exceptions for the Robot framework.
809fe05 jobs.select([ids]) added
8adc1d4 Updates to LHCb manual
edd034f added force_status() method to replace job.fail(), force_job_failed() and force_job_completed()
8e7307d Now raise ApplicationConfigurationError if there is an error when parsing the job options.
e2b0b36 bugfix 40110
caa1e1d *** empty log message ***
51a9a54 Changed the Dirac splitter to work with the modified LHCbDataset
539d8b0 Test case added
7806091 Added "datatype_string" to schema for LHCbDataset. This allows Ganga to run with cosmic data.
60a40f9 Automatically updated template for 5.0.5
7aa57c1 Added PACKAGES-5.0.5
98ea6e7 Added ReleaseNotes-5.0.5
a1ae740 Small fix to version number of updated manual
4d0eb91 Updated LHCb manual
266f7de Fixed gaudiPython to work with splitters
4e1e4a7  - remove redundant functions  - set minimum timeout of downloading oversized inputsandbox to 60 secs.
5f62ce7 - improving oversized inputsandbox downloading   * add more debug information   * automatically determine the lcg-cp timeout assuming the rate of 1MB/sec   * add config.LCG.SandboxTransferTimeout allowing user to set it manually
acd108f Fixed bug when detecting the source of inputdata. Improved logging messages.
29fba5b Added configuration option AllowedPlatforms to the DIRAC section. Only jobs of this configuration will be allowed for Dirac submission. At the moment defaults to just slc4_ia32_gcc34.
8a08bdd  AthenaMCLCGRTHandler.py: extend lfc patch to cavern and minbias input data. stage-in.sh: fixing a typo in previous bug fix
128c826 optsfiles can now be specified as a string rather than a list if only single file required.
e19bb76 Added a GaudiPython application handler with runtime handlers for Local/Batch and for DIRAC.
b47a0cc Bug fix for input lfcs: applying single quote prescription which is working for the other input arrays (lfns and turls). Also, force stage-in.sh to use all the lfcs in the array instead of just the first one
5f4a336 AthenaMCLCGRTHandler.py: Adding protection against writing output data to dedicated production endpoint. stage-out.sh: added -s flag to write to ATLASUSERDISK space token when the destination (se_name) is set to this kind of endpoint
6b9c93d Automatically updated template for 5.0.4
781bd01 Extended _get_user_dlls to pick up all relevant files under InstallArea/python in the users private project areas. This directory structure is now replicated in the job inputsandbox. PYTHONPATH in the job wrappers is modified to prepend it with `pwd`/python so that the python configurables in the input sandbox are picked up by the job.
84e0dc9 Added ReleaseNotes-5.0.4
0922e85 build external tarballs for slc3_ia32
4875691 paramiko syspath fix
91ceefa Added PACKAGES-5.0.4
15d1957 Added ReleaseNotes-5.0.4
5e6c315 bugfix: #39655: Problem with slice operations on templates
45125f0 pycrypto sys.path fix for both 32 and 64 platforms
6411171 fix output_location to handle srmv2 space tokens. This allows to write data files to srmv2 space tokens. Restriction to USER space tokens will be implemented when a critical mass of these dedicated endpoints will be reached
b4b455f Automatically updated template for 5.0.4
db70225 Added PACKAGES-5.0.4
c30af90 Added ReleaseNotes-5.0.4
6b8cf89 Merged the new Gaudi application handler from branch
e86c596 Automatically updated template for 5.0.4
5bdc03d Fixed a bug that caused new classes not to be found by gaudirun if a non-default CMT area was used.
3de1f4e typo fix
6fb9f81 Automatically updated template for 5.0.4
ba1e070 disabled tarball creation for scl3 (obsoleted)
61fff8c GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
5b891c1 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
b49f656 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
38f0eed GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
9aeaabe GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
21f4f1b GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
513f7ff GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
3cac121 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
bab33f6 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
69dd403 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
76eecdd GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
69ecb73 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
c531b38 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
8e7b2b8 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
2679fa5 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
c446ff0 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
a20f3aa GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
7426172 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
2e6f2df GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
bf6707b Added PACKAGES-5.0.4
f6f6e5d Added ReleaseNotes-5.0.4
76c18c5 Updates related to job slices.
edfbc65 added pycrypto and paramico to the list of external dependencies (for Remote backend)
214ef99 Remove enforcing of DQ2OutputDatatset loction
c5abf42 Automatically updated template for 5.0.3
f41932d config templates are now in a separate directory at top level ("templates")
7805048 config templates are now in a separate directory at top level ("templates")
03868e9 Fix but #39568: Panda/DQ2Dataset: Need better error message if DS doesn't exist
b28445d Updates and Bugfixes
bb3643c AthenaMC.py: added type checks to members. AthenaMCDatasets: bug fix for treatment of DQ2 datasets available only for Panda. AthenaMCLCGRTGandler.py and wrapper.sh: extension of the glite WMS workaround to input data
a78bb9d Fix bug #39549: raise execption if (athena_compile==True) and (NG==True)
3e19c1e Change DQPing to utils.ping
96bf774 Change DQConstants to constants
c6fee63 add timeouts of 300,1200,2400s for lcg-cr stage-out
5d8d1c2 fix indentation issue in the code
3f7b80f Print out warning to use jobs(id).subjobs instead of subjobs
d42a012 Debug print-outs during compilation
e149f45 cvs commit enabled (but does not work, TEMPLATES should be moved to a subdirectory which is always on HEAD).
ec89a24 commit config template files
d21193b Add  AthenaRemoteRTHandler class for the Remote backend
bc43299 Removed some remaining info messages
2b01bbc Updates and improvements
2020f5f Remove DQ2_OUTPUT_LOCATIONS again
19c12d0 Upgrade to ARC 0.6.3
dd82197 Added backup atlasgiis server, fixed autonaming of jobs with DQ2 datasets
0091c53 Changed configuration section for Robot to be open to allow new actions to be defined.
6c83a2c Small fix
fdca872 * ganga-stage-in-out-dq2.py:   - Add fix for DPM setup for NIKHEF and SARA   - remove special SE setup for RAL   - add MPPMU defaultSE=lcg-lrz-se.lrz-muenchen.de   - add DQ2_BACKUP_OUTPUT_LOCATIONS reading   - add DQ2_OUTPUT_SPACE_TOKENS reading   - change stage-out order for DQ2OutputDataset:   (1) j.outputdata.location, (2) DQ2_OUTUT_LOCATIONS,   (3) DQ2_BACKUP_OUTPUT_LOCATIONS (4) [ siteID ]
d6d0b07 Ensure curl is installed when loading Panda
3f01f4c list_sites now gets from Panda server
72e3ee8 update from upstream
855e1d6 Fill subjobs DQ2 datasetname with auto-gen name
4f71985 update from upstream
ec08490 Fix for bug #35256
95122c4 Add Remote backend submission to AthenaLocalRTHandler
8a3b584 * Upgrade to DQ2Clients 0.1.17 and DQ2 API * Add full support for DQ2 container datasets in DQ2Dataset * Change in DQ2OutputDataset.retrieve(): use dq2-get * Fix bug #39286: Athena().atlas_environment omits type_list
b1e5bc8 Updated the Remote backend that include:
f67a637 patching up to the up-to-date development after CVS migration
ed8e1e4 Adds support for merging dst files with DaVinci
e34127b adds support for root > 5.16.00
eb1d50c adds a CustomMerger
b152aa1 adds a CustomMerger
1e96078 Added ReleaseNotes-5.0.3
cf504e3 Added PACKAGES-5.0.3
2a4b033 Added ReleaseNotes-5.0.3
a215d46 Added PACKAGES-5.0.3
17a84ec Added ReleaseNotes-5.0.3
01d8b28 migration of 5.0.2 to HEAD
REVERT: 161626b 
REVERT: 2662b38 strip the location of the output file
REVERT: 6ade00d Fix for bug #90890
REVERT: fe76f0d Needed to handle default, string executable arguments
REVERT: cc26f4c 
REVERT: ddaba19 pass more files to the outputsandbox, the files that should be uploaded to lsg storage element
REVERT: 986ec28 
REVERT: e101e83 changes for uploading of the output files to lcg storage element
REVERT: 338406b Correctly ship sharedir contents with job to all backends
REVERT: ea6fddc Correctly ship sharedir contents with job to all backends
REVERT: 3bf4f5d 
REVERT: b99a17d 
REVERT: e817477 changes for uploading the output  to lcg storage element
REVERT: 95ec04f update the postprocess file on resubmit, there could be some changes in the outputfiles between submit and resubmit
REVERT: 9df333e reduce duplicate entries of the outputfiles list
REVERT: c9e0e75 fix for automatic output file type detection
REVERT: adee5f9 raise config error if filename is defined more than once in output config section
REVERT: 0df454c config default value changes
REVERT: e12ef10 remove repeated entries from input and output sandoxes
REVERT: 22e0dec add non repetead entries for input and output sandbox
REVERT: 4e3da2d 
REVERT: 9484175 
REVERT: 783e490 
REVERT: 67b823b kepp the postprocessoutput file always updated
REVERT: 76edbd8 
REVERT: 78fbffb 
REVERT: 826fbe4 
REVERT: 2fee826 
REVERT: 60a7e59 
REVERT: efca7c7 
REVERT: a146839 
REVERT: 0f47a8c 
REVERT: dcad0a1 
REVERT: a5ecc5f changed config values
REVERT: 25fc949 
REVERT: 5b1faab 
REVERT: 20329d4 
REVERT: 97b1ac1 changes for retrieving the output from the lcg se
REVERT: 489139b Check for existence of shared directories when we copy an application
REVERT: bf0292a 
REVERT: e61f90b Catching this exception was causing problems elsewhere
REVERT: 9780904 Iterating over each prepared application to check that it's sharedir is present is too resource intensive
REVERT: 52f835e changes for uploading the output of CREAM jobs to lsg storage element
REVERT: fba1c46 
REVERT: 15d7b60 
REVERT: cbd1f01 
REVERT: bcb07ad changes for the output files
REVERT: a2867e6 added get method that in principle should retrieve the output file locally
REVERT: aed8f00 Reverting changes made in revision 6936 (dynamic checking of ShareDir presence)
REVERT: 8e03755 
REVERT: a4e5c9c Switched to dynamic checking for presence of shared directory, rather than loading very job object when we exit ganga (which can be very expensive).
REVERT: ce8c6ed Removed commented line
REVERT: 353401e set the default objects properties to config values
REVERT: e1b3fdd instead from configuration , get the lfc host and se from the output object itself
REVERT: a41fd1b 
REVERT: 807ba5d setting the locations of the outputfiles
REVERT: da443cb added methods for setting the location of the uploaded output files
REVERT: 1aa8d34 changes for uploading of the output to lcg se
REVERT: 4433931 
REVERT: 95aae6a 
REVERT: a9f5faa 
REVERT: 3657fae 
REVERT: e43ee4e 
REVERT: 4e803ab 
REVERT: 87291a4 source code reorganisation
REVERT: 66d79f1 
REVERT: 463ea83 changes for uploading the output to SE
REVERT: 00eb540 
REVERT: 03ea8b8 Fixes for shareref counter and broken write-protect on protected=1 attributes
REVERT: 93aa32a Tweaks to fix sharedref counting
REVERT: 7aaa23f The unprepare method is now required
REVERT: 989a274 changed config value
REVERT: 4805d38 
REVERT: 658eee7 fixes for post processing of output
REVERT: 9c0016c changes for postprocessing of output
REVERT: 3d3febf Undoing the last commit. You fool
REVERT: fb8fff1 Fix for bug #89869
REVERT: e8f885e changes for postprocessing of output
REVERT: 735f376 We don't need to export the unprepare method for the Executable app
REVERT: 0e63225 
REVERT: 287f592 
REVERT: 7cc9488 
REVERT: 0378f50 
REVERT: a3a7b78 check for post process output file existance
REVERT: 468e99a adding the postprocessing file to the input sandbox
REVERT: ed9c481 added config section for lcg se output storage
REVERT: a2f70fa changes for output stored on lcg se
REVERT: 24772a9 check first if this postproces locations file exist
REVERT: 8c229ec 
REVERT: 76a5a6e postprocesslocations file could not exists, so make a check first
REVERT: c3b67dd 
REVERT: 4a6a89a Implementing feature request #89441
REVERT: 43d63d5 uncomment and empty the location list in the constructor
REVERT: 73103c3 uncommenting
REVERT: 62a3d1e commented one method
REVERT: 5089343 commented one method
REVERT: 352ca14 
REVERT: bc1ce02 
REVERT: cfb4944 
REVERT: 5a08b2b put in the locations list only diff paths
REVERT: 2cad9ee 
REVERT: e5d0f49 
REVERT: bdc15b0 
REVERT: a755cec 
REVERT: d830231 set location of the output files uploaded to mass storage
REVERT: 4b79a86 
REVERT: 26c01f5 small fixes
REVERT: fec2c1a set location of mass storage output file
REVERT: ea59478 setting location of output files stored on mass storage
REVERT: aa331b7 
REVERT: b0a35fd implemented method for local retrieval of outputfiles uploaded to mass storage
REVERT: 74b15c9 small fix
REVERT: dbf0cc5 
REVERT: 6873663 added comments
REVERT: b48504a remove the postprocesslocations file after job is completed
REVERT: a97c280 added abstract methods for location of the output file
REVERT: 00a1c2b added method for the location of the mass storage files
REVERT: c49c759 
REVERT: 9f617d0 set locations of the outputfiles
REVERT: 7e939f7 small fix
REVERT: 863930f pass info in the outputsandbox for output file location
REVERT: 4ded165 use glob instead of regex for names of the output files for post processing
REVERT: db82c7d use glob for post processing of output instead of regex
REVERT: ee20018 use glob instead of regex for output file names
REVERT: 6916274 removed comments
REVERT: cd3f51e small fix for glob
REVERT: 5456261 check for putputfiles pattern with glob instead of regex
REVERT: 53eda29 use glob instread of regex for file name patterns for output files
REVERT: 5088660 removed not needed code
REVERT: c1b2490 small fix
REVERT: bb64bfc replaced regex string with wild char string for output files
REVERT: cd67951 added new method of handling of the output
REVERT: 50b483c imports of file objects, added posibility to instantiate them from the config
REVERT: fd45647 import file classes
REVERT: 35ac1c4 fixed tabs
REVERT: f33eae0 added config options
REVERT: 978f2ab added config section
REVERT: ae1e836 Removed startup message informing users of new XML repository
REVERT: b7bec59 Enabling prepare() method for Root application
REVERT: e8290c8 Profile log messages are too verbose to have at INFO level
REVERT: 6c0c76c Slightly improved the tasks.table() formatting
REVERT: 88e6f47 Committed on behalf of Mark
REVERT: 278f656 Fix committed on behalf of Mark
REVERT: cf58beb postprocess of output changes
REVERT: 90b9737 some clean up
REVERT: 15bc12f post processing of output
REVERT: 86bee06 more debug info added
REVERT: 59e1f5c debug info added
REVERT: ed4560f more fixes
REVERT: 94723c9 more fixed for output post processing
REVERT: 0310353 more fixed for output post processing
REVERT: 3740dda more fixes for post processing of output
REVERT: f4de307 small fix
REVERT: e6e5721 add postprocessing output to the downloadable files on the grid
REVERT: cc932d6 changes for postprocessing of the output
REVERT: df1186f import subprocess package
REVERT: b093198 Fix for bug #89110
REVERT: e073817 forgot to import re
REVERT: f35351b add shared area to the report generated from the report() function
REVERT: 3d99568 changes for postprocessing of the output on the WN, upload to Castor
REVERT: c0f1a07 removed comment
REVERT: 90041ce added postprocessing of the output for CREAM on the client, for Castor uploads
REVERT: 9165c87 added postprocessing of the output on the grid, upload  to Castor mass storage
REVERT: dfeb8cf removed handling of the output at the client for Local backend
REVERT: 529705b if there is no CASTOR_HOME env variable set, set the default path to the home directory of the user in Castor
REVERT: 52cd02d more bug fixes
REVERT: 10a311d more fixes for mass storage uploads
REVERT: 85230f6 small fix
REVERT: c364042 small bug fix
REVERT: 18944e6 put the castor upload on the clinet, just for test
REVERT: 622c912 small fix
REVERT: 2af0155 small config fix
REVERT: 56ecebe expect regex instead of exact names for postprocessing of the output
REVERT: bc31728 fix for compressing output in the Batch, expect regex instead of exact names
REVERT: f23e176 file removed
REVERT: 88758c6 
REVERT: 62b9c71 
REVERT: a6d2e13 
REVERT: e7736a0 commented one config option
REVERT: 089eb72 test
REVERT: 1e0411e test
REVERT: deaa98e test
REVERT: bbb68c5 removed checks for castor upload, if there are error they are written to the stderr of the job
REVERT: 9f9b40c changes for Castor uploads from Local backend, write to stderr if there are problems with access to castor and copying the files there
REVERT: a1b74a3 bug fix
REVERT: 8356bd3 making tests
REVERT: 1fd3b32 Feature request #88651; make unprepare_on_copy a configurable option within gangarc
REVERT: e5901bc bug fix
REVERT: 7c45bc1 code reorganising
REVERT: c27b081 write to postprocessoutput file instructions for castor uploads, also check if castor uploads are made from lxplus, only from there they can be done
REVERT: 23800ee changes for Castor uploads for Local backend
REVERT: c362334 added section for mass storage output
REVERT: 479f544 mass storage file added
REVERT: 9aa5ed2 output file for mass storage
REVERT: 710c56f Implemented j.time.runtime() for batch backends (Bug #86537)
REVERT: fc82db0 Merging 5.7.0 from Ganga-ConfState branch into Trunk
REVERT: 9a09db8 changes to the job script for compressing the output if there were such instructions from the user
REVERT: e7695be changes for the post processing of the output
REVERT: b007db6 added OutputFileItem class for the outputfiles category that is used for the post processing of the Ganga output
REVERT: d1a42ca added configuration for the Output section - this defines which output files to be processed how
REVERT: 4d73cd4 added method for postprocessing output in the Ganga client, for batch submissions it is not possible to postprocess it on the worker node , i.e. __jobscript__ file
REVERT: 148b94f added postprocess method that will define the way of psotprocessing the output for backends that can not postrocess it on the worker node
REVERT: c591d42 import the output files
REVERT: 9cbebf8 added another class for post processing of the output files
REVERT: c1406a6 another class for output file post processing
REVERT: 392efdf added class for output file that have to go to mass storage
REVERT: e8b4160 added file that will mark output file for compressing
REVERT: 32e0fda added class that will be base class for outputfile classes which will define how the output should be postprocessed
REVERT: b88430f Fixed Savannah Bug #87477
REVERT: 132e73b Making tasks openable again if in completed status
REVERT: 6b3e450 Fix to Patricks ReleaseNote stuff
REVERT: 0cc4f36 Added hook for change in master job status
REVERT: e40dac7 Adding Patricks ReleaseNote printing
REVERT: 6d0b947 Improve documentation for box object
REVERT: dbfe50f Improve documentation for box object
REVERT: 9941203 fix the parser on glite-wms-job-list-match output to recognize the CREAM CE
REVERT: 8999132 Committing on behalf of Mark Slater
REVERT: ee62dd7 fixed 'fix' not working properly on master jobs, accept panda reporting of ddm site instead of analy_xyz for output data
REVERT: 15e7a09 Fixed attribfilter for non name attribs
REVERT: 2cd9c4b Fixed Savannah bug #85332 options file check not expanding env vars
REVERT: feac9a8 fix for bug 77962 and for test case test/GPI/auto_resubmit/auto_resubmit
REVERT: 1f2a82a Minor syntax bug
REVERT: a95c1f0 Changes to allow more intuitive and efficient use of FileBuffer.
REVERT: d1aa16a 
REVERT: bff9279 
REVERT: 89551ea Bugfix #83902
REVERT: a3f2e78 Upgrading stomputil to 2.4
REVERT: e827e6b Adding username/password for connection to the ActiveMQ server. Note that these are not sensitive passwords.
REVERT: 7e669b1 Implemented Mark's changes to enable the use of environment variables in .gangarc and .ini files.
REVERT: 671f2c2 Added --no-mon option to disable the monitoring loop at runtime.
REVERT: 952e1f2 Removing redundant popen2 import; we now use subprocess.Popen()
REVERT: c87f6ec Put something useful in the job's monitoring link attribute.
REVERT: e746541 Added a username/password to the stomp connection code; without this the MSG server gets upset.
REVERT: f962414 fixed bug #81654 by moving cwd from the schema into the python object
REVERT: a464055 demoted error to debug to remove follow-on errors from partially loaded tasks. fixes #78715
REVERT: 46cafcb - bug fix in using LCGSandboxCache
REVERT: ac2dd70 locking for jobtree cleanlinks fixed
REVERT: 50e9939 Committing files on behalf of Mark Slater. Some updates to Tasks didn't make it into 5.5.26.
REVERT: 4143bd0 Quick fix to print lists of Ganga objects with few items in a more meaningful way.
REVERT: a04ef18 added releaseWriteAccess to Objects and made jobtree release its lock to fix bug #74327
REVERT: 9c2876f Merge GangaLHCb-Tasks branch 5424:5641 into trunk
REVERT: cd28f16 Messaging server renamed from gridmsg101 to two aliases (ganga.msg.cern.ch and ganga-atlas.msg.cern.ch)
REVERT: 2cf5ad7 - handling environment variables with "space" characters
REVERT: 8900943 - force check the remaining credential lifetime when delegating proxy to CE
REVERT: d3562d9 - add debug information
REVERT: 03bc19a - add debug information
REVERT: cab5d7b - add debug information
REVERT: 390ecc4 - optimize the proxy delegation to CREAM CE
REVERT: 57515e9 - optimize the proxy delegation to CREAM CE
REVERT: 2b375ba - optimize the proxy delegation to CREAM CE
REVERT: 1af389d - optimize the proxy delegation to CREAM CE
REVERT: 961d2dc - add resubmit functions
REVERT: 8739f4f - add resubmit functions
REVERT: 1150645 - add resubmit functions
REVERT: e9a300d Reverted move from tasks to core. now only in LHCbTasks branch.
REVERT: 8182abc some fixups from the tasks migration
REVERT: 515809b missing __init__.py
REVERT: 6a8467c moves TaskRegistry, Task and Transform into Core without changing any functionality
REVERT: 5db9447 minor logging fix
REVERT: 1c8aa35 using bulk resubmission for automatic master job resubmission
REVERT: af5bb72 fix auto resubmission
REVERT: 16a53b8 Bugfix for #77962; an unnecessary call to increment() was causing j.resubmit() to increment the submission counter by 2.
REVERT: 5e20804 reverted accidentally commited local changes
REVERT: 5484c77 fixed auto-resubmission algorithm: for plain jobs it takes into account MaxNumResubmits, for master jobs it takes into account MaxNumResubmits FOR INDIVIDUAL SUBJOBS
REVERT: f2dd28b more user-friendly checks for validity of arguments passed to runMonitoring()
REVERT: 91a5e3a Fix for Bug #76973; Job templates had 'new' state and could be submitted
REVERT: 36a2da7 add fix for bug: https://savannah.cern.ch/bugs/index.php?77180
REVERT: e4c40a2 remove DQ2SandboxCache.py from LCG backend - move it to GangaAtlas
REVERT: 4e95ba4 Added bulk monitoring of all jobs from a single remote machine (Bugs 76808, 76295, 76294)
REVERT: 8bbd66c fixed: SmartMerger now inherits from IMerger
REVERT: 6639cf1 added IMerger.set_outputdir_for_automerge flag as a fix for previous bugfix 76445
REVERT: 1c00422 additional check hook for backends to restrict when auto resubmit happens
REVERT: e6f8a20 fixed the flawed rasubmit condition (which otherwise would allow to auto resubmit an empty list of failed jobs)
REVERT: 4bcbaaf small modifs to auto_resubmit trigger condition + a basic test
REVERT: d9279c7 implementation of auto_resubmit() from M.Slater with some modifications from me
REVERT: 58307f4 fix for bug #76733 - set ganga version before saving the user config where ConfigError can occur if some config section is not found
REVERT: 82f765d leave the output directory to the implementation (fix for http://savannah.cern.ch/bugs/?76445)
REVERT: be86738 bugfix https://savannah.cern.ch/bugs/index.php?76520
REVERT: 481d52a use globus-url-copy (instead of uberftp) to download output sandboxes
REVERT: a019ce1 
REVERT: 4bcf8c5 
REVERT: b812dad added util module
REVERT: 47c14a4 changed name of the LCGAthenaUtil module with AthenaUtil
REVERT: 2a53a0a bug fix
REVERT: 6c381bc bug fix
REVERT: 2ac2e49 bug fix
REVERT: 3af72f7 bug fix
REVERT: 0684f89 bug fix
REVERT: ee16bb7 fixed bug in the module for backends other than LCG and CREAM
REVERT: e76b8cb bug fix
REVERT: 8954ca6 bug fix
REVERT: a41b2a8 completed and failed method made to return compelted and failed status of the job
REVERT: f5fe332 add one more module in getSandboxModules in case of Batch submission
REVERT: b7c4823 moved import tarfile block a few lines above due to import error
REVERT: 2a9f993 JobTree will now handle job lists and slices, as requsted in Bug #74531
REVERT: 14d5346 fix for backend which use default implemenation of IBackend.master_submit() but which do not support keep_going flag
REVERT: 35f717f fixed way of retrieving of panda monitoring link
REVERT: 1ece472 added util module for Panda
REVERT: 54c461b set mon links for dashboard and panda
REVERT: a0904ed removed simplejson
REVERT: f05c64d updated monitoring link
REVERT: 7221c5c mon links is a list of tuples now
REVERT: 075b7cf mon links is list of tuples now
REVERT: 30e2fbf made mon links list of tuples
REVERT: 502af16 python 2.6 not compatibility error fix
REVERT: f40672c delay the sys.path.insert call in job wrapper - fix for different import behavior of PYTHON 2.6
REVERT: 89ab08b fixed odd import problem for python 2.6
REVERT: 8c94f75 fixed odd import problem for python 2.6
REVERT: fafb56b support for keep_going=True with j.submit()
REVERT: 64d707d show an error to the user if problem occured while preparing the error report
REVERT: 1949e68 improved version of backend option for j.resubmit()
REVERT: 50cea61 implement bulk killing for a master job
REVERT: 4fff29b added a possibility to change backend with j.resubmit()
REVERT: 1c0ab93 fixed typo
REVERT: 8298bac bug #74402: forcing into completed from failed does not work
REVERT: 46e388f bug #73328: Make Job.submit args configurable
REVERT: 46b8f9b fixes
REVERT: 4135de6 file added
REVERT: 8d93256 file added
REVERT: 13aebf5 file added
REVERT: 03305bb removed method
REVERT: f7907e4 set default middleware flavor from EDG to GLITE
REVERT: 9d0232a fixes
REVERT: 8fbb2f6 fixes
REVERT: e97eafc fixes
REVERT: 97d3523 fixes
REVERT: 755f928 fixes
REVERT: 5d0aa81 fixes
REVERT: f9a9886 fixes
REVERT: 01ddea3 fixes
REVERT: 35db314 fix
REVERT: e72ddce fixes
REVERT: 6f81be2 fixes
REVERT: 276da86 fixes
REVERT: 2bb28f0 tests
REVERT: 91a898f tests
REVERT: e5acbb2 added parameter to a function
REVERT: e49a576 pass parameter to a function
REVERT: c6f7767 file added
REVERT: 923ad1b file added
REVERT: 1351e38 file added
REVERT: cdcae84 file removed
REVERT: 8ef1b01 file removed
REVERT: c4ed8be fixes
REVERT: 142d090 fixes for generic backend MS
REVERT: c1c7c1b tests
REVERT: 3ba8dbd tests
REVERT: 168f4a9 tests
REVERT: c9b09c1 tests
REVERT: a98488b tests
REVERT: cd64e22 tests
REVERT: bb08696 added file for test
REVERT: e5f5e83 added file for test
REVERT: 67e9927 fixes
REVERT: bbb5634 bug fix
REVERT: 60f4a98 test
REVERT: 8fcf4e1 test
REVERT: 9172bb1 test
REVERT: 217cb33 test
REVERT: 5922311 added for test
REVERT: b88d9b7 this module should handle monitoring of all backends for Athena application
REVERT: 25ea229 this module should handle monitoring of all backends
REVERT: 3a8d8eb include in the message that is sent the runtime packages so that we can get the experiment
REVERT: e5d1684 place each mon link in a div, so that in the UI they apper one under another in the links column
REVERT: af94d2b bug fix for #74231
REVERT: 466c3a8 some warnings are made infos because they are not that important
REVERT: 3d06149 make heartbeat frenquence configurable
REVERT: 9f8d98e bug fix for bug #73416 - job slicing was wrongly documentated
REVERT: a0e5239 Added online help to jobSlice GPI function
REVERT: f58cce0 JobInfo property renamed from links to monitoring_links
REVERT: ae108f8 JobInfo property renamed from links to monitoring_links
REVERT: 4201078 JobInfo property renamed from links to monitoring_links
REVERT: 6baf230 fixed bug when there are no compelted jobs - the JSON creation for the chart succedded jobs in time failed
REVERT: d2287f7 added links property to the JobInfo schema that will be list of monitoring links
REVERT: ae74650 overrided submit method and added two monitoring links - this is for test
REVERT: 09c5d49 added monitoring links - extracted from job.info object, added test JSON
REVERT: e7837f3 upload the report if it is less then 100 MB, otherwise print message that it is too big and it was not uploaded
REVERT: 0cc24d0 print to the user the most important info that is collected in the tarball
REVERT: de46fb3 get the submission queue from configuration
REVERT: 769b286 added spyware infroamtion of submit and resubmit of jobs
REVERT: 2988bb9 added config setting for the name of the queue where job submission mesasges are sent
REVERT: e05523f added function for sending messages to msg queues when job is submitted
REVERT: f45db86 cleanup of timestamps and updateStatus
REVERT: fb359d4 forgot to make jobSlice function use an ordered dict instead of an regular dict
REVERT: 09af545 Added jobSlice function that makes a slice from a job list
REVERT: b18fda3 added methods for returning another google charts
REVERT: 5fd9e27 added colors in the returned json for pie charts for statuses, the colors of the pie charts should be the same as the status column colors
REVERT: 1222960 load all jobs at begining, not only for the last month, capture the messages that are coming from the http server and log them as ganga debug messages
REVERT: f30ce24 refactoring and added functionality for returning json prepared for drawing pie charts for jobs and subjobs for status, backend and application
REVERT: 211810b added support for numeric arguments in executable application, fix for bug #71343
REVERT: 3d75d05 get fully qualified id instead of id, bug fix for bug #70758
REVERT: 5943852 get fully qualified id instead of id, bug fix for bug #70758
REVERT: 01c5b53 fixed bug #69216: .ganga.log surprising behaviour
REVERT: 154befa fix: bug #68896: ganga - glite-wms-job-list-match fails - reports full output is in file ... - but file does not exist
REVERT: 1b60151 patched by S.Schmitt: bug #70712: SGE() should have information about job's worker node
REVERT: 4401d34 gather info for the pie charts for jobs/subjobs statuses,application and backend  that will be displayed, return this info as a JSON
REVERT: 3cc0ae0 refactoring and preparing data for pie charts that will be shown in ganga --webgui
REVERT: 76dd3fa print of the jobs added in the tarball
REVERT: a9b00db fix for bug #67690 - include Ganga. at the begining of the logger name so that it can inherit from Ganga logger
REVERT: b64ad82 bug #70796: Message after force_status not illustrative - My status completed completed
REVERT: 83fe85d path to ipython hsitory file is different now -> savannah bug #71912
REVERT: ae020cd Fixed typos in psychiatrist
REVERT: 65f109c add debug message to check how many output downloading agents are still alive in background
REVERT: 42d0d08 bug #71563: DeprecationWarnings in Core when run with python 2.6
REVERT: 453e154 bug #71912: Use custom ipython directory to save ganga history and configuration
REVERT: 5f20b61 fix: bug #71512: BackendError is not propagated to the user
REVERT: 77222ee fixed bug 44116: app.postprocess() may now raise PostprocessStatusUpdate('failed') exception
REVERT: 5b94966 added extended attributes to usage monitoring spyware and more configuration flexibility
REVERT: 11fd2d9 change config of DashboardMS: use production server gridmsg101 and rename topic names to better describe what they are
REVERT: 6d62edd 1) extra check (via postBootstrapHook()) for outdated config files for Atlas monitoring dashboard. 2) added loading of DashboardMS config section with other core plugins
REVERT: 54bd4bc fixing bug #70849 - when removing a template it was saying removing a job ..
REVERT: 5d1417e fix bug #53939 , first set the id of the subjob and then append to self.subjobs
REVERT: 9ec28ea reverse the order of arguments subjobs and sum_outputdir, fix for bugs #71333 and #71376
REVERT: 2fdf496 fix the bug in getting md5 checksum
REVERT: c75db7e some fixes for the report function to handle also subjob as argument
REVERT: 1e18e8d fix import typo
REVERT: a29fa6a replaced dummy user in the list with the user that has started ganga
REVERT: 658c0a6 remove license note
REVERT: 043bf9e  - add additional module "Compatibility.py" to handle python 2.6 compatibility issues  - replace "md5" with "hashlib" if python version larger than 2.5
REVERT: 0e9521a remove useless imports
REVERT: 8fd494e added gangarc file - the default configuration in the report
REVERT: 90a278f Fixed typo (bug #70750)
REVERT: ffa65ad fixed removing incomplete jobs
REVERT: 5669ff6  - fill CE attribute value for all subjobs  - add automatic CE selection if the getce method in the requirement is provided
REVERT: 9ddaa6f Job object was not recognised
REVERT: 96f8c20 set time period to lastMonth by default
REVERT: 19c0419 fixed returned link to return the local user, made jobs that have TestSubmitter backend for example to show in the page -> previously we catched an Error and skipped this jobs
REVERT: e62cebb fixed returned path to page, added exception handling for attributeError and registryError caused by testSubmitter backend
REVERT: d7920e2 bugfix #69715: traceback "raise Error" in WNSandbox.py
REVERT: 3194475 bugfix #69715: traceback "raise Error" in WNSandbox.py
REVERT: a14b3ed untabify
REVERT: 02c990b CREAM handler added
REVERT: 101b192 returned link to the page from where the jobs can be monitored, fixed bug when exiting the session whit ctrl+D
REVERT: 7de38c1 added a new hook postprocess_failed() which is only called from completing->failed, this avoids recursion with exisiting postprocess() hook and will allow to separate the two functionalities more cleanly
REVERT: 41b3018 web-based monitoring GUI
REVERT: 4aeacf0 bug #68547: A function to monitor some jobs by hand?
REVERT: 53694b1 bug #53751: Release reported wrong when checking out from SVN
REVERT: 4944f8e bug #55503: Jobs should not be able to be removed in completing status; bug #55952: postprocess_hook never called for failed jobs; bug #64514: ApplicationConfigurationError triggers WARNING instead of ERROR -level messages; bug #67876: Re-running the merger should be more straightforward
REVERT: 9d878b6 bug #52815: Color customisation for jobs display
REVERT: 4cd453e enabling CREAM backend in LCG module
REVERT: f586eff purge the jobs on CREAM CE if output are successfully fetched locally
REVERT: ae85652 full implementation of GridftpSandboxCache
REVERT: a280eb2 add support for parallel job submission
REVERT: 92e935a fix md5sum calculation
REVERT: cd2228f add sandbox cache module for gsiftp protocol
REVERT: ffdb2de patch
REVERT: 7d9f04b patch
REVERT: 35efbfa patch
REVERT: fe6f43b patch
REVERT: 730664a patch
REVERT: 536be34 patch
REVERT: 12cf5bb patch
REVERT: 34804fb add proxy delegation for CREAM jobs
REVERT: 7a3c23a patch
REVERT: 4e84b65 patch
REVERT: 8941efa patch
REVERT: adddf30 patch
REVERT: f74a4ed handling oversized inputsandbox
REVERT: 0c2557a update schema
REVERT: b5f99b4 introduce CREAM CE direct job submssion (preliminary test only)
REVERT: bb03f30 introduce config.LCG.StatusPollingTimeout to avoid blocking monitoring thread if glite-wms-job-status call is stuck.  The default timeout is set to 5 minutes.
REVERT: 78c5d00 disable the out-dated Octopus monitoring service
REVERT: 017c6b8 fixed bug #68800
REVERT: 9d5cf45 fixed bug #68535 make check for the type of the parameter passed to the report function
REVERT: 17d3278 Backend now caches passwords for keys Setup now only opens the transport on startup
REVERT: a2b982b Fixed bug #68317
REVERT: 2d125b5 reverted to an older (patched) version of ipython
REVERT: 98cdd2f Added polling framework to Registry/Repository
REVERT: 92af0bb - remove unused import: tempfile
REVERT: 925234f Only kill other session locks if monitoring loop is active
REVERT: b5bb6ca Added check for monitoring_component.enabled in Tasks loop and Session cleanup loop
REVERT: 00e881e Reformat taskname attribute of messages. Update task_meta and job_meta message.
REVERT: 103fcab - apply timeout feature in calling glite-wms-job-submit command - submission timeout is configurable via config.LCG.SubmissionTimeout.
REVERT: e13c9be bug #67938: LCG backend blocking during submission ?
REVERT: bcd3aa6 fixed: [bug #68006] Submit counter broken for subjobs
REVERT: ec5210f Use MSGUtil in UsageMonitoring as intended instead of MSGMS.
REVERT: e249b75 rollback to previous rev 3446
REVERT: 9570d6c applying new user dataset naming convention for dataset created by DQ2SandboxCache
REVERT: 8a1dfe0 ignore stdout/err from sourcing dq2 client setup script
REVERT: 738a0d2 Update stomputil to version 2.3.
REVERT: 494fbce ignore stdout/err while sourcing the dq2 setup script
REVERT: 964e812 Fix bug #67799: MSG traceback during job submission (Check for undefined inputdata / outputdata)
REVERT: 3902afe Fixed bug - was loading from backup instead of properly failing like it should be
REVERT: 26dea68 full_print function was not recognized - added import for that
REVERT: 495e219 do not get the whole IPython history, just the last 20 commands
REVERT: e367f18 user config was not getting the changed values, but the default ones
REVERT: 5bf6649 fixed upload server path
REVERT: fbc1c18 store the archive in tmp folder of the mashine, delete the tmp archive always - even on error occured, also delete the temp upload file that is internally created when uploading to a django server
REVERT: 0bb39b0 syntax fix
REVERT: 73cf979 fixed bug #66804: subjob resubmitting error: cannot compare datetime.datetime to lis
REVERT: 8400898 filename fix
REVERT: 79d25d0 integrated feedback report into GPI
REVERT: 80cd573 some work for Ivan
REVERT: 33a237c Separate out meta-data utility methods. Code formatting.
REVERT: 668f2b9 changed server upload path
REVERT: 268100e upload error report file on a server
REVERT: dad8b6d Move logger creation into MSGUtil
REVERT: b34465c Upgrade to stomputil 2.2
REVERT: c71f388 * fix bug #64041 - subjobs can now also be deleted in the repository
REVERT: d8575c2 report returns the physical path, not uploading to a server
REVERT: b4f3985 fixed bug #64512
REVERT: 7ce1756 Add more debug logging in shutdown method
REVERT: 6ba168d * try loading from backup file 'data~' if loading from 'data' fails * added IncompleteObject for cases where one wants to reload or remove objects where load failed
REVERT: 9c3087c * make 'select' available to all RegistrySlices * add incomplete_ids() to RegistrySliceProxy
REVERT: 1c335e0 * fixed jobtree.getjobs() * display now uses job slice display
REVERT: cf1e7ca added uploading of the error report to a server, fixed name of the archive
REVERT: 5ec82a0 fixed logger messaged: job id -> fqid
REVERT: 84c39d8 fix in the heartbeat
REVERT: abdbe89 test cleanup
REVERT: 4908c15 Second part of fix for bug #65797
REVERT: b96a401 Fixed bug #65797
REVERT: d71979e added heartbeat to fix an AFS problem: 'touch' is not propagated correctly anymore across multiple AFS clients at CERN
REVERT: 3eb7ea5 Pass DashboardMS logger to Publisher
REVERT: fba4c42 ipython setup: sys.path->PYTHONPATH
REVERT: b8ad108 bugfix 55568
REVERT: b2409b7 conditional transition message from 5.4 -> 5.5, will be disabled after some time
REVERT: 797107a move to ipython 0.8.4
REVERT: f3d89c3 Fix bug #65444: config['MonitoringServices']['Athena/LCG']=None breaks job submission
REVERT: 7fc08e9 converted tabs to spaces
REVERT: 5614461 this is my first commit blah...
REVERT: d7d3d9a Added GangaValueError for proper error reporting if wrong constructor argument
REVERT: 92ccf16 Make Job(JobTemplate()) and JobTemplate(Job()) work (allow down-copy and up-copy in class hierarchy)
REVERT: 6e89ca7 Changed TypeError to TypeMismatchError (GangaError)
REVERT: d59e537 Introduced __construct__ function to GangaObject. Constructor positional args are passed to this, by default this implements a copy constructor If a GangaObjects want to implement a constructor, it should implement this function, check if the args match and if not call the superclass __construct__
REVERT: adcc8ba make a new configuration variable: IgnoreGliteScriptHeader to turn on/off the trick for 32/64 bit library compatibility issue.
REVERT: c8594a3 minor fix
REVERT: f3c2535 print JDL when no matched resource.
REVERT: 22787ae fix 32/64 bit issue with glite-wms-job-status and glite-wms-job-logging-info (script-based) commands
REVERT: 2fe0bee bugfix #31690: Empty the outputdir of the subjob just before resubmitting it
REVERT: 1470ce0 bugfix #63157
REVERT: 50041a3 bugfix: #31690: Empty the outputdir of the subjob just before resubmitting it
REVERT: eb8fe58 bug #55568, ganga -o option does not accept =
REVERT: 5e4deb0 bugfix - bugfix #64081
REVERT: e5bb62e bugfix #59165
REVERT: 48d8512 Add a global flush in Batch job wrapper scripts (in some situations the status of a Batch job was not updated until the statusfile was closed at the end of the job's execution). The flushing function looks now as follows:
REVERT: ed34fe4 Fix bug #62543: Exception in thread GANGA_Update_Thread_shutdown (Avoid accessing globals between sleep and exit tests in shutdown and stomputil daemon threads)
REVERT: 4c75b0a Stopped continuous retries with lots of jobs (bug #58527)
REVERT: 28bf244 change logic of activating monitoring plugins: use all plugins which match all patterns defined in the config, rather a first pattern only
REVERT: 4c9f576 Fix bug #64080
REVERT: f61249e Fix bug #64080 Check for incomplete XML file
REVERT: 87d869d change the default dq2 client setup path
REVERT: e13d9a2 Fixed wrong behaviour for 'subjobs' fields without Ganga objects Fixed AnaTaskSplitterJob losing information
REVERT: 8e91926 remove lcg-infosite query when initializing Grid object, call it only when LFC_HOST is needed and not presented in user's shell environment
REVERT: 5917d3f update lcg-cr command (replace -T to -D in SE type specification)
REVERT: 1eee796 - skip calling lcg-infosites when LFC_HOST is already presented in user's shell environment - reduce the number of credential.voms query in job submission routine - remove all glite command log after the system call is finished
REVERT: f809f3f add debug messages
REVERT: 65989b5 Fixed subjobs not being marked as "clean" after flushing
REVERT: 2c9e5ad 
REVERT: fd78221 to be replaced by the XML branch verison 5.5.0
REVERT: 3863afa Add pypath and pyldpath in front of dq2-get calls
REVERT: 75e8b83 Split lcg-getturls call in chunks of 50 for DPM
REVERT: 68029cd Remove dq2clients requirments
REVERT: cbd0272 Remove sqlite download
REVERT: bd576a4 Update to 0.1.32, user10
REVERT: 9cb5cc2 Change order of glite UI and athena setup
REVERT: cc14cd1 Don't use local cmt_config. Get version dependent string from AthenaUtils.
REVERT: 385afd0 fix typo
REVERT: 4eacf39 use the libds cloud if libds supplied
REVERT: 39f9efa disable outputdata.location. Refer user to DaTRI.
REVERT: 9189d80 Fix 4 digit release brokering for Athena and AthenaMC
REVERT: 1994fa0 Minor bug fixes for stupid errors introduced previously
REVERT: 08cd451 Automatically updated template for 5.5.0
REVERT: ed2660c Added PACKAGES-5.5.0
REVERT: 24f2efe Added ReleaseNotes-5.5.0
REVERT: 31e3b77 Fix AtlasProduction tag problem
REVERT: 8b3c39c Automatically updated template for 5.5.0
REVERT: dca2bfd Automatically updated template for 5.5.0
REVERT: 1d6baae Automatically updated template for 5.5.0
REVERT: 24e981e Added PACKAGES-5.5.0
REVERT: 1a2f653 Added PACKAGES-5.5.0
REVERT: d971f2c Added ReleaseNotes-5.5.0
REVERT: 61efae0 Fix PhysicsAnalysis submission tag
REVERT: 5f6e227 make accessmode unhidden to work with GR
REVERT: 340d057 fix for https://savannah.cern.ch/bugs/?62031 add hidden accessmode for FILE_STAGER
REVERT: c26cd6e * Added GRL support to AMIDataset * Removed bad combinations of grid/local options * Made 'ALL' the default for the cloud in LCG * Fixed TAG bug - now TAG files are downloaded on the WN
REVERT: bcf54fe Merge AMIDataset to trunk
REVERT: f72d988 Automatically updated template for 5.5.0
REVERT: 7dc282e Automatically updated template for 5.5.0
REVERT: 5a3274e Automatically updated template for 5.5.0
REVERT: 94c0c7a Added PACKAGES-5.5.0
REVERT: 74a782d Added ReleaseNotes-5.5.0
REVERT: 4473f71 Added PACKAGES-5.5.0
REVERT: 2681841 Automatically updated template for 5.5.0
REVERT: c650e36 Automatically updated template for 5.5.0
REVERT: 78f2073 Automatically updated template for 5.5.0
REVERT: d762e79 Added PACKAGES-5.5.0
REVERT: 899dc17 Added ReleaseNotes-5.5.0
REVERT: 70d5dce Update externals
REVERT: e8ed857 Change log message
REVERT: 8f9b7b9 Modified JEM startup banner
REVERT: 72063c9 Code documentation
REVERT: cf42106 Update to panda-client 0.2.12
REVERT: 37afbd0 Fixes for stomp data transfer, edited default configuration values, bumped referenced JEM version
REVERT: bfa1520 * Add doc strings to critical property methods
REVERT: bd0bee0 Bumped referenced JEM version to 0.2.7. This version includes the new StompValve to use stomp (ActiveMQ/JMS, gridmsg-Server) for data transfer.
REVERT: c9db6b2 * set stomp.py default logging level to CRITICAL
REVERT: 45e24b9 * Remove debug print for batch shutdown
REVERT: 74c7480 * Set STOMP publisher thread as non-critical
REVERT: 6767821 * Add non-critical flag to GangaThread. * Prevent Ganga shutdown hanging for non-critical threads.
REVERT: 4c3adba Add AtlasLCGRequirements.list_release_blacklist() and black-list release warning in AthenaLCGRTHandler.py
REVERT: 6b418bd 
REVERT: c87d696 move pandatools.Client import into functions that use it. This prevents ganga startup blockage if the panda server is down.
REVERT: 50d1419 Add clean_duplicates_in_dataset routine to DQ2OutputDataset
REVERT: 79178fa Automatically updated template for 5.5.0
REVERT: b8f059a Automatically updated template for 5.5.0
REVERT: 7a9c2b0 Automatically updated template for 5.5.0
REVERT: 9f92abe Added PACKAGES-5.5.0
REVERT: 1ff27e0 Added ReleaseNotes-5.5.0
REVERT: fe8499e Automatically updated template for 5.4.5
REVERT: 7ffa8b3 Added PACKAGES-5.4.5
REVERT: 38251ce Added ReleaseNotes-5.4.5
REVERT: e704d6f Fix 32/64 bit setup and pybin problem for slc5
REVERT: 2c74807 Fix 32/64 bit setup and pybin problem for slc5
REVERT: fe8308b send cmtConfig with jobs
REVERT: 19d250d Automatically updated template for 5.4.5
REVERT: 141184f Added PACKAGES-5.4.5
REVERT: 6ed8056 Added ReleaseNotes-5.4.5
REVERT: fedc6fc Automatically updated template for 5.4.4
REVERT: 9616429 Automatically updated template for 5.4.4
REVERT: f6e0664 Automatically updated template for 5.4.4
REVERT: 84447f7 Added PACKAGES-5.4.4
REVERT: beb2b39 Added ReleaseNotes-5.4.4
REVERT: c127094 Add missing DQDatasetExistsException exception
REVERT: cf5a21a add overall dataset size to info log
REVERT: cf745db handle the unlimited filesize case
REVERT: 191fd6f Changed splitting algorithm to better respect the filesize constraint:
REVERT: 8b112ab Add option for WLCG formatter to include microseconds in date format
REVERT: 167c4f9 Fix cmtconfig SL5 problem
REVERT: 3dc087e Fix cmtconfig SL5 problem
REVERT: 8c44697 Fix return value of DQ2Dataset.get_content(size=True)
REVERT: 34128db Automatically updated template for 5.5.0
REVERT: 7e912fa Automatically updated template for 5.5.0
REVERT: 3321c95 Automatically updated template for 5.5.0
REVERT: 3c1bdd1 Added PACKAGES-5.5.0
REVERT: f784622 Added ReleaseNotes-5.5.0
REVERT: 7ce977e removed debug pritnout
REVERT: 49884cc Automatically updated template for 5.4.4
REVERT: ea8eefb Automatically updated template for 5.4.4
REVERT: 1a8d6fa Automatically updated template for 5.4.4
REVERT: 45b7a5e Added PACKAGES-5.4.4
REVERT: abc2e21 Added ReleaseNotes-5.4.4
REVERT: cd0d90e looking for all input files instead of files with 'root' suffix in the given local directory (e.g. the data09 AOD doesn't have .root extension)
REVERT: 7df15aa bug fix in generating the input.py
REVERT: 76d3aa8 
REVERT: cc2fc6a added MSG-based usage monitoring
REVERT: a95f25b Minor fix creating temp dir
REVERT: d790cc7 Change preJobO code to always prepend
REVERT: 991c8e0 Automatically updated template for 5.4.4
REVERT: 34b9138 Automatically updated template for 5.4.4
REVERT: 2747049 Added PACKAGES-5.4.4
REVERT: da7511d Added ReleaseNotes-5.4.4
REVERT: f9b7cbb bump external JEM version
REVERT: 4f84a43 Added PACKAGES-5.4.4
REVERT: dd2bae7 Added ReleaseNotes-5.4.4
REVERT: 02517f4 
REVERT: 193d6e8 
REVERT: aa91fa3 
REVERT: e2353f4 
REVERT: bdc80b9 Fix DPM workflow
REVERT: 35f4409 * Local, Batch backends: Enable job statistics in Athena.stats field * Local, Batch backends: Fix DQ2 output dataset creation workflow * Local, Batch backends: Fix setting of config.DQ2.DQ2_LOCAL_SITE_ID * Updates for InputFilePeeker/AutoConfiguration in   Athena/LCG/Local/Batch:   - search jobOptions if InputFilePeeker/AutoConfiguration is configured   - use preJobO.py and prepend it to jobOptions   - automatically works for DQ2_LOCAL, FILE_STAGER and DQ2_COPY   - Work-around for rfio at DPM sites: uses short TURL since Athena     bug (http://savannah.cern.ch/bugs/?59253) is not solved yet.
REVERT: 5dec6b0 add more sophisticated event loop specification in FileStager_jobOption.py for different use cases
REVERT: b3cfbd8 merge the latest change in GangaAtlas-AMAAthena-new-workflow
REVERT: ebd4dc3 Remove creation of datasets and set DQ2_LOCAL_SITE_ID
REVERT: 6db88e7 add subroutine for download DBRelease tarball
REVERT: 79854cc Added 'tail -f'-like live peeking to GangaJEM. Usage: j.livePeek(). This still is an experimental feature.
REVERT: f573dbd Fixed: JEM IMonitoringService callbacks are run even if JEM is globally disabled; this may lead to debug-noise like stacktraces.
REVERT: f7b14e2 Automatically updated template for 5.5.0
REVERT: 993e634 Added PACKAGES-5.5.0
REVERT: 985f078 Added ReleaseNotes-5.5.0
REVERT: 5dd8f65 
REVERT: d14c1f6 Fix to support inputFilePeeker
REVERT: 5dec396 Removed obsolete status vars
REVERT: 5e66db3 bugfix #58196
REVERT: 11d1e5e 
REVERT: 22f18d5 
REVERT: d0f1aa6 remove debug codes
REVERT: 66bbb3e Automatically updated template for 5.5.0
REVERT: 3a7d71e Automatically updated template for 5.5.0
REVERT: 5088472 Automatically updated template for 5.5.0
REVERT: 8e890c2 Added PACKAGES-5.5.0
REVERT: e5764ff Added ReleaseNotes-5.5.0
REVERT: ed1a986 Bumped referred external JEM version. Minor debug output addition.
REVERT: de590e3 Added PACKAGES-5.5.0
REVERT: b97d828 Added ReleaseNotes-5.5.0
REVERT: 517c4c7 new workflow for setting/preparing AMAAthena application:  - specify framework-level AMA job option files by setting flags, e.g. j.application.job_option_flags = ['DATA', 'TRIG']  - convert AMA configuration into user-level job options on the client side
REVERT: 24a9c1a add necessary changes to adopt the revised StagerDataset
REVERT: f615f36 rewriting the StagerDataset workflow:  - using the same code made for DQ2Dataset(type='FILE_STAGER')  - adding the workaround to work with inputFilePeeker
REVERT: 437f294 include AMAAthenaTask application in the new DQ2OutputDataset workflow
REVERT: 01ecedc adopts the common subroutines: filestager_setup and frontier_setup
REVERT: c459582 introduce two common subroutines: filestager_setup and frontier_setup
REVERT: a10ed11 bugfix in the function for making external system call
REVERT: 6a842f7 Automatically updated template for 5.4.3
REVERT: 71fbae9 Added PACKAGES-5.4.3
REVERT: 19e5fde Another fix for the TAG stuff. This time, it'll hopefully work!
REVERT: 05163dc Automatically updated template for 5.4.3
REVERT: 3ec07a8 Added PACKAGES-5.4.3
REVERT: 7c0cc97 Added ReleaseNotes-5.4.3
REVERT: 7443796 Added PACKAGES-5.4.3
REVERT: e2304f1 Added ReleaseNotes-5.4.3
REVERT: 83f0ed5 Added PACKAGES-5.4.3
REVERT: ce6ba38 Added ReleaseNotes-5.4.3
REVERT: c71289b show transExitCode
REVERT: c07846a Minor fix to stop crash for non-DQ2Datasets
REVERT: f8cc88e enhance the code for generating FileStager job options
REVERT: 0d52759 Automatically updated template for 5.4.3
REVERT: 6b5823f Added PACKAGES-5.4.3
REVERT: 00d4385 Added ReleaseNotes-5.4.3
REVERT: b309de7 Formatting fix
REVERT: 09ff098 remove unused function
REVERT: 4a62e92 Fix to properly use setup the Executable path on the remote machine
REVERT: b57d203 Minor fix to stop ALL option selecting T0
REVERT: 570abd9 Additional changes for new TAG prepare method
REVERT: b4de3a4 
REVERT: eca85a4 Add AMAAthena to DQ2OutputDataset fill and fix outputsandbox problem in fill
REVERT: db2b0c8 Fix bug #59849
REVERT: 368ef2d Fix for inputdata and StagerDataset
REVERT: efb2d63 Fix for older lcg-getturl versions and add fail-over for empty usedProtocol
REVERT: dbbab19 Remove dataset creation in AthenaLCGRTHandler.py
REVERT: 3102db9 Further hot fixes for JEMs splitjob capability
REVERT: 3282c2a Increased JobExecutionMonitorMS error verbosity for debugging purposes.
REVERT: 41affd7 workaround for */* issue. changed default listener PID to zero.
REVERT: 7a5c932 Automatically updated template for 5.4.2
REVERT: f6110d0 Added PACKAGES-5.4.2
REVERT: 8aa7ef8 Added ReleaseNotes-5.4.2
REVERT: 8fd5420 - Fixed handling of user-defined DBReleases stored in lfc with new path - Fixed removal of old files in srm when using resubmit()
REVERT: d5b54b4 
REVERT: fddb9ee Fix namespace issue in JEMs service handler for splitjobs.
REVERT: ed3626b BugFix for latest ROOT versions from Manosh
REVERT: e561e41 Automatically updated template for 5.4.2
REVERT: b44ce48 Added PACKAGES-5.4.2
REVERT: 8dc5afd Added ReleaseNotes-5.4.2
REVERT: 0fd5bdc fix for ipython 0.8.4+
REVERT: 9ad0085 fixed root arg bug
REVERT: 4c5df77 Added AthenaTask to Athena name at all occurrences
REVERT: 2b07208 fixed dirac root macro args bug
REVERT: 6874e4c Caching module from Manoj Jha
REVERT: 55877ec introduce new objects for running AMAAthena analysis with GangaTask
REVERT: 9578d35 small fix for catching the exitcode correctly from athena process
REVERT: 52d18fc Merge GangaAtlas-21-32-dq2 branch to trunk
REVERT: f2dcbc9 Revert rev 2743 - all code is on GangaAtlas-21-32-dq2 branch
REVERT: 73f8205 Revert rev 2743 - all code is on GangaAtlas-21-32-dq2 branch
REVERT: 4456c27 Added PACKAGES-5.4.2
REVERT: 8f26e15 Added ReleaseNotes-5.4.2
REVERT: 15f1bcf New libdcap.so with improved read-ahead buffer tuning
REVERT: 1d5dba2 Merge GangaAtlas-athena-autoconf to trunk
REVERT: d3b8673 fixed ipython args
REVERT: 13bda34 fixed monitoring warnings
REVERT: 297ec74 Fixed empty dataset bug
REVERT: 47b5a2e * Remove unused variable
REVERT: 8e156a3 * Documentation * Add second task-meta message on complete to provide additional data * Do not send job-processing-attributes message if job failed
REVERT: ea12888 
REVERT: 32b093a Bumped referenced external JEM version to 0.2.5.3
REVERT: 386a8b5 use extOutFile to specify AMA specific summary tarball
REVERT: d4938b1 Updated installation section of LHCb manual
REVERT: 904ac27 * Add job meta message
REVERT: d511e88 * Add job processing attributes message
REVERT: 8937ee0 * Update task meta message
REVERT: 2366329 * Add task meta message
REVERT: d5f3116 * Use singleton publisher. * Pull out LCGMS superclass for LCGAthenaMS. * Separate meta-data builders out into utility modules.
REVERT: 27e2e01 Update to maintain unchanged behaviour following removal of overloading of complete callback in IMonitoringService.
REVERT: 1f2f9b0 IMonitoringService: Remove overloading of complete callback. Add fail and kill callbacks.
REVERT: 15c5586 D.Tuckett:IMonitoringService: Remove overloading of complete callback. Add fail and kill callbacks.
REVERT: a44b1ae D.Tuckett: IMonitoringService: Remove overloading of complete callback. Add fail and kill callbacks
REVERT: 9bc3341 putting GangaSAGA on TRUNK for further development by Ole
REVERT: 5e32bf3 Begun working on https://svn.grid.uni-wuppertal.de/trac/JEM/ticket/20 - this is mostly needed for a nicer batch run possibility in GangaJEM.
REVERT: da0a1f2 Made JEM work area path-independent (JEM ticket #18)
REVERT: c51d906 Automatically updated template for 5.4.1
REVERT: 61f0255 Added PACKAGES-5.4.1
REVERT: 4b66979 Added ReleaseNotes-5.4.1
REVERT: 1f8b6cc Revert to module level singleton publisher to avoid creating excessive threads
REVERT: e8ffc30 Fixed extend method in LHCbDataset
REVERT: ffc2f9f Automatically updated template for 5.4.1
REVERT: 4857107 Added PACKAGES-5.4.1
REVERT: 18460df Added ReleaseNotes-5.4.1
REVERT: d38e17a Minor JEM stuff (updated website link, bumped referred JEM version)
REVERT: cb71833 Automatically updated template for 5.4.1
REVERT: efa0f53 Added PACKAGES-5.4.1
REVERT: a0fbdc3 Added ReleaseNotes-5.4.1
REVERT: 521610e Fixed bug in data extraction from opts file
REVERT: ab98e72 New extended dq2tools
REVERT: 74b882e Automatically updated template for 5.4.1
REVERT: b07d0ec Major rewrite of data handling of Athena/DQ2/LCG/Local * Clean-up and new workflows in ganga-stange-in-out-dq2.py   - New host detection algorithm based on DQ2_LOCAL_SITE_ID   - Move all SE protocol detection to ToA.py file   - Improve TURL generation algorithm and use bulk methods     and more retries   - Register output files already on worker node in DQ2 * Changes in DQ2OutputDataset   - change DQ2OutputDataset in Athena/DQ2/LCG/Local workflow     to register files already on worker node in DQ2   - Output datasets are now based on sites and dataset names     get space token name attached.   - If a master job is split among several sites there are output     dataset set per site created. These output datasets are attached     to a container at the end of the master job   - output datasets are frozen at the end of the master job   - No emptry dataset is created in AthenaLCGRTHandler or     AthenaLocalRTHandler   - config.DQ2Dataset.DQ2_LOCAL_SITE_ID should be set for a Local     or Batch backend job
REVERT: baf1d92 Added PACKAGES-5.4.1
REVERT: 12d87bf Added ReleaseNotes-5.4.1
REVERT: 9592243 additional checks when caching proxy information
REVERT: 0f0a1b2 Dashboard monitoring services based on MSG. N.B. This code is under development and should not generally be used or relied upon.
REVERT: b628575 avoid downloading output sandbox of removed and killed jobs.
REVERT: ced51b5 load panda Client late to speedup start
REVERT: 75f15f4 fix
REVERT: 3550a84 delat panda client loading
REVERT: 7ee8ae9 unique name for test
REVERT: 2bf3dd3 load panda Client.py in the functions (faster startup)
REVERT: 0927124 fill cloud if None from PsubUtils
REVERT: 0053da5 add usage note inline for online help
REVERT: 5dd1cbd restrict "DQ2" mode (copying file remotely with lcg-cp) to be used with local storage
REVERT: b76eccc check proxy validity before updating information cache
REVERT: 0c585d0 Automatically updated template for 5.4.1
REVERT: 2ce7a78 Automatically updated template for 5.4.1
REVERT: 1c0ff76 Automatically updated template for 5.4.1
REVERT: 6fb8bf7 Added PACKAGES-5.4.1
REVERT: c88205f Added ReleaseNotes-5.4.1
REVERT: 8e20c0e 
REVERT: 76cd5fa 
REVERT: fdf6d50 add xrootd support
REVERT: 95e5225 get default cloud from cert
REVERT: d3b90a9 add support for StagerDataset
REVERT: 6574390 Added caching of credential information
REVERT: 019189c remove StagerDataset/StagerJobSplitter from AMAAthena plugin
REVERT: f9b57bc introduce StagerDataset and it's corresponding splitter, StagerJobSplitter.
REVERT: 815803a Automatically updated template for 5.5.0
REVERT: 20fb725 Added PACKAGES-5.5.0
REVERT: 555df69 Added ReleaseNotes-5.5.0
REVERT: ba8dca2 Added PACKAGES-5.5.0
REVERT: 6bcd58e Added ReleaseNotes-5.5.0
REVERT: 1a18e54 Make MSGMS default configuration publish to /queue/ganga.status instead of /topic/ganga.status
REVERT: bc52a92 
REVERT: c1b25f5 Remove Athena MSGMS
REVERT: beba5bb fixes to resubmit function
REVERT: d4b4a2c Minor cloud=ALL bug fix for non-DQ2OutputDataset jobs
REVERT: 4c37de2 Use IMonitoringService support for automatically copying configuration dictionary to worker node. Extract reusable code to MSGUtil.
REVERT: 8f4e103 
REVERT: b59e3c4 
REVERT: 3b952d1 
REVERT: 42d83ea [by David Tuckett]
REVERT: 5dfd987 fixed cmt bug
REVERT: 0dada3a Fix for bug #58122: rename instance variable "complete" to "_complete" to avoid shadowing new method "complete" in IMonitoringService.
REVERT: 3c2b5c1 Automatically updated template for 5.5.0-beta
REVERT: b1f8f1c Automatically updated template for 5.5.0-beta
REVERT: 6defe3d Added PACKAGES-5.5.0-beta
REVERT: 1141b62 Added ReleaseNotes-5.5.0-beta
REVERT: e678970 Server can now handle 5000 jobs
REVERT: 53e28a5 indentation fix
REVERT: 16b24a0 AthenaMCRTHandler.py: a tabulation error in a previous bug fix was allowing only one output dataset to be created and therefore owned by the user. This is now fixed
REVERT: 56dd68d Bug fix for new method validate() of AthenaMCDatasets, allowing it to work with datasets generated by jobs submitted to the Panda backend
REVERT: e1a260c Fixed some problems with nonexisting paths, and updated the JEM external tarball
REVERT: 540794d add error code for the failure in packing output summary directory of AMA
REVERT: cd2040d Some quick fixes in sandbox preparation etc.
REVERT: a8fce4a Automatically updated template for 5.4.0
REVERT: 06ca34a Added PACKAGES-5.4.0
REVERT: 73a2e38 Added ReleaseNotes-5.4.0
REVERT: 2031b85 Removed the ban on t1-repro queues, per Andrejs request.
REVERT: 5e772d9 Fixed problems with multiple lfc path conventions. I imported the dq2 version of lfn_to_native as dq2_lfn_to_native, kept the version in LFCTools.py as lfn_to_native, modified behaviour of the part of the code that adds the lfc path. Problem: Submission is not slower, as we check the lfc path for every file...
REVERT: 2a809a9 support configurable serverMaxJobs. Default is 2000.
REVERT: 54d3270 Fix for bug 57480, change tar option order
REVERT: a3db5b3 enhance the event number parsing
REVERT: 6af45e3 add alternative way to get number of processed events: parse message from AMAEventCounter
REVERT: 76a398c fix message parser to get number of staged files to WN
REVERT: 1c92d9d remove useless imports of python modules
REVERT: 3b5e8c7 enhance the handler for uploading AMAAthena output files to a grid storage  - tar up the "summary" directory in which AMA stores ntuples and histograms  - specify the summary tarball in DQ2OutputDataset
REVERT: 0392dc6 add new AMAAthena runtime handler for Panda backend
REVERT: 2fcb876 Automatically updated template for 5.4.0
REVERT: e70bc68 Added PACKAGES-5.4.0
REVERT: dd54ca6 Added ReleaseNotes-5.4.0
REVERT: 312e8de Fix for groupname problem
REVERT: d84ff71 fix for https://savannah.cern.ch/bugs/?57112
REVERT: 8284f01 Added new TagPrepare application
REVERT: 2bbbc02 
REVERT: a6bc6b8 Automatically updated template for 5.4.0
REVERT: d4cfb5d Added PACKAGES-5.4.0
REVERT: 094c2ce Added ReleaseNotes-5.4.0
REVERT: 2e8c79d SLC5 DIRAC edits.
REVERT: 13be566 revert previous change, which breaks gangarobot
REVERT: 9c5d94d fix the requirement of the pyqt version
REVERT: bcaba8c set DQ2OutputDataset datasetname to not copyable: https://savannah.cern.ch/bugs/?55749
REVERT: 6524954 change the dependency to SLC5 external libraries
REVERT: f3d6c2e Change Milano access algorithm
REVERT: 73e202f 
REVERT: d182291 
REVERT: b6415e1 introduce the AMA specific getstats script for collecting application.stats
REVERT: 1ac0603 introduce the AMA specific getstats script for collecting application.stats
REVERT: b5ca026 updated to stomputil 2.0
REVERT: 4a9f1f3 Enable MSGMS for Athena/LCG in Atlas.ini
REVERT: d75a7ba Update to panda-client 0.1.94
REVERT: 200a74d Update to panda-client 0.1.94 and DQ2Clients 0.1.31
REVERT: b418fd5 add codes in job wrapper to collect simular runtime status in Athena module
REVERT: df8a22f - Added auto-generation of giises.txt in work dir - Fixed some SLOW id-info from proxy, stored in AthenaNGRTHandler directly
REVERT: 69944e2 * Added warning that files_per_job is changed if initializeFromDatasets is used
REVERT: 98a49c8 add more output files
REVERT: 2aa53e5 include env, various athena files
REVERT: 70bc080 should be working
REVERT: 09f027f MSGMS for Athena. Still not functional.
REVERT: 43ee186 Added support for the new method of dealing with TAGs
REVERT: 683eb55 Added support for the new method of dealing with TAGs
REVERT: 58168be Added the ability to supply TAG info for the dataset
REVERT: e467de0 Added the ability to split TagPrepare jobs and TAG info provided
REVERT: 33b8885 Added the TagPrepare application to allow more flexible TAG usage
REVERT: 04b46b4 Change in output dataset management: dataset freezing is no longer done automatically. Users have to run job.outputdata.validate() to do so. New application method: prepare() to append user area to the job. Added support for transforms inline option (new member transflags) and a new application method: diagostic() to identify jobs and subjobs in completed state but with some errors. AthenaMCPandaRTHandler: bug fix to prevent Client.addDataset() to be called once per submitted subjob (which causes a DQ2DatasetExists exception to be raised at the second subjob).
REVERT: 4b841be changes for new Athena interface
REVERT: ed41a5f changes for new Athena interface
REVERT: 21d8236 Changed pyqt version
REVERT: c82a665 adde external dependencies for GangaJEM
REVERT: b4082d5 Added PACKAGES-5.4.0
REVERT: 0914345 Added PACKAGES-5.4.0
REVERT: cee2b46 Added ReleaseNotes-5.4.0
REVERT: 4d11bbf Added PACKAGES-5.4.0
REVERT: 989d826 Added ReleaseNotes-5.4.0
REVERT: 547839b Modified __init__.py to prevent loading of Panda RT Handler if there is something wrong with the Panda module (Panda.lib.Panda classes failed imports)
REVERT: 6cb952f GANGA_NEVER_REEXEC is needed to prevent rexec if Ganga runs within another Ganga session (e.g. testing framework). Note that this requirement is exactly opposite to bug #40110: "GANGA_INTERNAL_PROCREEXEC should be unset after use". Both use-cases are valid and need be supported.
REVERT: b63025c migration asserts relaxed for LHCb use-case (Mike)
REVERT: 4291f71 rm outdated test
REVERT: dce3095 rm some files
REVERT: 6615da0 Merge LHCbSetupProject changes back onto trunk
REVERT: d713bbe Automatically updated template for 5.3.6
REVERT: af4aebc Automatically updated template for 5.3.6
REVERT: 1d8db5a Automatically updated template for 5.3.6
REVERT: ea32460 Added PACKAGES-5.3.6
REVERT: 35a3990 Added ReleaseNotes-5.3.6
REVERT: 2a7ec7a changed name for old cvs-based release tool
REVERT: 9431d21 changed name for old cvs-based release tool
REVERT: c28fb11 remove DQ2_DOWNLOAD
REVERT: c9a1b8d disable DQ2_DOWNLOAD
REVERT: fc4fd74  - set default LCG env. setup to /afs/cern.ch/sw/ganga/install/config/grid_env_auto.sh  - fix bug in composing JDL
REVERT: 2f98238 * Update to work with stomputil version 2.0. * Remove unused files.
REVERT: f0f74b2 Small fixes for GangaJEM, split job handling, default configuration creation, simple debug statistics
REVERT: db021fa change help link
REVERT: 4cb6dff fix the rfcp command issue with the TURL returned from CERN castor
REVERT: cd7ef12 another way to invoke https://savannah.cern.ch/bugs/index.php?53939
REVERT: cefa61a Worked on the split-job capability of GangaJEM
REVERT: 94635b1 Automatically updated template for 5.3.6
REVERT: 0ed527d Automatically updated template for 5.3.6
REVERT: d236d3e Automatically updated template for 5.3.6
REVERT: de095a8 Added PACKAGES-5.3.6
REVERT: 93e83ff Added ReleaseNotes-5.3.6
REVERT: 1a46843 Removed some obsolete warning messages. Fixed JobExecutionMonitor not being copy()-able while the watcher thread is running.
REVERT: addd64f Bumped version of external JEM library reference to the version that'll be used in the pre (0.2.5)
REVERT: 152e759 Merged GangaJEM plugin from Ganga-JEM-5.3.x-branch to trunk
REVERT: 3a891df added comments
REVERT: 9ae1cf0 merged from Ganga-JEM-5.3.x-branch
REVERT: 782e866 Automatically updated template for 5.3.6
REVERT: 0b1a4eb Added PACKAGES-5.3.6
REVERT: 454ff7d Added ReleaseNotes-5.3.6
REVERT: 5103eda Added PACKAGES-5.3.6
REVERT: 43aa46c Added ReleaseNotes-5.3.6
REVERT: 44471a1 Minor fix to sort out the environment for TAG retry
REVERT: 677011f Minor fix to sort out the environment for TAG retry
REVERT: 0df4379 removed printout
REVERT: f0f63b5 Automatically updated template for 5.3.6
REVERT: d38425d Automatically updated template for 5.3.6
REVERT: 9a6929a Automatically updated template for 5.3.6
REVERT: 9f2107a Added PACKAGES-5.3.6
REVERT: d7bb50d Added ReleaseNotes-5.3.6
REVERT: 905a965 Minor fix to stop US/NG being selected when specifying cloud==ALL
REVERT: 5612d3e panda-client 0.1.91
REVERT: 7cbb20b DQ2Clients 0.1.29
REVERT: 50ba089 fixed slc5 compiler to be 43 (NOT 34)
REVERT: b100c07 platform string for SLC5
REVERT: f814338 added test for https://savannah.cern.ch/bugs/index.php?53939
REVERT: 08861cf MERGE GangaAtlas-21-18-access 2245:2380 into trunk
REVERT: ab7578b Add extra ATLAS local setup for Frontier/Squid
REVERT: 4ea2aa0 removed typo
REVERT: 28310be Automatically updated template for 5.4.0-beta
REVERT: 253e53c Added PACKAGES-5.4.0-beta
REVERT: 49f1caf Added ReleaseNotes-5.4.0-beta
REVERT: f7e82cb move some Panda ptions to backend.requirements
REVERT: 760a7bf Automatically updated template for 5.4.0-beta
REVERT: 6e034f8 Automatically updated template for 5.4.0-beta
REVERT: 3911d75 Automatically updated template for 5.4.0-beta
REVERT: f97e934 Added PACKAGES-5.4.0-beta
REVERT: 48ef5d3 Added PACKAGES-5.4.0-beta
REVERT: b6e90d4 Added PACKAGES-5.4.0-beta
REVERT: f130db7 Added PACKAGES-5.4.0-beta
REVERT: fa50922 Added ReleaseNotes-5.4.0-beta
REVERT: e41ab23 AnaTransform fixes for incomplete datasets
REVERT: 7f1480c bug #55267: The output filename that is not present is now named in the stderr
REVERT: bcf859e Added PACKAGES-5.4.0-beta
REVERT: 1e5d85c Added PACKAGES-5.4.0-beta
REVERT: 4ac2859 Added ReleaseNotes-5.4.0-beta
REVERT: 2533d2e Extract dirac minor status from jobs
REVERT: eb39474 Fix bug 55633
REVERT: 38517fb allow empty input dataset.
REVERT: 1968e89 Minor fix for TAG usage with multiple datasets in one job
REVERT: b1a74d7 Fix setup of Athena.atlas_environment
REVERT: 8ff330c Automatically updated template for 5.3.5
REVERT: a9fc46b Added PACKAGES-5.3.5
REVERT: 541dce4 Added ReleaseNotes-5.3.5
REVERT: 521794b prepend username to outds
REVERT: b88ec3f https://savannah.cern.ch/bugs/?55411
REVERT: 2a6ca74 Changed indentation from 3 to 4 spaces (no other changes)
REVERT: cb6f482 Automatically updated template for 5.3.4
REVERT: cc3b6d8 Automatically updated template for 5.3.4
REVERT: 37e705b Automatically updated template for 5.3.4
REVERT: 871c9aa Added PACKAGES-5.3.4
REVERT: 6ca39d5 Added ReleaseNotes-5.3.4
REVERT: ad8a094 Updated uuid messages for master jobs / sub jobs
REVERT: 11a256b Upgrade to panda-client 0.1.87
REVERT: 494a3d4 latest panda client
REVERT: 20227a0 Creating output datasets from AthenaMCPandaRTHandler.py before job submission to make sure that the user owns his output datasets
REVERT: eccd7fc create output and lib datasets before submission.
REVERT: 8fd9961 
REVERT: 56818fe 
REVERT: e5fb8ea added replicate ability + a few others
REVERT: 6b85b55 added replicateFile
REVERT: 2ca8a1e Fixed typo (thanks, Johannes Ebke).
REVERT: f6da365 Added protection for odd DQ2 inconsistency
REVERT: d0921fd SVN commit test
REVERT: edf284b Fixing bug in setup-release.sh preventing kit setup on remote grid nodes.
REVERT: 6bfb851 Small fixes
REVERT: e51252d rm'ing backendapp...matrix stuff
REVERT: c3d6bc4 Adding dedicated run-time handler for Local and batch backends, which are now no longer convered by AthenaMCLCGRTHandler.py. Modified job scripts and AthenaMCDatasets.py to allow for potential DQ2 support for Local and batch backends (this requires a full DQ2 installation available locally)
REVERT: bdcb9e1 Added protection around lfc import, for systems with lfc present but dysfunctional binaries supplied.
REVERT: edb8af5 add workaround for compilation on SL5/64-bit machine   - search for gcc34 and g++34 and use them instead of the default ones
REVERT: c9b0efc 
REVERT: 77d6de0 Dan: added SetVerCommand
REVERT: 4b3747b corrected wrong global function export
REVERT: 5f0b32b set Ganga-SVN tag
REVERT: 5820a15 Added PACKAGES-5.3.3
REVERT: ffa3c14 Added ReleaseNotes-5.3.3
REVERT: 8197340 
REVERT: 3a85f24 Changed logic for updating job status to final value
REVERT: 8c59a9c Minor fix for checking TAG files have downloaded properly
REVERT: 70ba5b1 Fix missing % problem
REVERT: 03a405f Fix dcap/gsidcap and QMUl problem
REVERT: dfe5fac Automatically updated template for 5.3.3
REVERT: 8e25322 Added PACKAGES-5.3.3
REVERT: aa88fb9 Added PACKAGES-5.3.3
REVERT: 6bb4e78 Added ReleaseNotes-5.3.3
REVERT: 5d0cc7b set $Name to Ganga-SVN globally
REVERT: 1fb397f Updated master/subjob uuid for split jobs
REVERT: b33b018 uuid fix for subjobs
REVERT: 5cc67ab Updated version, same as Ganga-MSGMS-1-0 tag
REVERT: af17dff handle gridJobId as list
REVERT: a33a106 handle outMeta and outMS
REVERT: c7ddafb removed debugging statements
REVERT: e45f7c2 enhance match method  - if __jdlfile__ is presented, no need to emulate the job preparation  - restricted the method to jobs in one of the states: new, submitted, failed, completed
REVERT: d1ecbb0  - remove the change for deprecating config.LCG.VirtualOrganisation  - remove the change for setting middleware='GLITE' as default
REVERT: 92a8e83 introduce j.backend.match() method for job list match
REVERT: 251716e 
REVERT: 5b31e82 AthenaMC.py + wrapper.sh: restored handling of trf error codes. The job will return as 'completed', but AthenaMC.postprocess() will scan the logfiles and issue a WARNING if a non-zero transform code is found
REVERT: de4e564 Added failovers for TAG workflow
REVERT: b536711 AthenaMCDatasets.py: bug fix to support backend.requirements.excluded_sites. wrapper.sh: disabling trf error code return as it causes more troubles than necessary (CPU and storage waste coming from perfectly healthy output files lost by stupid trf errors)
REVERT: 66779db Fail over for new lcg-gt syntax
REVERT: 0ea7d35 Fail over for new lcg-gt syntax
REVERT: ac07574 Fix access for StoRM at Milano and clean-up some special SE access cases, use "file" protocol in FILE_STAGER mode for SToRM SE.
REVERT: e0808c0 Add additional protection against empty output_location
REVERT: 7bc5069 Fix bug 53707, filter lcg-cr against malformed output
REVERT: 8ea2f0f stage-in.sh: using dq2-get without -s flag first as it will look for local instances of input dataset first, AthenaMCDatasets.py: minor bug fixes and verbosity changes
REVERT: 94022b2 Fixed a one-liner for accesing datasets that are not containers but also not _tid sets (i.e. real data...)
REVERT: 9ab00ec test.py
REVERT: 38707cd none
REVERT: 1c35f30 Added PACKAGES-95.3.2
REVERT: b747404 Added ReleaseNotes-95.3.2
REVERT: 6fc4e0e final destination
REVERT: 2a9a20f updated SVN tools, reasonably tested with 5.3.1 and 95.3.1 releases
REVERT: 9612bb7 Second step when the jobs are finished
REVERT: 9c8e356 massive changes version2
REVERT: 9c76052 *** empty log message ***
REVERT: 056dcc3 TestPythonOptionsParser.py
REVERT: 1ff99ec Gauss-Job.py
REVERT: 1c5d293 *** empty log message ***
REVERT: a8940ff "remove config['Gridway']
REVERT: a9a8689 Forces the use of the GridCommandSet
REVERT: ec837d7 Updated GangaCron for Gaudi v23r3p1 and tutorial v8r2
REVERT: a54ecd5 Automatically updated template for 5.3.2
REVERT: d9729cf Added PACKAGES-5.3.2
REVERT: c7d2eff Added ReleaseNotes-5.3.2
REVERT: 6c8e499 removed the OSX fix (2 ganga processes would be run at the SAME time with very wierd side-effects)
REVERT: 3af56d4 added stomputil external package (for MSG monitoring)
REVERT: a87bbc0 PythonOptionsParser.py
REVERT: 2bfeb65 export TUTDIR to GPI
REVERT: 0e72615 Allowed the safe use of cloud=='ALL'
REVERT: d39a2a1 Improved cloud <-> site info retrieval
REVERT: 414db2b Short term fix for bug #53613
REVERT: 2f16537 DiracScript.py
REVERT: a6fe606 Removing default value provided to config[DQ2][usertag] as this has the potential to create wrongly named datasets. Adding extra try cases in functions to catch these wrong defaults at job submission time
REVERT: 7a3a88d LHCbDataset.py
REVERT: dcc35a3 Fixed the defaults for GangaRobot to use the repository in ~/gangadir_robot instead of ~/gangadir
REVERT: 64635a7 * put path into variable * print test success * change verbosity to INFO to avoid Athena crash
REVERT: 0d764f2 GangaKISTI first commit
REVERT: 6eea619 GangaKISTI first commit
REVERT: 965662f Added excluded_sites to PandaRequirements. Use like this to avoid matching to sites:    j.backend.requirements.excluded_sites = ['ANALY_A','ANALY_B']
REVERT: 4b9a2f2 Call gridProxy in master_prepare
REVERT: a290e6e Call gridProxy in master_prepare
REVERT: 748e9de Small fix
REVERT: 81ea7c4 Critical patch in AthenaMCDatasets.getDQ2Location(), preventing jobs to write into production space tokens if no specific output site is specified via app.se_name or backend.requirements.sites
REVERT: fd39154 DiracServer.py
REVERT: ce8c299 Fix: input/outputsandbox not settable
REVERT: 2970541 Fix #47680, restore orginal settings of numfiles and  numsubjobs for every subdataset
REVERT: 4273af5 DiracServer.py
REVERT: 4b62a12 *** empty log message ***
REVERT: 7555021 DiracUtils.py
REVERT: 8c0033a __init__.py
REVERT: d50a5af Further fixes for the AthenaMC tests
REVERT: 6aa2297  * enable local protocols of dCache: dcap, gsidcap  * add option to enforce the usage of lcg-cp:    - by setting Athena.atlas_environment += ['FILE_STAGER_IOMODE=remote']
REVERT: 9df41fc Added args to GaudiPython
REVERT: cd1fd43 Fixed numfiles bug (51299)
REVERT: 81a1f01 fix for https://savannah.cern.ch/bugs/?53470
REVERT: 69d0aed Removed length restriction and changed doc
REVERT: 546d6d0 * change default middleware to "GLITE" * remove VirtualOrganisation from config.LCG and rely on the credential package for voms proxy operations
REVERT: b327c98 AthenaMCLCGRTHandler.py, bug fix: app.se_name is restored as output data location handler. Also enabled support of requirements.excludedSites
REVERT: 2c468a3 workaround for wierd OSX execv behaviour (from Ole Weidner)
REVERT: dec46e1 Fix for bug in retry of dataset consistency check (52066)
REVERT: 0e0a01f Fix test reporting logic and decrease timeout to 3h
REVERT: 73dcbab Fix #44427, protection against listCEsInCloudByDomain
REVERT: 69880a7 Update all tests to 15.1.0, add LSF test and some clean-up
REVERT: cda3e4b Fix dataset naming problem
REVERT: f3f4466 Fix bug #53251: lcg_util problem
REVERT: 5875dae Fix #53251: add missin get_lcg_util call
REVERT: d68917b Fix #53251, short_filename as string
REVERT: 5fa1f8a Fix #53251: missing protection for RECEXTYPE, also improve X509
REVERT: a78b855 bugfix for https://savannah.cern.ch/bugs/?50530  - remove deprecated and redundent import statement from code
REVERT: e51a53d bugfix for https://savannah.cern.ch/bugs/?50048  - RetryCount and ShallowRetryCount to 0 is now included in JDL
REVERT: 712e462 Added PACKAGES-5.3.1
REVERT: bd7883a Added ReleaseNotes-5.3.1
REVERT: 8533b24 Enable group dataset names in LCG/Athena, #53155
REVERT: 0d4ce81 .
REVERT: 5c8eb94 add resource match-making as an option before doing real job submission to WMS.  - this option can be activated by setting config.LCG.MatchBeforeSubmit = True
REVERT: 52e6004 added uuid into JobInfo object (j.info.uuid)
REVERT: 7293e8d Added Fluka and some other major updates
REVERT: 8a102ac AthenaMCDatasets.py: forcing exact matching of input dataset first in order to cut down DQ2 query time, and therefore job submission time. Bug fix: new containers for output datasets can now be created at master job completion time instead of just trashing the master job completion
REVERT: 1bcec36 change pandamon url
REVERT: 3ba090b Added PACKAGES-5.3.1
REVERT: d2e4f9d Added ReleaseNotes-5.3.1
REVERT: 1238a68 TestFrancesc.py
REVERT: a3de071 DiracScript.py
REVERT: 2fcfb6d AthenaMC.py: adding a warning in the event jobs.force_status(completed) is called while the subjobs are still completing as we cannot prevent this command to freeze forever without breaking the normal job completion. wrapper.sh: addressing Savannah bug #51792.  AthenaMCLCGRTHandler.py: adding default value for backup output location. Value is CERN-PROD_SCRATCHDISK
REVERT: fbc8065 Initial commit of ArgTransform - a taskification of ArgSplitter
REVERT: 3657f39 Better protection in master jobs stats collection
REVERT: 85f78e3 Add --prepare_old option
REVERT: f70e5aa Increase timeout to 4h
REVERT: f1e7d4f Remove unneeded files
REVERT: 5ef872a Remove from release
REVERT: f6573fb DiracServer.py
REVERT: 568ae0f TestDaVinci.py
REVERT: c965d65 Disable sorting of job builder list view.
REVERT: 21462ea Fix symlink problem is staging out logfiles
REVERT: 4893cfd Automatically updated template for 5.3.0
REVERT: fb16e2f Added PACKAGES-5.3.0
REVERT: e11e105 Added ReleaseNotes-5.3.0
REVERT: 77ac8e7 fixes a problem when TMPDIR is set.
REVERT: 9ac6ad7 Small fix
REVERT: 45d14f4 Initial import
REVERT: 54b7487 Add TNTJobsSplitter in splitting
REVERT: 546a3f2 AthenaMC.py: fixing propagation of 'none' from outputdata.outrootfiles to final fileprefixes, allowing users to effectively disable output types in all modes
REVERT: aff0a16 stage-out: fixed a bug in guid extraction by forcing the scan to match to ID= instead of ID.
REVERT: 4a5bcb3 Revisiting dataset registration implementation to allow job.resubmit() to work without troubles. Also put stage-in back after athena setup, in order to avoid downloading DBrelease tarball if there is already a local setup available. Paving the way for support for perf/phys/trigger group production managers
REVERT: 37b11f2 Automatically updated template for 5.3.0
REVERT: 6281609 Added PACKAGES-5.3.0
REVERT: 509b219 Added ReleaseNotes-5.3.0
REVERT: 70750bf Remove getAgg call
REVERT: f8c5a7d Revert QMUL changes
REVERT: 1c236f6 Fix IFAE
REVERT: ea9ab34 Fix
REVERT: 28c7b58 Fix for QMUL
REVERT: 15c395e Adds an accidently deleted file and improves the caching proxy slightly so that it refreshes its environment if the grid proxy is renewed.
REVERT: 90d991a Fix for dq2-get enviroment problem
REVERT: 6d90114 Fix for IFAE scratchdisk token
REVERT: 878a606 Add file protocol in general and QMUL
REVERT: 1a4915e Automatically updated template for 5.3.0
REVERT: c9c2a95 Added ReleaseNotes-5.3.0
REVERT: 8267958 print first 4KB of error message from shell command (rather than only 255 bytes)
REVERT: 2b89eeb Added PACKAGES-5.3.0
REVERT: 2b26108 Added ReleaseNotes-5.3.0
REVERT: 2b0253a fixed a problem of the unregistered handler workaround (KeyError on logging shutdown) reported by J.Ebke
REVERT: c4efd25 fxed previous fix
REVERT: cf48b87 resort to using os.system('tar xfzv') for python < 2.5 -> tarfile module in v 2.4 is broken for some files
REVERT: 08258f6 fixed tabs
REVERT: 920459f Fixes to gsidcap, DN username extraction etc.
REVERT: 401af42 Changed kill command to do bulk kill of subjobs
REVERT: 8634fa3 Removed.
REVERT: c901be7 Changed to using wrapper-athena-ng.sh
REVERT: a417e69 Re-importing a hopefully executable wrapper...
REVERT: 834eaea Executable now?
REVERT: 010dfc7 Removing, to re-enter after changing to executable.
REVERT: f8ba261 Trying to make this wrapper executable...
REVERT: cb5b15b USERDISK->SCRATCHDISK
REVERT: 1205499 add support for two JDL attributes: DataRequirements & DataAccessProtocol
REVERT: c9caa2e Added direct gsidcap access functionality
REVERT: a6b331f Adding new series of tests for AthenaMC with the Panda backend
REVERT: 8ab38f3 localxml.ini
REVERT: 5d0c26e LHCb.ini
REVERT: 7da2f04 Removing extra printout from AthenaMCDatasets.fill()
REVERT: 80b9935 Also added transform_script to MCTasks test
REVERT: 7c171cc Added transform_script fields (now required) to AthenaMC tests
REVERT: 37583e2 Increase timeout to 10800s
REVERT: 067c4f3 USERDISK->SCRATCHDISK
REVERT: 1e4b9d0 Added better support for RecEx* analyses
REVERT: 29a92d6 Fixed cleanup being applied to task and not to test
REVERT: 85a2fe4 Panda RT Handler: adding .tgz extension to output logfiles from the runAthena job
REVERT: 4a2ed0d DiracServer.py
REVERT: 0d07d11 Automatically updated template for 5.3.0
REVERT: 6750845 Added PACKAGES-5.3.0
REVERT: 93c715d Added ReleaseNotes-5.3.0
REVERT: 42a46b9 panda-client 0.1.71 trust the information system (jobs won't submit if athena release not installed).
REVERT: 4898f28 panda-client 0.1.71
REVERT: 4d75524 Remove libgfal
REVERT: 6b49444 Major update to TnT module
REVERT: 7fe9f5f updates the version numbers
REVERT: 3026f1d DiracCommands.py
REVERT: 8d48dfa more fixes
REVERT: dda6b28 Bookkeeping.py
REVERT: 97a6cd8 Adds a versioning mechanism to the DSTMerger
REVERT: 0f160fa avoid operations on the log files from the glite commands if the log files do not exist/produced
REVERT: a44ccd7 fixed 'remove jobs' for task - slices do not work yet
REVERT: 3f14180 *** empty log message ***
REVERT: 9812091 Fixed initial bugs
REVERT: cafa175 Added all Atlas tests to gpip
REVERT: b304d35 Committed initial Atlas GPIP test containing three AnaTask tests
REVERT: 92cc1d2 TestDiracSplitter.py
REVERT: f5dc77b BenderExample.py
REVERT: 208aec4 Replaced references to USERDISK by SCRATCHDISK, removed default for evgen transform_script, replaced by documented exception thrown if not set by the user
REVERT: f2b6a1a Updating evgen test with new space token for output destination
REVERT: 1c57f30 Fix AthenaMCSplitterJob to use the use_partition_numbers flag
REVERT: 6b68047 Stopped multiple data copying for DQ2_COPY mode (bug 51512)
REVERT: 609a919 DST/
REVERT: cbffcc5 TestDSTMerger.py
REVERT: f9d05fc Internals/
REVERT: 7aec5fb BOOT.py
REVERT: 9ac2d89 BOOT.py
REVERT: 9d9577c Bookkeeping.py
REVERT: 714e41b Dirac/
REVERT: 6c0b541 __init__.py
REVERT: c1c75fa TestDirac.py
REVERT: 37d817c Dirac/
REVERT: 1c01d8e BackendApplicationMatrix.gpim
REVERT: 041e1f3 CVS
REVERT: 6f9c285 testRootVersionsError.gpi
REVERT: c448632 Savannah40219.gpi
REVERT: f18a483 Really fix getJobs()
REVERT: 187a2f3 Fixed getJobs; added try-finally blocks to check() so that the task is always in the correct status
REVERT: 400aaaf Small change
REVERT: f5232aa added backend/cloud detection and initializeFromDatasets to AnaTasks
REVERT: fe4c020 Add getJobs() to Task and Transform and getPartitionJobs() to Transform, returning the job slice corresponding to that task/transform
REVERT: 2a80664 If a non-taskified Application is set in a transform, it is tried to copy all values to a taskified version of the same application.
REVERT: e008ca4 Added functionality to use a user-speficied database release, as set in j.application.atlas_dbrelease. Same syntax as on lcg.
REVERT: 776737a Adds a binding for dst files to the DSTMerger
REVERT: 92f56a2 Automatically updated template for 5.2.4
REVERT: 579bc5e Added PACKAGES-5.2.4
REVERT: a743026 Added ReleaseNotes-5.2.4
REVERT: ea99aed adds a check so that the output is not stored if the output could not be parsed
REVERT: 04ac279 fixed bug #51592: Add self to logger
REVERT: 9764ee7 pycrypto and paramiko python 2.5 hack in syspath
REVERT: 3e69c99 Check for NULL return string of Panda job Id and suggest to shorten dataset name
REVERT: 7a03bb5 Fixed issues with the Remote backend and python 2.5
REVERT: 9984677 Changed print to error in TaskList.py
REVERT: 70ab289 Automatically updated template for 5.2.4
REVERT: 522e096 Added PACKAGES-5.2.4
REVERT: 3ba65e9 Added ReleaseNotes-5.2.4
REVERT: a74e13f Increase time-outs
REVERT: d76fa12 Fix bug #51300
REVERT: 71869f3 bug #51369: the new prepare method generates bad requirements file
REVERT: 285ff35 bugfix: https://savannah.cern.ch/bugs/?50589
REVERT: bc6a514 Merge from Ganga-MSG-branch
REVERT: 4bceb30 fixed bug #51051: Failure to load XML repo (https://savannah.cern.ch/bugs/?51051)
REVERT: b59c248 fixed bug #50351: Logging messages aren't displayed while waiting for commands to finish
REVERT: 0852528 removed obsolete variable
REVERT: d3798c5 Added proper backend treatment of raw input_sandbox and output_sandbox entries; fixed handling of the case where we get a master jid back but all subjobs have empty jid. Will happen e.g. when a site does not have the right release installed.
REVERT: 80116a3 set default site to CERN-PROD_SCRATCHDISK
REVERT: 5fa2a7e fix Ganga to work with newer versions of ipython (-noautocall option was removed in newer ipython versions)
REVERT: d643225 force to submitted (because jobs can go from running to activated)
REVERT: 76956a8 added CPC program summary
REVERT: 8c8c0db backend.CE doesn't exist
REVERT: 3669ea2 close ~/.ganga.log if redirected (to avoid expired AFS token in $HOME for long running Ganga processes) - merged from the Ganga_MSG_branch
REVERT: 8200970 set MaxFileSizePandaDQ2JobSplitter to 10GB
REVERT: 8f81674 bugfix for https://savannah.cern.ch/bugs/?51298
REVERT: d81b221 Automatically updated template for 5.2.4
REVERT: 51d7245 Added PACKAGES-5.2.4
REVERT: c95825a Added ReleaseNotes-5.2.4
REVERT: 1ac803f Add exception for listMetaReplica
REVERT: b046908 Aquire locks for dq2 calls in dq2_list_locations_siteindex
REVERT: 0ff1ab9 Update DQ2Client 0.1.28
REVERT: de7b16d Update to panda-client 0.1.64
REVERT: 30651d6 Added AthenaMCTaskSplitterJob to the possible splitters
REVERT: e687c7f Adding asymptotic limit to proxy lifetime estimate to allow big jobs to complete.
REVERT: b8591d8 Adds the LHCbCachingProxy fix to the default configoration as well as updating the versions of Root used in lhcb.
REVERT: a8fa14d Fix minor bug: if transforms complete while task is paused, they will now be "completed" if the task is run()
REVERT: ee004f6 Changed local dq2 site discovery, relies more on environment Works too if the site still has _USERDISK set and is not in TiersOfATLAS
REVERT: edb1058 Re-fixed a bug for treating ATLAS_PRODUCTION in rel. 14-series
REVERT: 0a624b7 uncomments the exception file, so we fall back on lhcb-proxy-info if something goes wrong.
REVERT: d257ef4 Bookkeeping.py
REVERT: f7e8818 DiracCommands.py
REVERT: 8f503ac Protection for fileinput.input
REVERT: 0efb1c9 Protection for fileinput.input
REVERT: 40042a4 Protection for itotalevents
REVERT: aed2c9c Protection for listMetaDataReplica
REVERT: 9834046 DATASETLOCATION protection
REVERT: 4b4663d Removed a remaining reference to USERDISK
REVERT: 713ac97 limit submit to 2000 subjobs
REVERT: 9e8ff4d Panda server has a per call limit of 2500 subjobs per getFullJobStatus.
REVERT: c36cd31 getFullJobStatus
REVERT: 356d3a3 fill libds if buildjob succeeds.
REVERT: f2daa7b fix the broken alder32 checksum calculation - read file data piece-by-piece
REVERT: 8fc5729 disable the usage of dccp protocol in FileStager
REVERT: b1a0664 Cache proxy identity for subjobs
REVERT: 2f0a196 processingType=ganga
REVERT: 75649b6 get sqlite files to local working directory
REVERT: f3657ec add a function to get SQLite files to local working directory
REVERT: 9fac60a * reduced usage of _name * moved adding handlers to the initialization of TaskList, since not all handlers are added before tasks loading * Added .jobid.date to dataset name for AnaTask if datasetname is not set automatically (necessary at least for Panda)
REVERT: a6e4c1f Skip splitter check for dbrelease if AnaTask is used (splitter already used internally)
REVERT: 7a56eaf Increased speed of loop testing for db_release presence
REVERT: 373cc99 Fixed no AOD files in recon by adding "force AOD" jobOption
REVERT: 9138e9b Changed AnaTask test to different dataset
REVERT: 09fbef9 Outputdata now filled even if enableDownloadLogs disabled
REVERT: 989b169 Added gziping of athena log files, settable log file names through environment variables etc. Also fixed the propagation of atlas_production (again). Note: This update looses us live log file peeking of athena jobs. Must look into this later.
REVERT: c9f5721 Increase timeouts to 3600s
REVERT: 92b86b8 Fix totalevents problem
REVERT: 88de895 New placement of Athena.prepare() in help text
REVERT: 3bdcb11 Add appendTransform for convenience
REVERT: 10c0f9d Make task list wider (jobs also uses more space)
REVERT: f988228 fixes a typo which meant that the wrong doc string was being set
REVERT: 247fd18 Add check for atlas_run_dir
REVERT: 2eac240 Changed WARNING to INFO if pausing completed transform
REVERT: 2bc7a12 *** empty log message ***
REVERT: 5b38ba2 Fixed typos
REVERT: 13b28c7 Changed dataset to merged
REVERT: 5f8648c Fix so that pause() works with the new task status update
REVERT: 33ac1dd * Added state running/pause to Task to show that some transforms are paused * Set Task state dependent on transform states
REVERT: 343286b Added general test for the Tasks functions.
REVERT: 8b9b5ef finishing touches - last(?) round of corrections
REVERT: 047d3ae Automatically updated template for 5.2.3
REVERT: 3606d02 Added PACKAGES-5.2.3
REVERT: 19e136e Added ReleaseNotes-5.2.3
REVERT: 1e7c64f Fixed some hardcoding for rel15
REVERT: ec3a732 Remove libgfal.so
REVERT: 97d43fd Attempt to improve input data downloading by clearing up dead copies from failed attempts. Also, reorder list of input sites by forcing local site first.
REVERT: 8802994 More robust code for SL5 and minor fixes for lcg-cr
REVERT: f6063dd initial import
REVERT: 22aea80 Fixed a problem with previous upload
REVERT: 122011b Changed Scratch disk to cambridge scratch disk
REVERT: 61c14be Added preliminary AnaTask_Panda test and MCTask_LCG test
REVERT: c5863d1 remove task at the end
REVERT: 34086bd Protection against task pausing
REVERT: a2afb80 Allow AnaTasks splitter on Panda
REVERT: 32fc03b Changed AthenaMC test to use the dryrun facility
REVERT: 3fc03ab Switched off random failing in wrapper.sh on DRYRUN to make GangaTests repeatable
REVERT: 520abb6 Added use_partition_numbers to AthenaMCInputDatasets to make Tasks safe and still accommodate loose matching.
REVERT: 3f0d7bb Added test for AnaTask, the test took 15min for me.
REVERT: 6940812 added filter property for GangaObjects
REVERT: f1e5a67 Fix for new lcg-cr timeout syntax
REVERT: 715b598 DQ2Dataset->DQ2dataset
REVERT: 1f77954 debug directory (https://savannah.cern.ch/bugs/?50305)
REVERT: fc53956 Fix AnaTransform to work for NG and Panda
REVERT: 91f49b7 AthenaMCDatasets.py: adding proxy lifetime check in outputdata.fill() to force the job to abort output dataset registration if the proxy is about to be expired. wrapper.sh: now catching the exit status code of stage-in.sh to abort job execution if any single input data file is missing (Savannah bug 50622). stage-out.sh: removing -t timeout flag in lcg-cr as it has been made obsolete in the recent version of lcg-utils
REVERT: 5e44cc2 debug directory (https://savannah.cern.ch/bugs/?50305)
REVERT: 62c8929 Add feature #50558 and remove getJobObject() dependency for self.names, self.exclude_names
REVERT: 0939d18 Add CPU speed Architecure to stats
REVERT: 6ee083c adds some extra checks that the tempory directories from the outputsandbox download are correct
REVERT: ebe419a Bug fix: extFile -> extOutFile in AthenaMCPandaRTHandler.py, improving logging messages in AthenaMCDatasets.py
REVERT: 77492ef startime -> starttime
REVERT: 4246334 Adds the initial pass at an LHCb caching proxy script which is stand alone.
REVERT: af6e69e setPath PANDA_SYS
REVERT: d97c871 applied comments from the reviewers applied corrections from Ulrik and Mike
REVERT: 9831498 Revisited output site allocation mecanism in LCG (RTHandler), added some info level messages to help passing submission time...(MCDatasets and RTHandler)
REVERT: 3c84b76 Added PACKAGES-5.2.2
REVERT: f19e6e8 Added ReleaseNotes-5.2.2
REVERT: 8cce58d revert fix (bad software practices)
REVERT: fa0ee78 Fixed TAG/TNT usuage
REVERT: 0adf5c7 export PANDA_SYS
REVERT: 349e465 added option 'append_to_user_area'
REVERT: 36bc4b4 bug fix in wrapper.sh and stage-in.sh for input file downloading : clashing of environment variables now sorted
REVERT: 2a8c3e2 Added PACKAGES-5.2.1
REVERT: dbfa4de Added ReleaseNotes-5.2.1
REVERT: 748761e Timeout to 2700s
REVERT: d854be0 srm url corrected for lfc registration
REVERT: 8fad6fe CMTscript.py
REVERT: 0aa9653 AthenaMCLCGRTHandler.py: fixing bug preventing job submission to sites where the input data is located
REVERT: 93a7f32 Added PACKAGES-5.2.1
REVERT: d466e35 Added ReleaseNotes-5.2.1
REVERT: 02c4763 Increase timeout to 2700s
REVERT: 57bd2b7 Bug fix for cavern/minbias file handling
REVERT: d4f161b Added PACKAGES-5.2.1
REVERT: cebdeb1 Added ReleaseNotes-5.2.1
REVERT: b78ead9 concat all ErrorDiags for backend reason
REVERT: 21d2426 Correct Panda retrieve
REVERT: 1794063 Correct logic for enableDownloadLogs
REVERT: 508bf12 Add config[enableDownloadLogs] and remove build job output download
REVERT: 0efbbe0 Change Panda retrieve logic
REVERT: 5528d46 use spacetoken in srm url
REVERT: 300e000 Fixing https://savannah.cern.ch/bugs/?46482 in wrapper.sh by moving the call to stage-in.sh before setup-release.sh
REVERT: 85db35b Change to user09
REVERT: 483bb11 export prepare_old method
REVERT: 7120b56 DB Release treatment: now parsing properly DBRelease=DBRelease-x.y.z.tar.gz in extraArgs
REVERT: 1d31b40 Splitters.py
REVERT: b12f439 added modules for pdf processing
REVERT: 18cb591 added imense scripts
REVERT: a9b027a added imense scripts
REVERT: ad7bb08 use panda-client 0.1.53
REVERT: da2926c fix bugs in falling back to lcg-cp protocol when local protocol fails
REVERT: 695295d Fix for https://savannah.cern.ch/bugs/index.php?50086
REVERT: e4af71e Added GangaPanda
REVERT: fb3bf95 use additional packages such as GangaPanda for generation of config templates etc...
REVERT: a08e7f0 Bug fix in stage-in.sh: prevent dq2-get from downloading the whole dataset, overriding input list of files when the input dataset has more than one source. Plus some debug lines in AthenaMCDatasets.py
REVERT: fa95712 fix escape order to be symmetric with unescape.
REVERT: f72c43c fix a problem of  in the strings... use repr() rather than custom quoting...
REVERT: 1f5bf58 Fixed fix...
REVERT: 4ca8458 Adding protection to master job completion in AthenaMC.postprocess(), forcing the thread to wait for subjobs in completing state to finish before running outputdata.fill()
REVERT: 52b8430 Added PACKAGES-5.2.0
REVERT: f657eb7 Added ReleaseNotes-5.2.0
REVERT: 7160b36 Added log file downloading and outputdata filling (DQ2Dataset) to Panda backend Not tested AthenaMCDataset yet!
REVERT: bc768f0 Fix #49823, proper setup for local DB
REVERT: 1bb8bfb Expected number of files fixed in error message
REVERT: 89b1e88 Fixed bug #46825
REVERT: 753194b bugfix: https://savannah.cern.ch/bugs/?49806
REVERT: 1fbb0b1 Changed misleading wording of WARNING
REVERT: b3975da Removed spurrious warnings
REVERT: 22644d7 Change schema version of Athena class
REVERT: 64faa02 unique outds for tests
REVERT: 63d84ca TestFrancesc/TestFrancesc.py
REVERT: 85535e8 Added close statements for tempfiles
REVERT: cb94f6d fall back to lcgcp if trials with local protocol were all failed
REVERT: 7e92049 resolve the transfer protocol based on the SE name associated with the DQ2 site id instead of using close SE
REVERT: 2b35a45 Added PACKAGES-5.2.0
REVERT: e0cca0f Added ReleaseNotes-5.2.0
REVERT: 621af15 Small fix
REVERT: b6375a8 simplified handling of logging filters
REVERT: c85d9fd simplified logging filters
REVERT: 37ca7f0 Removed trailing newline from DQ2OutputDataset.output
REVERT: d52bdf6 Added the option of skipping PoolFileCatalog generation for local backends
REVERT: 823bae2 Fix if numfiles is empty string
REVERT: 245a071 Removed black magic since the core bug for multiple inheritance is now fixed
REVERT: 4674af6 Fix bug #49365
REVERT: 7bdad44 Fixed ARA again Fixed libds support changes to ARA test case new libds testcase
REVERT: aa08fd2 Change prepare calls
REVERT: ef40165 * Fix for bug #49337: PYTHONPATH cmt setup problem in 15.0.0   should be solved now. * New: prepare() has been rewritten   - The functionality should be the same   - New CMT detection and tarfile creation for Panda   - New Athena.athena_compile variable   - prepare(athena_compile=True/False) is obsolete * renamed original method prepare() to prepare_old()
REVERT: a67ecaa don't overwrite outputdata for --panda
REVERT: f81e625 CMTscript.py
REVERT: 9b15d28 loggers created in interactive session are now mapped to Ganga.GPI loggers created in other packages (not starting with "Ganga.") are now correctly using the log file
REVERT: 9643e59 Added ARA test case
REVERT: ade350a Do not uploadSources if libds is set
REVERT: 152d5ae Added user-settable libds support, and fixed submission without local athena setup
REVERT: b5eff29 Securing dq2.freezeDataset() statement in AthenaMCDatasets.py fill() function
REVERT: 1f4cb9a fix #29745: use __mro__ rather than first-generation of base classes
REVERT: cedbb3a Fixed dbrelease not available (necessary for tasks.overview())
REVERT: d3351f5 extra file sanitation
REVERT: 89d5575 Fixed copy() function of task; added "print" on exceptions
REVERT: 602a142 Exclude ganga lfc lib
REVERT: 95f98c9 #49529: provide name for GangaThread
REVERT: 875e289 removed __init__ function , moved initialization of private members to master_configure()
REVERT: c84863f fixed GPIExport of functions added NG RT Handler (not functional yet) added pi example
REVERT: b5102e0 Fixed multiple resubmission
REVERT: 6ae1098 Adds a section on the Dirac.reset command
REVERT: 04100bb Added __init__ function to avoid global scope of lists
REVERT: 6a4630b Added protection and error message when dq2client_version is not set nor read from configuration
REVERT: cc997f3 Restoring dq2client_version in Atlas.ini, as it is needed by AthenaMC
REVERT: a262fb1 bug fix for evgen job option file handling and missing transforms flags in panda
REVERT: ef9bade TestGPUserDLLs.gpi
REVERT: ff44e73 ResubmitGaudiDirac.gpi
REVERT: 3fcb41d TestDaVinci.py
REVERT: 1d4a445 PythonOptionsParser.py
REVERT: 655fc74 change stats. schema 2.0
REVERT: ee0bb22 obsolete test
REVERT: 162f2dc TestSplitters.py
REVERT: e4083a5 *** empty log message ***
REVERT: 64f5b60 Error codes in the Panda object
REVERT: 40b918c percentcpu is an int
REVERT: ea4a3ed collect stats for Panda backend
REVERT: d42b52c Use PandaRequirements
REVERT: 1e3b223 - Move requirements to PandaRequirements - Store the Panda JobSpec in a backend.jobSpec dictionary - Added backend.get_stats()
REVERT: d1c2873 Fixed bug to allow ArgSplitter to work (added check of wether application was really athena in one crucial location
REVERT: 34c2aef Changed because of changed behaviour of AthenaMCDatasets
REVERT: c47d552 fixed integer conversion
REVERT: b3d84ec fix the // issue for queriesto the bookkeeping
REVERT: 07a276d Fixed not printing logger from Thread (again)
REVERT: 76f3d5e Added PACKAGES-5.2.0
REVERT: d893dd4 Added ReleaseNotes-5.2.0
REVERT: 853eba2 Rename prepare->prepare_old and panda_prepare->prepare
REVERT: 6a8ad40 Redistribution of code between AthenaMC.py and the RTHandlers. The application handles the preparation of input/output data through AthenaMCDatasets functions, while the RTHandlers just deal with backend specific issues and formatting, reading all the data they need from AthenaMC members.
REVERT: 1c29b09 Ulrik: Several updates to the Batch backends. The bugs and features requests numbered 35994, 36651, 45219, 47814 and 49078 have been addressed. The most important change is that batch jobs killed by the batch system itself (like if they overrun their CPU limit) are now correctly registered in Ganga as failed.
REVERT: 3abc894 Ulrik: Several updates to the Batch backends. The bugs and features requests numbered 35994, 36651, 45219, 47814 and 49078 have been addressed. The most important change is that batch jobs killed by the batch system itself (like if they overrun their CPU limit) are now correctly registered in Ganga as failed.
REVERT: d67c794 GaudiUtils.py
REVERT: ed974aa add processingType
REVERT: 8e8d94b Revmoved Panoramix
REVERT: e0eb9be Revmoved Panoramix
REVERT: 8d01a4c Fixed test
REVERT: 7f43236 Savannah 48760: Adds a reset method to the Dirac backend which clears out the jobs output and resets its status to the monitorable state 'submitted'.
REVERT: ffdcebc Savannah 49245: Adds a cache to the gridProxy validity check in Dirac.
REVERT: 187a6dc Added PACKAGES-5.1.10
REVERT: 441bf06 Added ReleaseNotes-5.1.10
REVERT: 7b876dc handle multi datasets in subjob
REVERT: 96b1adc fix
REVERT: 34a083a typo
REVERT: ec9e4c7 prepend username to datasetname if not set
REVERT: 01819a8 uses the version of setenvProject on the path
REVERT: 3f4741a don't use user_area_path
REVERT: a2d5323 removed dependency on the LHCBSCRIPTS environment variable
REVERT: de54116 Patching logic hole in output storage token allocation which was causing output data to be written to a forbidden production space token when the input dataset was located in a production space token
REVERT: e5ed8d4 runPandaBrokerage works for no inputdata
REVERT: b4d1584 Removed setting the inputGenerator file from tasks, this is set in AthenaMC automatically
REVERT: 0b5d0f6 Added safeguard so that manually resubmitting split jobs that failed on submission works
REVERT: aa86b78 handle empty inputsandbox
REVERT: cfd24c2 remove pandatools from pythonpath
REVERT: 4cdcc81 use pandatools
REVERT: b35ae0f Introduce panda_prepare, dependent now on panda-client external
REVERT: 5666bea massive Panda changes:   Many Panda options moved to Athena.py.   Athena RT handler now uses prepare from Athena.py   Added Executable RT handler. Not working for me yet.   Added test cases for Athena and Executable
REVERT: 712f418 Small fix
REVERT: 2bffaed Fix for #48964, athena v15 setup
REVERT: 164e78f Adding new Run Time Handler for the Panda backend. Simplified AthenaMC interface: production_name is now inactive, and process_name can be used to fill in for run_number as well. Finally, an attempt to extract process_name for input job options and/or dataset has been implemented, reducing the total number of mandatory fields. Native backend member to select a given storage can be used to replace se_name as well. Arguments in extraIncArgs can be put in extraArgs as well, using the dedicated key word . Added length check on output dataset names.
REVERT: 9e3fe11 Automatically updated template for 5.1.10
REVERT: 603ac92 Added PACKAGES-5.1.10
REVERT: 1cca4f2 Added ReleaseNotes-5.1.10
REVERT: 192f4de warn if BASH_ENV script is missing on worker node
REVERT: 3718c02 Fixes the getOutputData method so is doen't produce an error incorrectly. Test now passes.
REVERT: f516e54 Further fixed copying of tasks
REVERT: 985d261 Fixed not being able to modify copied transforms
REVERT: 3f772bf Added input/outputsandbox to Transform.py
REVERT: 7b950ca Fixed logging from thread not being visible
REVERT: ca99a99 fixed wrong bugfix (#47065)
REVERT: 502658b AthenaMCLCGRTHandler.py: bugfix for min bias and cavern files counting
REVERT: 7c5d172 fixed initializeFromGenerator
REVERT: df660f9 First upload of Imense/ILexR Spidering plugin
REVERT: 18614d6 Removed Euler
REVERT: e155388 Return the list of LFNs prefixed with LFN (uppercase) instead of lfn (lowercase). Seems as if the dirac tool expect this :()
REVERT: 30ad86b Adding new user function, outputdata.recover(). This function runs fill() over all subjobs, then the master job itself and allow to register output files of failed subjobs or subjobs stuck in the completing state. It also allows to freeze the output datasets if the master job is declared as failed
REVERT: 470aef1 Replace USERDISK with SCRATCHDISK token
REVERT: 8e93211 synchronize the data iterm insertion
REVERT: df515f9 improve logging information handling
REVERT: 7b0a5a6 fix race condition issue: https://savannah.cern.ch/bugs/?48435
REVERT: 2eec310 fix bugs in adopting GangaThread module
REVERT: 7be30dd set default monitoring service to the analysis Dashboard
REVERT: f1f89f2 Fix for Bug #48578
REVERT: d6590a8 Fixed outputdata copy bug.
REVERT: e9093f0 Automatically updated template for 5.1.9
REVERT: 523388d Added PACKAGES-5.1.9
REVERT: 10b6b8d Added ReleaseNotes-5.1.9
REVERT: 3380313 check value returned by voname()
REVERT: 20a11f8 Fixed bug resolving LFN's for split jobs.
REVERT: 9b27343 Automatically updated template for 5.1.9
REVERT: 3a9bcdf Added PACKAGES-5.1.9
REVERT: 1fcae80 Added ReleaseNotes-5.1.9
REVERT: 19bccf2 import os
REVERT: c7193a0 small fix
REVERT: cde15a0 Fixed startup bug due to missing GPI.config
REVERT: 6d8b03d in getTidDatasetnames use listDatasetsInContainer
REVERT: 43e9c7c add import statement
REVERT: 4f277bc Patch for CNAF
REVERT: 7e9e2bb added config parameter: forced_shutdown_first_prompt_time = 5 seconds
REVERT: ac822a9 Initial import
REVERT: 7d957b7 * Made GangaThreads quit faster by adding checkpoints * prevented multiple ganga sessions from modifying the same tasks file * Used fsync to make Tasks repository safer * Changed TaskList.py and BOOT.py to use time-based locking * Created new Save-Thread "GangaTasksRepository" to make time-based locking work
REVERT: 72cdac4 update the LCG test case
REVERT: 38c772b Thread->GangaThread
REVERT: 1cd199d Thread->GangaThread. Need to check 'shouldStop' in while loop.
REVERT: 17203bd Automatically updated template for 5.1.9
REVERT: 03e349d removed StuckThread (debugging)
REVERT: 30b7989 Automatically updated template for 5.1.9
REVERT: 049dc18 Added PACKAGES-5.1.9
REVERT: 9d2ce7c Added ReleaseNotes-5.1.9
REVERT: 3d76106 Small fix
REVERT: 4906bc0 Remove dq2 calls
REVERT: c8c2cbb check VO name and provide identity info
REVERT: dbf516f *** empty log message ***
REVERT: d686eb5 Ugh!
REVERT: d313ded I can't get rid of these
REVERT: c235af7 This thing won't die!
REVERT: ef2d5c1 Added first summary line to the docstring of browseBK
REVERT: cfc915b fixed a bug which caused datasets created with browseBK() to be interpreted as PFNs not LFNs. File returned from the bookkeeping interface are now explicitly prefixed with 'lfn:/'
REVERT: d00d3b2 Update to 300 sec update
REVERT: dbdedd8 Update to DQ2Clients 0.1.27
REVERT: 7868f69 hacked get_cmd_prefix_hack
REVERT: 8b10b6c removed CE specification
REVERT: a2e0acc adds __copy__ and __deepcopy__ to GangaList at the object level (only)
REVERT: 8f8bcd2 make the backup of the essential env. variables (PATH, LD_LIBRARY_PATH, PYTHONPATH) globally available
REVERT: c9c5ba5 enhancement for file stager wrapper  - slim the wrapper log  - introduce the detection and the usage of a local protocol
REVERT: 0b8144f Adds an args option to the RootMerger
REVERT: 088ef1d remove the codes for excluding lib64/python in make_filestager_option routine  - it solves the problem on some sites (e.g. TRIUMF) in loading the lfc python library
REVERT: fa1027a Added getenv method
REVERT: f7ba522 csh alias fix
REVERT: 780e7b0 Fixing critical bug implemented in revision 1.28 of AthenaMCLCGRTHandler.py that caused all jobs to fail with lcg error code=2. Also improved support of DBRelease, forcing the job to look for preinstalled, local version of the requested DBRelease in the local software area. Local installation is used if available, otherwise the tarball is downloaded from local SE.
REVERT: 04dd673 TestGaudiLSFRunTimeHandler.py
REVERT: e5064d0 TestGaudiPythonLSFRunTimeHandler.py
REVERT: 46ed356 New test for scaling issue
REVERT: 83d5cf9 Sped it up a bit
REVERT: cfeef52 Updated
REVERT: c3f1a7c Fixed the scaling issue.
REVERT: 26faeb9 merging bug fixes from branch Ganga-LCG-old-MTRunner to trunk
REVERT: cf67780 adopting GangaThread in Ganga.Core
REVERT: 2bc19fd remove codes as they are integrated into Ganga.Core
REVERT: fa9ad86 remove codes as they are migrated into Ganga.Core
REVERT: 6f2a54f fix the broken AMA_FLAG_LIST while setting multiple flag/value pairs
REVERT: ddb639d changed semantics of BOOT.py -> don't import the names into GPI namespace automatically...
REVERT: 690f9bc added GangaThread base class for uniform handling of exit from backgorund threads (the same mechanism is used now for LCG download threads and monitoring worker threads)
REVERT: 42f0dc0 Adding support for AtlasTier0 production caches (to be tested), sort input sites by decreasing dataset shares and allow aggregation of output data files to the site specified in app.se_name by subscribing the frozen datasets at the completion of the master job
REVERT: 3a8669f bug #40460: Little documentation for setting resource requirements for batch jobs
REVERT: 29f5dee bug #40333: Enhance peek functionality to look inside tar files
REVERT: ab081e7 reverted the previous commit (mistake)
REVERT: bd4e1dd fixed: bug #47065: Syntax error in Ganga/Runtime/Repository_runtime.py
REVERT: 51115e4 Force update of outputdata.output_dataset, disable partition number matching if inputdata.redefine_partitions is set to a list of lfns, added support for ATLASSCRATCHDISK space token
REVERT: 7eadd2c deal with monitoring of empty queues
REVERT: b7bf1ae modified behaviour of kill() method
REVERT: 17b6ad0 Adds the GangaNG tests to the release
REVERT: c07ea52 Adds a testing config for the localxml repository
REVERT: a284b06 Automatically updated template for 5.1.8
REVERT: 1be07d9 Added PACKAGES-5.1.8
REVERT: 2e5a377 Added ReleaseNotes-5.1.8
REVERT: 957e206 Fixed conflict w/ DiracSplitter
REVERT: 7cda183 Updated
REVERT: bffd619 Fixed logging problem with missing job IDs. Again.
REVERT: ee77702 This test is outdated.
REVERT: b943a71 Updated
REVERT: 00472fd Creating this subdir
REVERT: d7b44ec moving to subdirs
REVERT: 7e66269 Adding
REVERT: a7b9965 first version
REVERT: 70faa35 Major arch. change. Gaudi/GaudiPython now inherit from common parent class and share run-time handlers.
REVERT: e530690 Minor changes to handle new Gaudi/GaudiPython arch.
REVERT: 2d0f42a No longer need this
REVERT: 6580c47 Separated the classes and moved in some utils from Lib/Gaudi
REVERT: ad4e745 print statements removed
REVERT: e907d7e athena_dbrelease option
REVERT: 6ee5876 removes test for outputdata, as jobs can also get outputdata from options files
REVERT: 1000134 adds benchmark test
REVERT: 6b8bd7b https://savannah.cern.ch/bugs/index.php?47473 dbRelease option is now deprecated in Panda backend.
REVERT: cca3a82 https://savannah.cern.ch/bugs/?46836
REVERT: de595e4 First upload of basic NA48 module to run Compact and Nasim
REVERT: 6476c54 Add size parameter to get_contents
REVERT: 260d6b7 Increase timeouts
REVERT: 53f2893 Change dq2clients config
REVERT: 16ca93b Change retcode
REVERT: 30decc0 Testing; no changes
REVERT: 37bce20 Resume, resubmit, ngclean
REVERT: 143b8eb retcode=0 in get_pybin
REVERT: 0b4cbbc patching stage-in.sh to simplify treatment of input data downloaded with dq2
REVERT: fac8c3b Fix bug #47262, athena wrapper should stop now if there is a compilation problem
REVERT: eaad585 Fix feature request #47506
REVERT: 00f1f47 Added support for storing log files in DQ2
REVERT: bbbc837 Fix typo in AthenaMCLCGRTHandler.py
REVERT: c75bba6 Fix bug #47643
REVERT: e492280 implemented checks on output dataset name length, removed -t -l transform flags in template mode
REVERT: 3f69f5d Fix if dataset only matches a container
REVERT: b68ad59 Fix for bug #47364.
REVERT: 56e4d02 bug fix for: https://savannah.cern.ch/bugs/?46915
REVERT: 7cf3bb2 Fix #47520: TypeError when no file size information available in DDM
REVERT: fbdc612 Change uuid generation
REVERT: 97f8f61 Small updates
REVERT: e7b9768 Added DQ2 tracer reporting
REVERT: 76ae424 Re-Added black magic since the core patch for inheritance from grandparents is not yet in (bug #29745)
REVERT: dd3c0e2 Updates the GangaList to deal with nested lists more carefully
REVERT: b15e0c0 Added small section to documentation on use of ~/.ganga.py file for execution of commands at startup.
REVERT: cd3b9ff AthenaMCLCGRTHandler.py: solved Savannah bug # 47417, added DQ2 tag in job requirements list. AthenaMCDatasets.py: reintroduced output containers and job's subdatasets, since AthenaMCInputDatasets.prep_data can handle them now. Jobs' subdatasets are frozen and associated to their container upon master job's completion. Also fixed potential abuse of dataset naming convention
REVERT: 1c642e6 Add fail over for several output files produced by athena and remove DISK appending for Tier1 site IDs
REVERT: 77b25cc Adds localxml.ini back into the repository types to be used
REVERT: d33cf01 Set empty string for job ID for jobs that failed to submit
REVERT: 8fc8c5b Sort DQ2Dataset by lfn/names
REVERT: 51bb155 Automatically updated template for 5.1.8
REVERT: 6511dbb Added PACKAGES-5.1.8
REVERT: 984cfb5 Added ReleaseNotes-5.1.8
REVERT: 73976b8 Input datasets: now performs a loose match from input dataset name and collect all matching datasets in order to catch all registered input datasets from a container, including files registered in opened (unfrozen) tid subdatasets. Strict input dataset matching is performed on DBRelease datasets only. Automatic input/output dataset naming: removed substrings datafiles and logfiles from paths and dataset names.
REVERT: e14becb introduce and adopt the basic class for Ganga multi-thread handler
REVERT: 64ee517 remove old MTRunner
REVERT: 3b5b254 updated outdated testcases (while trying to make them pass with XML repository)
REVERT: 17e9413 extra comments, remove tabs
REVERT: bf2f13d to ease debugging repository problems, make a dump into a tarfile if there are unhandled exception when scanning the repository
REVERT: df7ff23 when removing jobs which are in the "incomplete" or "unknown" status, do not trigger callbacks on application and backend -> they may be missing!
REVERT: 34d0294 set parent correctly for GangaList items (in __setstate__)
REVERT: e03d27e fixed evaluation of File objects (by value), the implementation uses Utility.Config.config_scope
REVERT: 804504f metadata selection not implemented warning adjust the name of masterJob argument to match AMGA implementation
REVERT: aa0ed8b Automatically updated template for 5.1.7
REVERT: a1a5df3 Added ReleaseNotes-5.1.7
REVERT: 3cf53d4 removed obsoleted test case and fixed an outdated one
REVERT: f1a9b39 introduce log_level attribute in AMAAthena, default level 'INFO'
REVERT: 2ff5a5f fix broken requirement statements on cputime specification
REVERT: ca1859a Automatically updated template for 5.1.7
REVERT: c7c6fc0 Added PACKAGES-5.1.7
REVERT: abc9a9b Added ReleaseNotes-5.1.7
REVERT: 37e0965 Savannah bug #46592: restablishing automatic splitting of input datasets with missing files
REVERT: f9e9695 New numfiles3 stats number
REVERT: 4f1d5dd Correct md5sum length
REVERT: 9a39d58 Corrections for Network stats
REVERT: 0ed5515 use daemonic worker threads
REVERT: 17e8231 do not copy grid proxy when running with Local backend
REVERT: 2600834 MD5 Length set to 32 instead of 36
REVERT: c76b17f bug #47015: adding debugging flag to dq2-get in stage-in.sh
REVERT: e11a30e Automatically updated template for 5.1.7
REVERT: 48ec10c Added PACKAGES-5.1.7
REVERT: c40da25 Added ReleaseNotes-5.1.7
REVERT: d469225 Fix for network
REVERT: a3343bf Fix for network
REVERT: 98fa9b8 Fix for network
REVERT: d3e15da Fix for no replica datasets
REVERT: 4af53f6 Change to error message in case in subjobs are produced
REVERT: 3177c9c Procetion against foul dataset containers
REVERT: 9ef50a8 add a temporary workaround for an issue in FileStager-00-00-28: copy command arguments wrongly passed to the copy command  - workaround: always using the default arguments in the copy command wrapper
REVERT: 51de41b collect all root files in summary/ directory in output sandbox
REVERT: a8e093e return exit code from athena_compile routine
REVERT: bf203be Update network algorithm
REVERT: a006bb3 fix the ama_athena wrappers to generate proper extra job option files for loading AMAAthena Driver
REVERT: bacb42d add 'driver_flags' attribute in AMAAthena for setting AMA driver flags
REVERT: b183341 Added support for ssh keys
REVERT: 64d8df6 Added capability to move files to local disk from symlinks, also added banned reprod queues to xrsl
REVERT: 5f41298 Added capability to move files to local disk from symlinks
REVERT: 251546f Fix container submission problem
REVERT: 6998bc4 fixed xml catalog call to not happen when backend=dirac
REVERT: 584b2c4 updates tests - at lease OutputData passes
REVERT: 7bfd2c9 Add network traffic information to Athena.stats
REVERT: df02d0d fixes typo in name
REVERT: 556bfe9 Add --client-id=ganga tp dq2-get calls
REVERT: 80495ad Add proxy.identity(safe=True)
REVERT: 846fdc7 add proper DATASETLOCATION for  FILE_STAGER mode
REVERT: 97b6480 improved warning message if no subjobs have been created
REVERT: 58521fa bug #46700: Propages the job name to Dirac.
REVERT: 19048a1 Added NGStatTools.py
REVERT: 30f0cf8 Fixed guid handling
REVERT: a59827d Fixed guid handling
REVERT: 5d2d243 Added NGStatTools
REVERT: ba83432 Fixed issue with returning an LHCbDataset object instead of an LHCbDataset Proxy object
REVERT: 2d44a35 added pdflatex flag for arxiv submission
REVERT: 219d2a3 Fixed handling of master package for Gaudi jobs
REVERT: 250b02a *** empty log message ***
REVERT: 5e04094 schema changes
REVERT: 71799fd list files in sandbox cache in color representing different status of the files  - expose list_cached_files() to user interface
REVERT: 2c35306 change basedir of DQ2SandboxCache from users to userxx where xx represents the last two digits of year
REVERT: b19155b Start of implementation for Savannah #46952 - checks Dirac services using dirac.ping....
REVERT: 8bce94a use 0.1.25
REVERT: 57bac09 source cleanup (unneeded comments removed)
REVERT: 3368bfc include the sites advertising 0 MaxCPUTime and MaxCPUWallTime
REVERT: d083777 Added NGInputData
REVERT: c031966 Small updates
REVERT: 52421b5 Initial import
REVERT: 8f67102 Initial import
REVERT: d2508c7 fix the issue that reading sample files from a local directory in StagerDataset was broken in current releases
REVERT: d50b50e Added timeout functionality to ngsub
REVERT: 3447b42 Initial import of NGInputData
REVERT: 2475837 Automatically updated template for 5.1.7
REVERT: b08901c Added PACKAGES-5.1.7
REVERT: d23f116 Added ReleaseNotes-5.1.7
REVERT: a25c8df thread->threading
REVERT: fb5479c CHanged from thread to threading to enable Ganga messages from thread
REVERT: be4638b Replaced many Exceptions with ApplicationConfigurationErrors Suggested at the Developer Days
REVERT: 147a144 Removed NG_DQ2.py
REVERT: 1bd3f49 Removed AthenaNGRTHandler_DQ2.py
REVERT: 4f1d483 Removed reference to outputdata.use_dataset
REVERT: f3c6616 protection added to cover cases where extractFileNumber() returns an empty string
REVERT: 5c69410 added latex compatibility (so one can use pdflatex or latex) + eps figures for that purpose
REVERT: bf336ca Using permanent dataset names for simul and recon jobs
REVERT: 958b16a Added a bookkeeping_command
REVERT: eace40c removed print statements
REVERT: ef95155 add the import for the Bookkeeping
REVERT: 15f0ce8 Added Utilityfunction to start the Bookkeeping
REVERT: 2b17570 New Bookkeeping GUI main code
REVERT: df1b22a readInputData:         Better documentation including examples
REVERT: 6cc867a promoted use of new script adler32.py in stage-out script
REVERT: 1cdb0b7 Fixed crash if task is paused due to too many failing jobs
REVERT: 4218f33 *** empty log message ***
REVERT: 958cc3f indentation fix in AthenaMCLCGRTHandler.py and added adler32.py
REVERT: 08f735c Migration to use of dq2-get for input data and strict job to data policy. Added migration classes in AthenaMCDatasets.py to allow display of old jobs. Bug fix for collection of adler32 checksum numbers of output files and improved debug output from stage-out script.
REVERT: 54cbf93 corrections from Glenn
REVERT: 45dfd42 Updating input dataset names used in simul and recon jobs, clean up unused lines
REVERT: 8d20eca Forgot to add random_seeds to MCTransforms
REVERT: 29b6712 Fixed typeinfo of schema
REVERT: f7c82f8 in DQ2OuputDataset, actual_output is named "output"
REVERT: c31f870 Fixed bug not setting site for split AnaTask jobs
REVERT: 80cf3c0 fixed lib tests
REVERT: c237d52 GaudiUtils.py
REVERT: 4cd11bf GaudiPython.py
REVERT: 581dc4f *** empty log message ***
REVERT: 2485d2a Added setFailed() to Transform GPI
REVERT: 19aef1c Avoid crashes if empty datasets are in containers
REVERT: 2438d06 Adds depth to the MockDataset
REVERT: f979417 Automatically updated template for 5.1.6
REVERT: 8bb7e63 Added PACKAGES-5.1.6
REVERT: b82e096 GridSandboxCache enhancement:  - put cached file information in job repository (instead of __iocache__ file)  - add and expose method: list_cached_files()
REVERT: 1e1c960 Improve DQ2Clients setup
REVERT: a96587c fixed frontmatter and HC affiliation (NIKHEF)
REVERT: b28e04c Automatically updated template for 5.1.6
REVERT: 64f6bf6 Added PACKAGES-5.1.6
REVERT: a92c723 Added ReleaseNotes-5.1.6
REVERT: a368822 Change to adler32
REVERT: 18ef790 Remove DQ2OutputDataset.use_datasetname
REVERT: 1741a84 Adds depth propagation to the splitters
REVERT: 403ac65 add AllowZippedISB=false to glite JDL  - workaround for WMS bug: https://savannah.cern.ch/bugs/index.php?32345
REVERT: 6fa33d4 Changed to using gserver for CVS Changed some platform strings from slc3_gcc323 to slc4_ia32_gcc34
REVERT: e3ca53c Added PACKAGES-5.1.6
REVERT: 7bc800f Added ReleaseNotes-5.1.6
REVERT: 8bf2272 added depth to catalog
REVERT: 2b09931 enhancement for bug: https://savannah.cern.ch/bugs/?43502
REVERT: 68c3b38 Roll back creation of output containers and job subdatasets, added protection against cross-cloud input data replication, support for adler32 and use of lcg tools built-in timeout mecanism on top of existing timeout mecanism
REVERT: 040030d updated plugin diagram
REVERT: 5aba336 promoted "interfacing" to a separate secition added glue paragraph for ARC updated plugin diagram
REVERT: 70adbf2 added section on GUI minor updates from Karl
REVERT: 1e13883 *** empty log message ***
REVERT: c7e65f0 LHCb statistics updates DIANE updates contributor's list update
REVERT: 0d1b576 bug fix: https://savannah.cern.ch/bugs/?45071
REVERT: f8443e1 Changes the default value of depth to 1
REVERT: 4ea2869 Change dq2info.tar.gz to DQ2Clients setup
REVERT: c4b417d Correct python setup for dbrelease
REVERT: 31f12eb Correction for dbrelease dataset handling
REVERT: d929c52 Changed CVS repository from kserver to gserver.
REVERT: 9663853 Savannah 46559 - Adds support for Ancestors to the LHCbDataset and the various Dirac handlers. Also fixes a few problems where inputdata was broken for Root and Executable jobs.
REVERT: fd21313 Increase time-out
REVERT: 71d49bd Savanna 46526 - Adds support for merging textfiles which are gziped.
REVERT: ca0f69a 	minor bugfix for setParameter
REVERT: fc24c14 Added PACKAGES-5.1.5
REVERT: cdd5015 Added ReleaseNotes-5.1.5
REVERT: fe9df2e Added PACKAGES-5.1.5
REVERT: ad1a814 Added ReleaseNotes-5.1.5
REVERT: 41d0a72 fixed: bug #43249: jobs.remove(10) works, removes all jobs
REVERT: 1c2b61e Added the readInputData method to the application. This is an exported method visible at the GPI level.
REVERT: 60dc491 added testecase for bug #40399: Problems with GenericSplitter
REVERT: c507274 fixed: bug #44934: Didn't create .gangarc on first usage
REVERT: 4e4b0bf Small fix for odd dq2-get issue
REVERT: b75ad68 bugfix: bug #45679: j.application.transition_update("removed") is not called on j.remove()
REVERT: 28fca9d Correct middleware
REVERT: 29f0ac9 Revert FileStager inclusion
REVERT: 2e0ab47 Improve TRF support
REVERT: cc70415 Improve TRF support - enable ATLASLocalDataset
REVERT: cedca94 Add FileStager files
REVERT: 55f1c5b revert to Ganga-Credentials-20-00
REVERT: 337df66 safe option for GridProxy.identity()
REVERT: fe86c54 *** empty log message ***
REVERT: 2a56790 corrected front matter (split title over 2 pages) implemeneted a large number of Karl's suggestions for the text content corrected et. al. in references etc.
REVERT: 638d3e3 modified BOOT.py
REVERT: 1b4c9f6 Bugfix for overwriting extraArgs!
REVERT: c9cef5c Merge from 1.7.2.1
REVERT: c95393e updated monitoring figure
REVERT: 066217f new monitoring figure
REVERT: 885799b Automatically updated template for 5.1.5
REVERT: 86e166c Added PACKAGES-5.1.5
REVERT: 9ddb179 Added ReleaseNotes-5.1.5
REVERT: 2eaba59 added AMAAthena following a recommendation of Hurng-Chun
REVERT: 5eb2599 Small fix
REVERT: 0c33004 Remove apostrophe from DN
REVERT: 3972ca5 Fix input.txt and dbsetup
REVERT: beaf953 extFile option for additional files to ship to worker node
REVERT: 6a788bf fix for ares
REVERT: 20e1a6d Small fix
REVERT: 6f2fcdb Add #46268 - create input.txt
REVERT: 92089a6 Removed unnecessary print statement in Download class
REVERT: bd6002a fix for https://savannah.cern.ch/bugs/?46304
REVERT: 5f34f0c #4620, Add dbrelease support
REVERT: 761903d Fix to DQ2OutputDataset.retrieve() method
REVERT: e403991 Fix to make sure the merger works on DQ2OutputDataset
REVERT: 6132092 Small fix to make sure the DATASETLOCATION env variable is set properly when split
REVERT: fefcf9f Update to DQ2Clients 0.1.24
REVERT: cae3c3f use panda-client 0.1.6 use AthenaUtils to extract run config and detect athena env
REVERT: 9a11e5a Added ability for user to see gaudirun.py output if an error occurs
REVERT: 2b92e1b introduce the copy command wrapper for FileStager: fs-copy.py
REVERT: fdb0d94 dump checksum information from LFC for all sample files in pickle format: lfc_checksum.pickle
REVERT: 9d7406b Fixed a typo
REVERT: 3c71f71 added coverage for gpi
REVERT: 0331306 Support for TRFs in the Athena application
REVERT: c9d3a9a added triggerConfig to simulation jobs
REVERT: 129d4ec environment setup on execution node
REVERT: 8107002 Added replacement for genCatalog
REVERT: 78f846f Removed a spurious print statement and added a few utils
REVERT: 2566521 modification for handling stdout/err in different ways  - add config.LCG.JobLogHandler, default value is 'WMS', meaning that stdout/err    will be shipped back to user via WMS's output sandbox mechanism  - set config.LCG.JobLogHandler to other values will remove stdout/err from WMS's output sandbox    and the application can pick it up accordingly to handle stdout/err in different ways    (e.g. store it in a DQ2 dataset)
REVERT: f3826ba Adding Bender tests
REVERT: 67334f4 Adding Bender support
REVERT: 8c60014 version2 of images
REVERT: 2f9f476 version2
REVERT: 89e43e6 added changes from Bjorn, new diagrams and a few minor corrections
REVERT: 25b2248 Adds a method for getting the outputdata lfns and updates one of the tests for it
REVERT: 7af6517 propagating changes from AthenaMC classes
REVERT: bb18360 moved this test to the GPIP file
REVERT: 24986f7 Savannah 46184 - Allows the downloading of the outputsandbox on demand.
REVERT: 1fb708a handle TAGDATASETNAME list
REVERT: 8991eeb Updates the status mapping so that Completed is treated as running. This is because the data transfers are not finished until Dirac is in the Done state.
REVERT: 6a4b4f8 AthenaMCLCGRTHandler.py: bugfix for template mode, removed unwanted default parameters breaking non-evgen transforms. AthenaMCDatasets.py: full support for dataset containers, in both reading and writing modes
REVERT: 052b45c Automatically updated template for 5.1.4
REVERT: 0a1b5b6 Added PACKAGES-5.1.4
REVERT: 548d59e Added ReleaseNotes-5.1.4
REVERT: 3bb264a typo
REVERT: 7cf1069 Fixed adding further files to AthenaMC, e.g cavern/pileup samples Fix by Frederic Brochu
REVERT: 92f07e8 Automatically updated template for 5.1.4
REVERT: 0922c0a Added PACKAGES-5.1.4
REVERT: cfb5909 Added ReleaseNotes-5.1.4
REVERT: de81542 Upgraded to lfc binary 1.7.0 to be compatible with ARC 0.6.5
REVERT: a37330c small typos fixed
REVERT: 2f1663d Automatically updated template for 5.1.4
REVERT: b7c732a Added PACKAGES-5.1.4
REVERT: 9f1c297 Added ReleaseNotes-5.1.4
REVERT: f4df3ba Added support for systems where /bin/sh is not bash
REVERT: a3815a7 Add ServiceMgr for DATA
REVERT: 5d3a32e using athena utility scripts
REVERT: 65eb2e3  - add switch to allow external lcg-cp wrapper script  - enforce downloaded files to be stored in the current working directory
REVERT: 38c0cca fixed a bug in one of the tests
REVERT: 16fe9e8 fixed a bug one of the tests
REVERT: 097b309 Added dryrun functionality to AthenaMC
REVERT: 8389c96 Fix for wrong subjob number for incomplete datasets
REVERT: 9d041f5 fix for glite perusable function
REVERT: 62fe7ac integrated fixes from Will, Frederic, Johannes and Bjorn
REVERT: 90c4bb6 Made Athena also accept an AnaTaskSplitterJob
REVERT: 4d26e80 make redefined_partitions safe, so that they can be used before get_dataset
REVERT: 0a86d47 Fixes TestLFNDownload.gpi so that arbitrary LFNs can also be downloaded. Test now passes.
REVERT: 1f5e348 small fix for dry-run mode
REVERT: 2eeb7ea killing partially submitted bulk jobs on WMS immediately if the whole job submission is not done properly
REVERT: f9bff9c Upped to ARC 0.6.5
REVERT: 1383ea9 Removed Debug
REVERT: 2a02d41 small fix in fqn()
REVERT: cd90acf added check
REVERT: d06c980 Fix for firstevent in AthenaMC
REVERT: 9ff1775 Some NG statistics patches
REVERT: 517555d Big Tasks update: * Transforms and Tasks now use _getParent() * tasks handling is more similar to jobs now * Task initializeFromX work now, and input datasets are displayed in overview() * AnaTask initial working commit * moving setup to check, which can be manually called * several smaller fixes * help update
REVERT: e387cea * partition_number=None made evgen fail, fixed * getPartitionList now works without job object, then returns an open range starting at the first partition
REVERT: 7693f54 Add --splitfilesize option
REVERT: 9c2d174 fix glite-wms-job-submit command-line option composition for multi-thread job submission. bug #45897
REVERT: 9814e47 config['AllowedSitesNGDQ2JobSplitter']
REVERT: 150f4a0 Adds an explicit test that SetupProject Dirac returned as expected.
REVERT: fcf8e86 Change algorithm for filesize retrieval and return
REVERT: 9a14463 Add 'filesize' attribute, which allow spitting on dataset size Add config['MaxFileSizePandaDQ2JobSplitter']=5000 Panda splitting ist done on dataset size now
REVERT: 448bb1b Added support for giis list, fixed bug in assigning subjob IDs when some jobs failed to submit
REVERT: abe77e5 Added giis list file
REVERT: 2b7ed9a Introduce NG splitting based on filesize - maximum is MaxFileSizeNGDQ2JobSplitter
REVERT: 0013b1a Introduce get_contents(filesize=True) - returns the dataset filesize
REVERT: 26caea1 added mailspam
REVERT: 18b4c5c sites takes priority over cloud
REVERT: 3a2649b Restructure athena_setup
REVERT: e612229 Patches for MuonCalibStream dataaccess fror FILE_STAGER and DQ2_COPY mode
REVERT: 1117413 Adds an info log on the dirac version
REVERT: 7f734be Put patch at correct place
REVERT: f884768 Improve athena package setup and tarball packing
REVERT: cf010da Fix typo TNTJobSplitter
REVERT: 596d419 Bugfix for http://savannah.cern.ch/bugs/?45593
REVERT: 6982f88 Added PACKAGES-5.1.3
REVERT: d03f6b3 Added ReleaseNotes-5.1.3
REVERT: 7bdb5b2 fixed references
REVERT: 74468d5 Kuba's updates
REVERT: 1cb0df1 use a net alias for the monalisa server
REVERT: 48d2a18 applied comments
REVERT: 2c31f27 Added PACKAGES-5.1.3
REVERT: d9cb676 Added ReleaseNotes-5.1.3
REVERT: fa5fb2d stdout.txt -> stdout
REVERT: 00ba675 Fixed bad lfc registration on NG backend
REVERT: 6034607 Fixed bad lfc registration on NG backend
REVERT: a4eab56 applied changes
REVERT: c45569a fixed minor bug - forgot to import something
REVERT: caf97a7 Automatically updated template for 5.1.3
REVERT: 50600c6 Added PACKAGES-5.1.3
REVERT: 902801c Added ReleaseNotes-5.1.3
REVERT: 1e6a914 override theApp.EvtMax in the FileStager specific input.py
REVERT: 28b75fd dbRelease option
REVERT: 437a0de Small fix
REVERT: dcbf6ec typelist fix for GUI
REVERT: 02b1213 adds missing import
REVERT: 16d9a13 fix application.mode unset
REVERT: f7e268e quick fix for startup problem
REVERT: 1443ffd Changes for improved data handling and more splitting capabilities, goes along with new Tasks version from Johannes Ebke
REVERT: a142fb0 Changes for new Tasks package
REVERT: 5dd152c Remove old Tasks package, Initial import of new version from Johannes Ebke
REVERT: 01ff571 internal code factorization
REVERT: 8cabc17 include the monitoring service
REVERT: 1d3875a fix sample file name of the FileStager
REVERT: 3eabe08 run time exception fix. failed to find module
REVERT: b8f8c8c PythonOptionsParser.py
REVERT: b127337 full DaVinci tests
REVERT: 277ab5f adding gpip test
REVERT: c0189ec moved into gpip test
REVERT: ed9a1d7 moved to gpip test
REVERT: c079c5d moved these into gpip test
REVERT: 6fda135 unit testing for Lib/Dirac
REVERT: 56a45ad Moved all DIRAC script-like commands into DiracUtils.py. Also, did some work on bringing things into python std.
REVERT: e6820e5 Code updates for NIKHEF/SARA
REVERT: abd8177 avoid having two same downloading tasks in the task pool
REVERT: c13bbd2 clean up logging messages
REVERT: 6582b95 Add Athena.user_area_path: configure path where to put user_area     file which is created during call of prepare()     Changed default to $TMPDIR or /tmp/$USER     If Athena.user_area_path='workspace' old default of ganga workspace     is used
REVERT: 0984503 allow to set the max. node number of a glite bulk job
REVERT: c59a069 last commit was incorrect file
REVERT: 4697065 Updates for sandboxes. Will only work in the latest Dirac client
REVERT: 3fa7fb6 Fix for the master job numfiles2
REVERT: 28fc0f3 Change in the .gz reading and fixes for numfiles for the FILE_STAGER
REVERT: 43698c4 moved future import to 1st line
REVERT: 7206fe1 fixed the gaudirun.py + stdout issue
REVERT: f54b5a0 removes dependency on version of root no longer on the Grid
REVERT: 4d710de use sampleFile argument instead of sampleList in the FileStager input.py
REVERT: 28992c9 add support for FILE_STAGER mode remove support for DQ2_COPY mode
REVERT: 1236f36 enhance the ama specific athena job wrapper  - introduce exit code  - add compatible timing report to adopt the state collection feature in Ganga/Athena module
REVERT: c833b76 set default max_event to -1 indicating nolimit
REVERT: 5e7e027 Add OUTPUT_FILE_NUMBER reading for Tasks package
REVERT: 2d63c15 stdout.txt gzipped
REVERT: f0cdd10 Make logfile reading more memory friendly
REVERT: 0e2b26a Improves the overlarge sandbox handling to make use of the Dirac OutputSandboxLFN parameter and do the untarring.
REVERT: d3e09f3 make the number of output downloader threads configurable
REVERT: 60e7abc Committed Hurng-Chun's changes for implementing a file access mode using the FileStager
REVERT: 29db8e0 Fixed a buggy equality
REVERT: 8327552 Small fix
REVERT: 469a997 Add try/except protection
REVERT: 25529cd Introduce Athena.collect_stats switch and add master job statistics collection
REVERT: 3f01c75 Enable DQ2JobsSplitter for NG
REVERT: f8fabd7 dataset name list fix
REVERT: 3557eb8 Fixed bug in the schema relating to 'project'
REVERT: f5961c2 Changes the SetupProject calls to use LHCBSCRIPTS
REVERT: dd95ddd introduce histogram chart
REVERT: 21773a5 Take lfc, srm info from ToA, allow for writing to remote storage, add timing info for HammerCloud
REVERT: b737125 Moved these to Gaudi unit testing
REVERT: 2ae1ff2   - Use search/replace in TURL list for file protocol     sites IFIC, LIP-LISBON   - Patch for RALPP, IFAE and PIC localsiteid   - rfio/DPM fail-over improved   - print warning if lcg-gt times-out   - Add extra hosts to defaultSE for GRIF-LAL, GRIF-SACLAY
REVERT: 8cffffa Moved this test to LHCbDataset unit testing
REVERT: 3248b03 Moved this test to LHCbDataset unit testing
REVERT: 35b0047 Removed some unnecessary print statements
REVERT: f4f2fa5 show the test failure with exception traceback on stdout
REVERT: 9115fa8 tip from Andrei on multiple lfns
REVERT: e140136 Adding unit testing for LHCbDataset
REVERT: 025df55 Bringing code upto Python standards
REVERT: 2897f60 savannah 44923: use the dirac job parameters to construct the lfn rather than guessing
REVERT: 4fc8d41 Added stuff about controlling output destination
REVERT: a0fb8a7 Another place where [] is the default value
REVERT: 90a037f Small fix
REVERT: 85b8665 Introduce use_blacklist switch and correctly apply GangaRobot blacklist
REVERT: 04443f6 Revert NIKHEF change
REVERT: 4bfd159 Patch for NIKHEF due to unexpected ToA entry for the Topology
REVERT: 4434720 fixed small bug in matching a description to the wrong field
REVERT: be51383 updated fort DIRAC 3
REVERT: f6a4083 Updated GangaRobot examples for Ganga 5.1 and Dirac 3
REVERT: 76988e1 Fixes for NIKHEF/SARA
REVERT: f4415f1 Fix for SARA
REVERT: 8b163d6 Better protection against wrong LFC output, correct IFIC and LIP_LISBON protocols, improve IFAE and PIC localsiteid, set GLASGOW to single TURL mode
REVERT: 5a17822 remove useless file
REVERT: 4bb2416 Automatically updated template for 5.1.2
REVERT: b2700da Added PACKAGES-5.1.2
REVERT: 7857471 Added ReleaseNotes-5.1.2
REVERT: 8e788ac use Batch rather than explicitly LSF
REVERT: f2e7e26 extra exception output if cannot load the plugins...
REVERT: 52e21ce improve get_file_location routine
REVERT: e4b5e88 include ama_athena_utility.py in the input sandbox
REVERT: 7a0c401 enhance ama athena job wrappers
REVERT: 5a5ff9d load SimpleStagerBroker module
REVERT: 7685317 introduce new class for data-oriented brokering based on file locations
REVERT: c45e4b2 introduce dm_util.py containing utility subroutines for DM
REVERT: 5b834ed introduce shell subroutines specific for AMAAthena
REVERT: 428c2d0 bug #44393: wrong redirection of stdout/stderr in Shell
REVERT: 0b6feb9 fix the confusing Grid Proxy validity warning and the warning will be printed when the lifetime is shorter than 1 hour
REVERT: 9ae7fe7 Fix timezone
REVERT: 57b4bda fixed bug in python script written to run jobs
REVERT: 4e91916 fix for savannah 44414 - recursive listing fails if called more than once due to the reference being kept between calls.
REVERT: 635183b Change to dq2client_version = 0.1.22 and add it to Atlas.ini
REVERT: 51707ad Athena.stats updates for GangaNG
REVERT: 5027fbd Added -v to time
REVERT: 38858b2 Removed superflous useridARC check
REVERT: 3a98813 Small change for DQ2_COPY workflow
REVERT: b236d59 Add dq2clients requirements for DQ2_COPY mode to jdl
REVERT: 828bb2b small fix for join() method
REVERT: 33c4a11 remove the uesless import: threading override Thread.join() to see if it helps in shuting down the daemon-like MTRunner
REVERT: 5865ead Small fix
REVERT: c228783 removed localxml.ini and remote.ini from the test configurations
REVERT: 49bb0c5 introduce configurable number of worker threads
REVERT: 2295407 Added time information to athena run
REVERT: 47b1ceb GPIP (parallel) tests from Mason
REVERT: 89edcdd GPIP (parallel) tests from Mason
REVERT: fcc11ae PYTF version 1.5: GPIP (parallel) tests from Mason
REVERT: cb19962 GPIP (parallel) tests from Mason untabified test.py
REVERT: c349a96 GPIP (parallel) tests from Mason untabified the driver.py file
REVERT: 9d48978 Fix logic of siteindex_update
REVERT: ae3464a Small fix
REVERT: 776f86b Add numfiles2
REVERT: b663427 Add numfiles2
REVERT: 90a12e3 bug fix: #106226  - wrong type setting for 'software' and 'others' attributes
REVERT: b15fade introducing "SubmissionThread" configuration variable for setting the concurrent number of job submission threads
REVERT: b112947 Add addition time parameters for Athena.stats
REVERT: 2bb89fd *** empty log message ***
REVERT: 233481b bugfix to allow production with locally modified production tarball. Also prevent creation of illegal dataset names in DQ2 by removing single quotes in user names and cropping long dataset names.
REVERT: c6e5fd0 Small fix
REVERT: a54923e Factorize and extend job statistics
REVERT: 40334ce Update of timeout
REVERT: cdbd9de First Version: athena HelloWorldOptions.py in 14.2.20 on LCG backend
REVERT: 1c83173 First Version: athena HelloWorldOptions.py in 14.2.20 on Local backend
REVERT: eac9003 Fix #44426 - DCACHE_RA_BUFFER should be an str for Local/Batch
REVERT: cdbd6f0 print the "waiting for output downloading processes to be finished" warning message when needed (the downloader has been running for longer than 10 sec. when closing the Ganga session)
REVERT: 033264a bug #43917: Implement Batch backend as alias to default backend at a given site
REVERT: 33974e7 bug #44259: GangaLHCb tests fail due to gridProxy check
REVERT: 5d1875f Automatically updated template for 5.1.1
REVERT: d3d1db0 Added PACKAGES-5.1.1
REVERT: 310bc97 bug #44334: Ganga/Utility/Shell.py does not save environ
REVERT: 9de346b #bug #44256: Job in state "incomplete" is impossible to remove
REVERT: 8478377 Fix bug #44372, change md5sum calculation procedure
REVERT: dee7fe9 Automatically updated template for 5.1.1
REVERT: 85e689b Added PACKAGES-5.1.1
REVERT: 3955f74 Added ReleaseNotes-5.1.1
REVERT: 2ef8275 switched back to setenvProject.sh
REVERT: 7fc1e66 Removes print statement
REVERT: db3fdf6 Savannah 44261: Deprecates DiracTopDir in favour of DiracVersion. If DiracTopDir is set, but DiracVersion is not set then the current behaviour will be kept. If DiracVersion is set, it will override the value set in DiracTopDir. The default for both is to take the latest.
REVERT: 3c86665 Automatically updated template for 5.1.1
REVERT: f12377f Added PACKAGES-5.1.1
REVERT: a49adf2 Added ReleaseNotes-5.1.1
REVERT: 9ee2102 Automatically updated template for 5.1.1
REVERT: 9f82b58 *** empty log message ***
REVERT: 10bb0d0 first version
REVERT: 4c7e003 Added PACKAGES-5.1.1
REVERT: 06241a5 Added ReleaseNotes-5.1.1
REVERT: 8cb7ee7 *** empty log message ***
REVERT: 009d68b first version
REVERT: 804548e try
REVERT: 05e6f1f Fix bug #43702 - add exception and non-zero-return code
REVERT: 8603f90 Fix bug #44213, input.txt is now written in DQ2_COPY mode
REVERT: 7a79355 Improves the error reporting for monitoring
REVERT: bd8da49 New version with official patch against RA_BUFFER problems
REVERT: 6e4d036 Add -d option to dq2-get
REVERT: 46c61cc test for the grid proxy problems I've been seeing in the GangaLHCb tests
REVERT: 697b603 Add DQ2JobSplitter.update_siteindex and small fixes
REVERT: 7b93e42 fixed bug #44181
REVERT: 6bfb31f Small fix
REVERT: d89a170 Change single DATASETNAME env to DQ2_LOCAL and DQ2_COPY
REVERT: 2a1fe9d Change log level of dataset registration
REVERT: 9d71e48 Change splitting on dataset level only
REVERT: 52c475b Add retry loop for dq2-get
REVERT: 9d4bcb9 Add MaxJobsAthenaSplitterJobLCG again
REVERT: ae13c34 Make DCACHE_RA_BUFFER configurable
REVERT: 857c8f1 Fix bug #43946, DQ2_COPY mode for multiple datasets
REVERT: 48784ac Add  Add check if input files has copied in DQ2_COPY
REVERT: d3d9df0 Add Athena.stats
REVERT: e5286f7 * AthenaLCGRTHandler.py: Fix bug #43946, DQ2_COPY mode for   multiple datasets * AthenaLCGRTHandler.py: Add warning message for failed   submission in dataset splitting mode
REVERT: e63cc1f added property for defining Grid storage
REVERT: ddc6185 joining the MTRunner thread to synchronize with the shutdown service
REVERT: d1e6556 supStream support: suppress some output streams. e.g., ['ESD','TAG'] improved logging messages
REVERT: f3dac6b update master job's status at the end of the master_updateMonitorInformation() in any case
REVERT: 59bbba6 update the master job's status after the downloading process is finished  - avoid leaving the master job's status not updated in the repository
REVERT: 689bfe3 remove too frequent logging message
REVERT: 1d379ad Moved these files to options dir.
REVERT: 395da24 Adding unit-testing for the Gaudi directory.
REVERT: 68474eb Adding unit-testing for Gaudi.py
REVERT: 95be196 Adding unit-testing for DSTMerger.py
REVERT: 6f99af1 Removed GaudiVersions test as what used to be in GaudiVersions.py is now fully tested via the unit-testing done on GaudiUtils.py.
REVERT: 7d16562 Minor edits due to the move of DiracSplitter to the Lib/Dirac dir.
REVERT: 7dbb17a Removed some hard-coding of platforms which caused the tests to fail.
REVERT: 93b199a Added addional config option to control output destinations.
REVERT: c85edfd Modified all source files to bring them (mostly) inline w/ Python standards. Also, added functionality to give user full control over destination of output files.
REVERT: 1649208 moved DSTMerger.opts and GaudiPythonExample.py into the options subdir (to separate them from the source code)
REVERT: c684cc2 Contents of env.py and GaudiVersions.py have been moved to GaudiUtils.py
REVERT: a6438de removing this files - all tests should be under GangaLHCb/test/
REVERT: 2dd3d0b Moved DiracSplitter into the Dirac directory. Also, made a few small changes to the GaudiDiracRunTimeHandler to work w/ the new Gaudi outputsandbox/outputdata format.
REVERT: 461097b AthenaMCLCGRTHandler.py: Fixing condition trigger false warning in input data splitting
REVERT: f0f20f3 small correction
REVERT: 93a6d2a Fix for bug 42661: User packages not being recognised in Athena Root dir
REVERT: 4b8a19d uertag in dataset name and outpu_lfn
REVERT: bcf8d69 fix wrong error message: savannah 43844
REVERT: 2130b1f Updating default output storage location to CERN-PROD_USERDISK as CERNCAF is long obsolete. Relaxing constraint on job matching to let jobs running at CERN as CERN Tier 2 facility is not properly tagged in LCG BDII
REVERT: 52143c9 modified cleanup
REVERT: 0651e30 Disable credential creation/renewal when Internal Services has been disabled.
REVERT: 31f37b2 Added mechanism to differentiate exitcodes. Fix for bug #40111.
REVERT: 82c9604 modified LD_LIBRARY_PATH
REVERT: d12c364 Automatically updated template for 5.1.0
REVERT: b65661c Added PACKAGES-5.1.0
REVERT: d01fd6f Added ReleaseNotes-5.1.0
REVERT: 1c9c028 file name fix
REVERT: 8edf132 reduce the task queue checking duration to once per 0.5 sec.
REVERT: db5f9ee expand $VAR and '~' when setting path-like options
REVERT: 2cb674b added j.submit(keep_on_fail=False) option (request #43143)
REVERT: 8319617 fixed bug #40271
REVERT: d1d5737 fixed Shell in case the setup script produces garbage on stdout (now discarded to /dev/null)
REVERT: 9abe142 introduce the exitcode in the athena_setup routine and stop the main wrapper when the exitcode returned is not 0
REVERT: e2ff453 Automatically updated template for 5.1.0
REVERT: f57dbe6 Added PACKAGES-5.1.0
REVERT: ae6dde1 Added ReleaseNotes-5.1.0
REVERT: 53e3510 add workaround for the gLite WMS issue:  http://savannah.cern.ch/bugs/?43632
REVERT: 16a49a1 use the type system completely
REVERT: b8382c0 remove useless routines
REVERT: 19bd4af update the code annotation
REVERT: b8ceead fix the bug in passing LFC_HOST to the job wrapper while using LCGSandboxCache
REVERT: 88cba46 Updates the lhcb workbook for Dirac 3
REVERT: 87f2c15 fix the bug triggering the annoying warning message after subjob resubmission
REVERT: 76f40ae Automatically updated template for 5.1.0
REVERT: ad9c4c9 Added PACKAGES-5.1.0
REVERT: 4750341 Added ReleaseNotes-5.1.0
REVERT: 7c9b146 Added PACKAGES-5.0.11
REVERT: 49ffb35 Added ReleaseNotes-5.0.11
REVERT: 2a420e7 enhance the internal setup for the SandboxCache
REVERT: 4c596a1 remove the LFC module python path coming from the external LFC library for GangaNG
REVERT: ee6aee1 updated config for release
REVERT: 8886556 removed spurious print statement
REVERT: dc3dd6e Adds a warning if the sandbox was oversized.
REVERT: d1f1309 Increases the timeout on this test
REVERT: a9770fd Added PACKAGES-5.0.11
REVERT: cad1ace Added ReleaseNotes-5.0.11
REVERT: 266accb fixed typo 'destionationSE'
REVERT: b24839d AthenaMCDatasets: addding exception handling of incomplete records of input datasets in the DQ2 database (datasets with no known locations)
REVERT: dce0456 Updated AthenaMC.gpi input datasets. The previously used datasets have been removed from DQ2 database, causing the tests to crash
REVERT: c062b3f Removes some useless test cases and fixes some which didn't make any sense
REVERT: c1c70e4 Fixes the test after the changes to getpack. I wonder if this breaks ganga somehow?
REVERT: f2cd58e Automatically updated template for 5.0.11
REVERT: c19c747 print added
REVERT: 02bfd47 Added PACKAGES-5.0.11
REVERT: 7601f1b Added ReleaseNotes-5.0.11
REVERT: 78e1783 Upgrade to new DQ2Clients 0.1.22
REVERT: 6334267 Fix bug #42944
REVERT: 2a8477a old file name convention compatibility
REVERT: a9677fe Fix for bug 43312. Test case exists and passes
REVERT: e12d6a8 test case for bug 43312
REVERT: b6ae925 Copies the diracMain into the inputdir
REVERT: f0c6336 Updates the URL for the monitoring page
REVERT: a3ffdc7 stops failed jobs from downloading their sandboxes
REVERT: c9dc4f5 Fixes a bug in the error msg formatting that caused an exception
REVERT: c6ff7c1 Dirac 3 merged to HEAD. Need to perform lots of testing
REVERT: e931144 Correct setting of max_event
REVERT: 7f2c43c Introduce splitting per dataset
REVERT: 8e2ac00 stop logging stuff
REVERT: d3d5d22 request #41137: Please add a nice level option to the Local backend
REVERT: 0449f01 bugfix #40932: Ganga incompatible with shell functions (using os.environ directly instead of printenv)
REVERT: 5bf6cad test obsolete
REVERT: cc2acea test obsolete
REVERT: 924eb82 disabled for LocalXML
REVERT: 4144bdc fixed wrong asserts
REVERT: e0c1756 fixed the rollback to new state problem of split jobs: reset the subjob counter to the original value so that j.subjobs[0].id == 0 (Ganga/test/GPI/CrashMultipleSubmitSubjobs testcase)
REVERT: 6125491 move installation of  the shutdown manager for atexit handlers to bootstrap.py
REVERT: f9bdca7 install the shutdown manager for atexit handlers before loading system plugins (e.g. LCG download thread registers the atexit handler using a tuple (priority,handler))
REVERT: 1ff437c Added dependency on LCG/external Python2.5, pyqt (3.17.3_python2.5) and sip (4.7.7_python2.5).
REVERT: f13a050 Automatically updated template for 5.0.10
REVERT: c6e150d Added PACKAGES-5.0.10
REVERT: eb4a228 handle max_event as an int
REVERT: 396ba46 Added PACKAGES-5.0.10
REVERT: c79fcc0 Added ReleaseNotes-5.0.10
REVERT: 1e6f810 Removed debugging message
REVERT: 9f97c15 Fix for maxevt
REVERT: c79655f maxevt is now int
REVERT: 600650c min. limit for sandbox/groupArea upload 5M
REVERT: c5bb05e Removes an incorrect test and adds small fix to make another pass.
REVERT: 7e576a1 Changes default to be [] for gList.
REVERT: bc4ec83 Changes the Robot testing to use a different gangadi and different user name such that it can be run at the same time as the Ganga core testing.
REVERT: ade9fa1 potential fix to the dataset thing.
REVERT: 03a5f05 most stupid bug in the world (I really wonder how it ever worked)
REVERT: ab58df0 comment out prints
REVERT: d0eb9e1 Automatically updated template for 5.0.10
REVERT: 3991884 Added PACKAGES-5.0.10
REVERT: 9c319b4 Automatically updated template for 5.0.10
REVERT: 45057c9 Added PACKAGES-5.0.10
REVERT: 9487435 Added ReleaseNotes-5.0.10
REVERT: 2f731fa Added ARA support, setup of local databases
REVERT: 997e595 Add AtlasTier0
REVERT: d95bfeb creativ bug fixing
REVERT: 02cec08 Change defvalue of partition_number
REVERT: 8974733 add lib64/python removal
REVERT: eb30a8e Add dcap config for SFU, TRIUMF, ALBERTA
REVERT: 2ced13b Change python32 workflow
REVERT: 273b463 Remove special MPPMU setup
REVERT: 0502d26 Fix HelloWorld job for Local/Batch backend
REVERT: ee43125 add AtlasPoint1 and AtlasTier0 projects
REVERT: c3d8719 fix typo: self.nrjob -> nrjob
REVERT: af4d461 add 'AtlasTier0','AtlasP1HLT' to alternateCMTextractor
REVERT: c66f0ca removed dependency on external pyqt.
REVERT: 2e2c07a Remove PandaJobSplitter (implemented in DQ2JobSplitter) Add brokerage if not using splitter
REVERT: dc33e58 add runPandaBrokerage and queueToAllowedSites functions
REVERT: e662dbf Use DQ2JobSplitter for Panda backend
REVERT: b32a83e Support Panda backend; performs brokering and handles incomplete datasets.
REVERT: 4d57521 Add _append_files routine and add libdcap.so
REVERT: 6221ee7 Set read ahead buffer for dcap access to DCACHE_RA_BUFFER=32768
REVERT: 44fffdf athena-utility.sh: remove duplicated ganga-stage-in-out-dq2.py statement that could cause false double output file registration
REVERT: 4f093d2 DQ2JobSplitter creates the correct number of subjobs for complete datasets (Bugs #41726 and #42856) Creates at max MaxJobsDQ2JobSplitter per site for incomplete datasets
REVERT: 94a66c7 Athena.max_events has to been an integer now (Bug #42613)
REVERT: 0d39386 Bug fix for inputdata.inputfiles type, improving timeout evaluation for stage-out, allowing support for stage-in files from SRMV2 space tokens, extending splitter to allow partial resubmission of master jobs (all failed subjobs at once instead of one by one.)
REVERT: 57bd834 Modified Gaudi application and runtime handlers so that they now place all .py files under InstallArea/python into the job input sandbox. Previously only *_merged_confDb.py, *_confDb.py, *Conf.py and _init_.py were copied (Configuration.py was missing).
REVERT: 59534e2 fixed symlink creation
REVERT: 3c0686c removed dependency on Python 2.3
REVERT: 6fa65ac bug fix
REVERT: 278bfa5 lfc updates
REVERT: bb072fc - Added test for GaudiPython on DIRAC. - Removed some duplicate tests.
REVERT: 5855936 Bug in translation of dataset object for GaudiPython jobs running on DIRAC.
REVERT: 11927cf cmt setup, adler checksum
REVERT: abe6f4f LFC tools for lfc registration
REVERT: ee0e2be Transition to LFC
REVERT: 5a8ad86 Update ARC client 0.6.4
REVERT: 3285b91 uegede: separate TEST REPORT step with automatic download of testing from external sites (Imperial)
REVERT: da91cc0 Automatically updated template for 5.0.9
REVERT: f25a1e0 fix the symlink to coverage reports
REVERT: a8812bd Added a new action to the TestRobot which is responsible for creating an archive .tgz file with all the test results.
REVERT: 2d8f95c PandaJobSplitter now handles the brokerage and splitting of complete/incomplete datasets.
REVERT: 5e215d1 support dataset files in local directories
REVERT: e7e042a Added PACKAGES-5.0.9
REVERT: fc4c2f4 Added ReleaseNotes-5.0.9
REVERT: 1bd5a64 added typelist information for submit_options
REVERT: cf7888c avoid doing glite-wms-job-cancel on jobs which is in a final state  - glite bulk job status is now correctly stored as master job's status
REVERT: 18ff7c8 Increase maximum number of subjobs
REVERT: c9e632c Another fix for bug #41726
REVERT: b0debe2 add extOutFile
REVERT: ccc344e update to Ganga 5
REVERT: 4680621 update to Ganga 5
REVERT: cfe41a0 First import to new Ganga repository
REVERT: 8c25824 Now give more advice about what to check if there is an error when eval()'ing the output from gaudirun.py.
REVERT: 5913fe5 Can now pick up multiple ntuple output files.
REVERT: e6f1609 Small fix
REVERT: ac85443 avoid downloading again the output of the jobs failed in the previous downloading task
REVERT: 231ec0c Fix _USERDISK problem
REVERT: c2eb077 Remove forcing of outputlocation cloud and allow number_of_files again
REVERT: 00a02b9 Added PACKAGES-5.0.9
REVERT: 567cc89 Added ReleaseNotes-5.0.9
REVERT: 8edab2a deprecated slc3 platform and replaced with slc4_ia32_gcc34
REVERT: 58f4e43 bugfix #41372: added backend.remove() method to support job removal on the Remote backend
REVERT: 8dfd9bc updated configuration URL for monalisa usage monitoring server
REVERT: d219a7e deprecated slc3 plaftorm and replaced with slc4_ia32_gcc34
REVERT: b915e3e Traceback information added if eval() of options fails.
REVERT: 4ebb976 Bugfix. Removed [0] since the microDST name is not returned in a list, but just as a string.
REVERT: a488818 get_output_file now detects if the user wants to write a microDST file and adds it to the job outputsandbox.
REVERT: a3d0eec initial import of PandaPilot application.
REVERT: 88c0a27 Added PACKAGES-5.0.9
REVERT: df4182b Added ReleaseNotes-5.0.9
REVERT: aa8329c Small fix
REVERT: 01fc0fe fine tune the typelist attribute in the schema
REVERT: 35dc8be Small bug fix for local dataset merging
REVERT: 18afbbd Small fix
REVERT: f8f89c1 Add failover mode
REVERT: 7ac6644 Add failover
REVERT: 3a7d845 Add DQ2Dataset.failover
REVERT: 911d8a1 stage-out.sh: prevent script from creating incomplete output_data lines which are screwing up completion stage (jobs remain forever in completing state if the output_data lines are not complete)
REVERT: 762c82b change support line to new DA help forum
REVERT: 6a68465 Update to use dq2-get for tagdatasets
REVERT: 7da954a Fix workflow for dq2-get
REVERT: 2aabbb4 New DQ2_LOCAL_SITE_ID detection
REVERT: 734c7cc small fix for dq2client version requirement
REVERT: 48cd424 make dq2client_version configurable through AtlasLCGRequirements
REVERT: 3d8dad8 Switch off RA_BUFFER
REVERT: ec7c3cd replacing dq2_get with dq2-get
REVERT: 656a18c fix the type checking issue
REVERT: b8bf74f fix the type checking issue
REVERT: 6d41429 Fix workflow for DQ2_COPY mode
REVERT: bdf9bbd Fixed type checking for NG.py and NGRequirements.py
REVERT: 9b7cb2b Small fix
REVERT: 0311a36 Change dq2info.tar.gz unpacking
REVERT: bc0355f fix for type checking
REVERT: 4f877fc Fix DPM problem and add python32bin and set RA_BUFFER=32768
REVERT: baeeef8 New version
REVERT: dbb83a2 Add typelist=['type(None)','str']
REVERT: 8c453a9 introduce prepare_athena
REVERT: a7d41af remove dq2_get and use dq2-get instead in DQ2_COPY mode
REVERT: cb51345 Better protection for TiersOfATLAS.getSiteProperty(sitename,'srm') calls
REVERT: 1a819ef Fix submission problem with DQ2JobSplitter
REVERT: bfb16a1 Add typelist=['type(None)','int']
REVERT: f54c3d8 Remove obsolte DQ2Output class
REVERT: 653600b More bug fixing on space token treatment
REVERT: dc6f102 Updates the schema definitions for Ganga 5.0.9-pre. Just adds a few type(None)s to the allowed types.
REVERT: 9bb1568 fixing AthenaMCOutputDatasets startup warnings
REVERT: 020b2af - Bug fixes to the test Robot - Fixed bug in initialisation of TestRobot package which prevented a Ganga   session from starting the TestRobot.
REVERT: 321b304 Added PACKAGES-5.0.9
REVERT: 5a058f3 Added ReleaseNotes-5.0.9
REVERT: b0cb699 Combined the functionality in lcg and local scripts. Created new versions to aid rollback.
REVERT: b0293f3 Added resubmit and kill methods
REVERT: c04ce0c Print warning if srm entry is empty in TiersOfATLAS
REVERT: 1cd1310 Fix bux 42044 - drop used guids in DQ2JobsSplitter
REVERT: f1b11cf Disallow submission if number_of_files for incomplete dataset is used in splitting mode
REVERT: a1b5af8 include AMAAthena module
REVERT: c86364b update ama specific athena wrapper script
REVERT: 17fea47 Bug fix for DBRelease mecanism. User can now use any kind of DBRelease available as DQ2 dataset by setting application.extraArgs='DBRelease=X.Y.Z'. The job will then look for a matching DQ2 dataset and add it to the list of input datasets to be handled by the jobs
REVERT: c38bbe3 fix the status update logic
REVERT: a8a07fe commit the LCGOutputDownloader.py
REVERT: b362e8b cache the logging information coming out from the LCGOutputDownloader threads
REVERT: 9fd20b2 turn off debug message of the MTRunner objects try to avoid the race condition amoung concurrent threads
REVERT: 87c147a improving job submission/output fetching performance
REVERT: 6273933 fixed bug 41726
REVERT: b7be69e improve the MTRunner:  - allow dynamically inserting data to be processed  - allow daemon-mode MTRunner listening on the data queue
REVERT: 4b1b581 athena-lcg.sh: tentative fix for database lock by making copies of the DB flat files on the worker node. AthenaMC: bug fix to open support to LOCALGROUPDISK space token and better general handling of space tokens for writing
REVERT: 0e0a0f7 add SandboxCache implementation based on ATLAS DQ2Client
REVERT: 691905e improve sandbox cache handler and adopt it in the LCG backend
REVERT: 08428ab disabled a complex condition
REVERT: 2386849 Automatically updated template for 5.0.8
REVERT: f14b50e Added PACKAGES-5.0.8
REVERT: 539f612 Added ReleaseNotes-5.0.8
REVERT: a95e97b Automatically updated template for 5.0.8
REVERT: 9e3319e Added PACKAGES-5.0.8
REVERT: a34fa2d Added ReleaseNotes-5.0.8
REVERT: 664672e use DQ2 config usertag
REVERT: a750c7c temporarily disabled running Root tests on LCG and LSF (due to timeouts), a proper solution is needed for the future (either splitting the submit and verify phases, or to make a test backend that woudl exercise the rt-handlers or to move it to a generic robot).
REVERT: 1907720 Removed AthenaMCDatasets config block from Atlas.ini and modified AthenaMCDatasets.py to use DQ2 config block instead
REVERT: 1d88fc8 adds returns to the example jobs
REVERT: 6929a10 Add usertag config option and change DQ2OutputDataset name
REVERT: 8ebc45f initial implementation for extensible GridSandboxCache
REVERT: 0b86e71 Added PACKAGES-5.0.8
REVERT: dbf1dfa Added ReleaseNotes-5.0.8
REVERT: 9aafa4c Fix srmv1 site problem
REVERT: ad0d34a Added AthenaMCDatasets config member to upload userXX project tag from Atlas.ini. This tag is used in building output dataset names
REVERT: 92e3a29 fix for bug #40334
REVERT: a8d3b62 setup for loading pre-deployed dq2client on the WN
REVERT: fd2350d fixed: j.submit() and j.resubmit() do not return the status but raise exceptions on error
REVERT: bf9498c fixed typeinfo
REVERT: dcc8bdf bug #40696: Exception raised during resubmit() should be propagated to caller
REVERT: c963379 bugfix #40220: Ensure that default values satisfy the declared types in the schema
REVERT: 4e0fb1a #38646 bugfix: accept jobs.select(xrange(..))
REVERT: 0c25ba4 Automatically updated template for 5.0.7
REVERT: 618af97 Added ReleaseNotes-5.0.7
REVERT: a75a15a remove the fixpython magic function which has been merged into GangaAtlas package
REVERT: ba72e8d Another fix for PIC
REVERT: cd88a3e Automatically updated template for 5.0.7
REVERT: 85b4415 Added PACKAGES-5.0.7
REVERT: d20bf27 Added ReleaseNotes-5.0.7
REVERT: 442e25f less spammy status changes. (Only updateStatus when panda status has changed).
REVERT: f1d36f9 don't marked completed when build job finishes!
REVERT: b8e9d6a Automatically updated template for 5.0.7
REVERT: 88355d7 Added PACKAGES-5.0.7
REVERT: 6c8a268 Added ReleaseNotes-5.0.7
REVERT: e199512 XML differenciater added (from Ulrik)
REVERT: d65e026 A few more files missed in the first update
REVERT: b6e0e6f Fix a few bugs in test case comparisons and in the Test instance of the Robot
REVERT: 2c6bbc1 Automatically updated template for 5.0.7
REVERT: bf3fa11 Added PACKAGES-5.0.7
REVERT: d226019 fix bug in update
REVERT: a22fc7a removed file again as it will cause trouble with the way tester works at CERN.
REVERT: 23204c3 Removed file as it will cause trouble at CERN in current way test framework works.
REVERT: 10bc38c Automatically updated template for 5.0.7
REVERT: d7b35db Added PACKAGES-5.0.7
REVERT: 5b61d08 Added ReleaseNotes-5.0.7
REVERT: be49f69 Updated file to write to correct directory.
REVERT: 7292e1a Added file for testing at Imperial
REVERT: f6719e1 Added Imperial.ini file for the testing to happen at Imperial.
REVERT: 2213d80 Update of default INI file with more sensible parameters.
REVERT: 4e9ed6d Forgot to add Framework/xmldifferencer.py
REVERT: 14b44d1 New features: - Remote testing of Ganga thorugh the use of the GangaRobot - Ability to create test reports showing differences between releases.
REVERT: f4900c4 Automatically updated template for 5.0.7
REVERT: aad8d27 Added PACKAGES-5.0.7
REVERT: 8ac53d2 Added ReleaseNotes-5.0.7
REVERT: b95a7f4 Fix alternateName and PIC problem
REVERT: fd5f229 removed 'completing' state
REVERT: dec86f2 added unknown, starting panda statuses
REVERT: 2f62f41 Automatically updated template for 5.0.7
REVERT: 646d0cd Added PACKAGES-5.0.7
REVERT: 0364420 Added ReleaseNotes-5.0.7
REVERT: 7347f42 fix the type-checking issue when setting up CE attribute
REVERT: 1d9dec9 Add fixpython magic function
REVERT: 9f01ba9 Added PACKAGES-5.0.7
REVERT: 7640e9e Added ReleaseNotes-5.0.7
REVERT: 7491834 removed panda client
REVERT: b96a946 removed dq2_get
REVERT: cf3e1ca removed config extractor
REVERT: 3e6865d verify outDS starts with user08.DN.ganga. rewrite cloud in case user specified a 'site'
REVERT: c8cc476 Add machine and time info print outs
REVERT: 11ae7f0 Small fix
REVERT: 5293a31 Add machine and time info print outs
REVERT: 2dbcae5 Small fix
REVERT: 8125fa3 Add cleaner error message
REVERT: 513622e Remove tabs
REVERT: 3d463e8 Add glite UI setup
REVERT: a18ab0d Remove gsidcap for UVIC
REVERT: 0596393 Use external PandaTools Added cloud Removed useless dq2_get and getQueue EXPERIMENTAL: Added resubmission:     job(x).resubmit() will resubmit the _failed_ subjobs to Panda. Removed useless gridshell Cleaned up status update function
REVERT: c54fbae Use external PandaTools Change outDS to user08... Add brokering with PandaTools runBrokerage to find best site in cloud
REVERT: 63d3740 Make PandaTools an external dep
REVERT: 573cc1a add a magic function for fixing python conflict between ATLAS and LCG
REVERT: 0ee64b9 resolve the TMPDIR issue on LSF machines at CERN  - take WORKDIR instead of TMPDIR for job's scratch dir.
REVERT: a073d45 Savannah 40910 - Also problem with datasets in options files
REVERT: ae5ed9e Convert to unix and remove tabs
REVERT: 82ecfd1 Update to 0.1.20 DQ2Clients
REVERT: 75854eb bug fix #40268
REVERT: d2c60fc Add gsidcap access rule for UVIC
REVERT: 207b50b Athena: * Fix SE type detection problems for space tokens at DPM sites * Change default DQ2 stage-out to use _USERDISK token * Change default DQ2Dataset default values of   DQ2_BACKUP_OUTPUT_LOCATIONS * Disable functionality of DQ2Dataset.match_ce_all and   DQ2Dataset.min_num_files and print out warning.   These are obsolete and DQ2JobSplitter should be used instead. * Add option config['DQ2']['USE_STAGEOUT_SUBSCRIPTION'] to allow DQ2   subscription to final output SE destinations instead of "remote lcg-cr"   Will be enabled in future version if DQ2 site services are ready for this ============================================= A few changes to enforce the computing model: ============================================= * DQ2OutputDataset: j.backend.location is verified to be in the same cloud   as j.backend.requirements.cloud during job submission to LCG * Add AtlasLCGRequirements.cloud option.   Use: T0, IT, ES, FR, UK, DE, NL, TW, CA, (US, NG) or        CERN, ITALYSITES, SPAINSITES, FRANCESITES, UKSITES, FZKSITES,        NLSITES, TAIWANSITES, CANADASITES, (USASITES, NDGF)
REVERT: 8f20a44 updated athenamc with proper types for j.application.firstevent and j.application.number_events_job
REVERT: 8aaa6ed Gave a type and a default value to app.number_events_job
REVERT: a3edfb9 set default DQ2_LOCAL_SITE_ID to CERN-PROD_DATADISK
REVERT: 0269b27 introduce AMAAthena module into GangaAtlas package
REVERT: 0ecb06f Savannah #40903 - Update to use same mechanism as Ulrik with the Gaudi apps. Tests still pass.
REVERT: e019c2b Savannah #40903 - Submission of unsupported architestures for root jobs. Now the architecture is taken from the ROOT config section of the gangarc file. Also adds test.
REVERT: eae4fc4 Test for Savannah 40910. Waiting for options file from Mitesh to verify that results are not stupid.
REVERT: 240dc76 Savannah 40527: Updates the default root version and architecture to solve a failing test. Should only really affect the generic tests.
REVERT: 796def2 fix for Savannah 40219 - Manages cache updating better.
REVERT: fdf2a29 Modified failing test cases.
REVERT: bd969ea Added PACKAGES-5.0.6
REVERT: e5806f8 Added ReleaseNotes-5.0.6
REVERT: 92a336f Fix _color problem
REVERT: d079208 fixed problems found in 5.0.6-pre:
REVERT: e13384a Automatically updated template for 5.0.6
REVERT: f9627d2 Added PACKAGES-5.0.6
REVERT: 6403200 Added ReleaseNotes-5.0.6
REVERT: 6f92de4 removed python 2.3 dependency (not needed anymore)
REVERT: db78d5b New features: ============= The Gaudi and GaudiPython applications have a new attribute called 'setupProjectOptions'. It contains extra options to be passed onto the SetupProject command used for configuring the environment. As an example setting it to '--dev' will give access to the DEV area. For full documentation of the available options see https://twiki.cern.ch/twiki/bin/view/LHCb/SetupProject. The 'lhcb_release_area' attribute has been taken away as it was not useful.
REVERT: 34a027b Removed python dependency
REVERT: 14246a6 remove python2.3 dependency
REVERT: 17abfe0 Bug fix to avoid problems in tcsh
REVERT: fd26335 Work around for #30882
REVERT: 015b125 Fix bug #40269, ATLAS_PRODUCTION environment variable is now handled properly in Local/Batch backend
REVERT: 7c047f2 Fix bug #39010, Add ATLASLocalDataset support for list of datasets
REVERT: 4784ef8 Introduced exceptions for the Robot framework.
REVERT: 6f5ea81 jobs.select([ids]) added
REVERT: 6d24851 Updates to LHCb manual
REVERT: ef0abbf added force_status() method to replace job.fail(), force_job_failed() and force_job_completed()
REVERT: 7911507 Now raise ApplicationConfigurationError if there is an error when parsing the job options.
REVERT: 947a7b4 bugfix 40110
REVERT: 7a4be46 *** empty log message ***
REVERT: 2590f58 Changed the Dirac splitter to work with the modified LHCbDataset
REVERT: a576f39 Test case added
REVERT: e700cde Added "datatype_string" to schema for LHCbDataset. This allows Ganga to run with cosmic data.
REVERT: 65acae5 Automatically updated template for 5.0.5
REVERT: 8ccad8a Added PACKAGES-5.0.5
REVERT: 660df55 Added ReleaseNotes-5.0.5
REVERT: ef75c25 Small fix to version number of updated manual
REVERT: 8b49b31 Updated LHCb manual
REVERT: 7fde1a6 Fixed gaudiPython to work with splitters
REVERT: 97b162c  - remove redundant functions  - set minimum timeout of downloading oversized inputsandbox to 60 secs.
REVERT: 0748e0b - improving oversized inputsandbox downloading   * add more debug information   * automatically determine the lcg-cp timeout assuming the rate of 1MB/sec   * add config.LCG.SandboxTransferTimeout allowing user to set it manually
REVERT: cfba14d Fixed bug when detecting the source of inputdata. Improved logging messages.
REVERT: f3f0654 Added configuration option AllowedPlatforms to the DIRAC section. Only jobs of this configuration will be allowed for Dirac submission. At the moment defaults to just slc4_ia32_gcc34.
REVERT: a98800e  AthenaMCLCGRTHandler.py: extend lfc patch to cavern and minbias input data. stage-in.sh: fixing a typo in previous bug fix
REVERT: 9b68e05 optsfiles can now be specified as a string rather than a list if only single file required.
REVERT: 35435e0 Added a GaudiPython application handler with runtime handlers for Local/Batch and for DIRAC.
REVERT: 3731ed8 Bug fix for input lfcs: applying single quote prescription which is working for the other input arrays (lfns and turls). Also, force stage-in.sh to use all the lfcs in the array instead of just the first one
REVERT: ff94690 AthenaMCLCGRTHandler.py: Adding protection against writing output data to dedicated production endpoint. stage-out.sh: added -s flag to write to ATLASUSERDISK space token when the destination (se_name) is set to this kind of endpoint
REVERT: 1720df9 Automatically updated template for 5.0.4
REVERT: 0152f75 Extended _get_user_dlls to pick up all relevant files under InstallArea/python in the users private project areas. This directory structure is now replicated in the job inputsandbox. PYTHONPATH in the job wrappers is modified to prepend it with `pwd`/python so that the python configurables in the input sandbox are picked up by the job.
REVERT: 9142617 Added ReleaseNotes-5.0.4
REVERT: 8603a36 build external tarballs for slc3_ia32
REVERT: 2031fb4 paramiko syspath fix
REVERT: 63a2bb8 Added PACKAGES-5.0.4
REVERT: ba4ab1e Added ReleaseNotes-5.0.4
REVERT: 3cf9d74 bugfix: #39655: Problem with slice operations on templates
REVERT: 6b52afe pycrypto sys.path fix for both 32 and 64 platforms
REVERT: cc2f2c4 fix output_location to handle srmv2 space tokens. This allows to write data files to srmv2 space tokens. Restriction to USER space tokens will be implemented when a critical mass of these dedicated endpoints will be reached
REVERT: af010a8 Automatically updated template for 5.0.4
REVERT: 76f1ac6 Added PACKAGES-5.0.4
REVERT: 643de8d Added ReleaseNotes-5.0.4
REVERT: c6bf1e7 Merged the new Gaudi application handler from branch
REVERT: 059f2a4 Automatically updated template for 5.0.4
REVERT: 3dadfd6 Fixed a bug that caused new classes not to be found by gaudirun if a non-default CMT area was used.
REVERT: 86513a4 typo fix
REVERT: 2643298 Automatically updated template for 5.0.4
REVERT: 5907eb0 disabled tarball creation for scl3 (obsoleted)
REVERT: 1a926d5 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 3d72aae GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 91e5ba2 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: d478b96 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: d2d1a77 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 96928ab GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 599f613 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 8724258 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 832efe4 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 9cf8c3d GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: d15c0ed GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: eb76d3f GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: a33a97c GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 96db0aa GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: bc5a0dc GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 83db800 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 88faeab GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 4785033 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: fc438d8 GOING UP TO RELEASE 1.19 (CVS REPOSITORY CHANGE SIDE EFFECT)
REVERT: 5453282 Added PACKAGES-5.0.4
REVERT: 2499752 Added ReleaseNotes-5.0.4
REVERT: 186a12b Updates related to job slices.
REVERT: 2459e70 added pycrypto and paramico to the list of external dependencies (for Remote backend)
REVERT: e208638 Remove enforcing of DQ2OutputDatatset loction
REVERT: 5a7d0c7 Automatically updated template for 5.0.3
REVERT: 82de9b4 config templates are now in a separate directory at top level ("templates")
REVERT: 3a14fe1 config templates are now in a separate directory at top level ("templates")
REVERT: 0397898 Fix but #39568: Panda/DQ2Dataset: Need better error message if DS doesn't exist
REVERT: 771f14d Updates and Bugfixes
REVERT: dec81e2 AthenaMC.py: added type checks to members. AthenaMCDatasets: bug fix for treatment of DQ2 datasets available only for Panda. AthenaMCLCGRTGandler.py and wrapper.sh: extension of the glite WMS workaround to input data
REVERT: a730f36 Fix bug #39549: raise execption if (athena_compile==True) and (NG==True)
REVERT: 7328bb8 Change DQPing to utils.ping
REVERT: 19c3675 Change DQConstants to constants
REVERT: e3aff7a add timeouts of 300,1200,2400s for lcg-cr stage-out
REVERT: dc51c15 fix indentation issue in the code
REVERT: 442820c Print out warning to use jobs(id).subjobs instead of subjobs
REVERT: d56e060 Debug print-outs during compilation
REVERT: 1efba44 cvs commit enabled (but does not work, TEMPLATES should be moved to a subdirectory which is always on HEAD).
REVERT: f160e90 commit config template files
REVERT: 64557bb Add  AthenaRemoteRTHandler class for the Remote backend
REVERT: 83e2519 Removed some remaining info messages
REVERT: a7a4b68 Updates and improvements
REVERT: 1c5d263 Remove DQ2_OUTPUT_LOCATIONS again
REVERT: 7e29d4c Upgrade to ARC 0.6.3
REVERT: d4d711b Added backup atlasgiis server, fixed autonaming of jobs with DQ2 datasets
REVERT: 95d67bc Changed configuration section for Robot to be open to allow new actions to be defined.
REVERT: d76077c Small fix
REVERT: 48cb680 * ganga-stage-in-out-dq2.py:   - Add fix for DPM setup for NIKHEF and SARA   - remove special SE setup for RAL   - add MPPMU defaultSE=lcg-lrz-se.lrz-muenchen.de   - add DQ2_BACKUP_OUTPUT_LOCATIONS reading   - add DQ2_OUTPUT_SPACE_TOKENS reading   - change stage-out order for DQ2OutputDataset:   (1) j.outputdata.location, (2) DQ2_OUTUT_LOCATIONS,   (3) DQ2_BACKUP_OUTPUT_LOCATIONS (4) [ siteID ]
REVERT: 8e4ec10 Ensure curl is installed when loading Panda
REVERT: b745abc list_sites now gets from Panda server
REVERT: b67f0fd update from upstream
REVERT: 043d763 Fill subjobs DQ2 datasetname with auto-gen name
REVERT: 586a60a update from upstream
REVERT: c2549b4 Fix for bug #35256
REVERT: d58fc0a Add Remote backend submission to AthenaLocalRTHandler
REVERT: c6fd64f * Upgrade to DQ2Clients 0.1.17 and DQ2 API * Add full support for DQ2 container datasets in DQ2Dataset * Change in DQ2OutputDataset.retrieve(): use dq2-get * Fix bug #39286: Athena().atlas_environment omits type_list
REVERT: 387ca79 Updated the Remote backend that include:
REVERT: e7a419a patching up to the up-to-date development after CVS migration
REVERT: b671a87 Adds support for merging dst files with DaVinci
REVERT: 1ab3f64 adds support for root > 5.16.00
REVERT: c1d0453 adds a CustomMerger
REVERT: 971152c adds a CustomMerger
REVERT: fc18ffd Added ReleaseNotes-5.0.3
REVERT: 7e5f103 Added PACKAGES-5.0.3
REVERT: fa0d3fc Added ReleaseNotes-5.0.3
REVERT: 97e8ad6 Added PACKAGES-5.0.3
REVERT: a442b03 Added ReleaseNotes-5.0.3
REVERT: 50ae857 migration of 5.0.2 to HEAD

git-subtree-dir: src/Ganga
git-subtree-split: c7a261ee32a484456ebdfc520a75fbca77a15ddc
Resync Ganga 5.7.7 for git svn
Squashed 'src/Ganga/' changes from c7a261e..c27e25c

c27e25c small import fix
0b7d95c use fnmatch module for automatic file type detection
6ab4379 modofied comments
8b71338 modified comments
a5256ac Minor updates to ITasks
d70596d Changes required to implement prepare application checksum. See feature request #93682
549d79d Just adding a docstring
c43f3e6 Minor tweak for preparable application hash calculations
15cca4f fix for savannah 50217
e35231d more fixes for savannah 76289
36e8263 (ganga@7404)
96f3746 fix for savannah 76289
b7443d4 This attribute should be hidden from the user
d8a499f Implementation of MD5 checksumming for preparable applications; currently beta-testing for the Executable() app.
58ed828 calc_hash() method for checksumming prepared applications; improved error checking during prepare() method
6b7c6de (ganga@7391)
b1ba0a6 Improve error checking in prepare() method; add unprepare() method
dd12d22 refactoring of output handling
997fe24 refactoring output postprocessing
f734d9b (ganga@7387)
efb618e (ganga@7386)
0d915cc (ganga@7383)
a7dd9bc refactoring
35bc213 refactoring
d0b69c8 (ganga@7378)
86d6b96 (ganga@7377)
b1493d4 refactoring
9786ee7 refactoring
6fa3ad0 removed config section
f54567b (ganga@7373)
79ac5ac (ganga@7372)
8083bc1 (ganga@7371)
4f730e6 changes for compression
1dd22cf changes for output file compression
8540ba3 (ganga@7368)
66a0b99 added comment
b70a731 (ganga@7366)
a643755 (ganga@7365)
980bca0 (ganga@7364)
fcda0bc (ganga@7363)
52a70e3 (ganga@7362)
6084a04 (ganga@7361)
f625740 (ganga@7360)
b6bfa61 (ganga@7359)
af2e8c3 handle the option for compression of the output file
ac88a71 Minor improvement to shareref decrement method
4735b45 Removed unnecessary Atlas related imports
4e9d450 Added subjob status to job metadata for Tasks
f3628a3 Beta upload of revamped Tasks system
03b4118 changed configuration retrieval
6b20008 removed commented code
2873980 SharedItem is now redundant
ff519e8 change in mass storage configuration retrieval
ae7cfa1 change in mass storage configuration retrieval
1cfd7e9 refactoring of output files configuration
f7065c3 (ganga@7331)
2adbf13 Improvements to shareref.rebuild() method and inclusion of shareref.lookup()
4f69646 Force closedown of prepared repository before others
6fc042a fix for savannah item 93085
0adf036 Changing the default setting of deleteUnusedShareDir to 'always'
3959036 Add mechanism to interactively determine whether unreferenced ShareDirs should be deleted upon Ganga exit
b0c4ea0 small config fixes
eccca59 removed not used line
c577a03 merging configurations
9e4d2bb Add ability to rebuild the shareref table from the GPI
204a8a9 Removing redundant lines
5aabed3 load backend_output_postprocess dictionary from configuration
8133e4b minor fix
1547abf changed configuration for output files
3b05f67 Final change for GangaList fix
9ca5936 No real usecase at the moment to allow assignment of ShareDirs to is_prepared from within the GPI
2cace5f Locking of GangaLists in prepared state
ef676ac Code tidy-up
437011d Changes to effect feature request #90360, including new tests
56a945a Allow us to attach ShareDirs to the is_prepared attribute
33eee5c Allow us to attach ShareDirs to the is_prepared attribute
5c98bea refactoring output handling
6c249f2 output refactoring
61b3add refactoring output handling
3fc5a9b refactoring output handling
deb2b4e output refactoring
e9b9e37 (ganga@7240)
a0b5d52 (ganga@7239)
42d20ec new methods implemented
a3602cd (ganga@7237)
cf2316d (ganga@7236)
23397f7 (ganga@7235)
713b660 (ganga@7234)
46f7587 (ganga@7233)
a007896 (ganga@7232)
d3b0d91 (ganga@7231)
cce1058 (ganga@7230)
2ef0864 (ganga@7229)
1a3e1d6 (ganga@7228)
d301c67 (ganga@7227)
c45cd52 (ganga@7226)
e0d2595 (ganga@7225)
afedd34 (ganga@7224)
ee3e412 (ganga@7223)
5e5addb (ganga@7222)
8f098d3 (ganga@7221)
41a1ac8 (ganga@7220)
858614b (ganga@7219)
b3f1fb0 (ganga@7218)
01b25eb (ganga@7217)
9090dc1 (ganga@7216)
fc1bc5f changes needed due to CREAM backend strange behaviour
685d907 small fixes
5349076 (ganga@7213)
0d174da (ganga@7212)
af21d8d (ganga@7211)
f593cb6 (ganga@7210)
36f3efb (ganga@7209)
6a35919 refactoring of output postprocessing
ec3a364 refactoring output handling
4661f48 fix
cd9e507 refactoring output postprocessing
d34a926 refactoring output postprocessing
4acbbac bug fixes
3027c20 bug fixes
59e588f bug fixes
9a72117 small bug fix
ba6e136 refactoring of output processing
05e30a5 output processing refactoring
87dd6aa output processing refactoring
4c90c9c (ganga@7189)
a8493e2 small fix
4c9ee54 fix
b65d27c refactoring
1934bf6 output processing refactoring
462f03f (ganga@7184)
e831ec6 small fix
ec5ac16 output handling refactoring
58f4096 outputhandling refactoring
f109473 refactoring
211c9fe refactoring
ec921f8 Bugfix #92233; remove hardcoded ~/gangadir
c8fecee Fix for #88166; provide a user-configurable option to allow/prevent resubmission of subjobs with job.resubmit().
b3e44cf updated backend_output_postprocess dict
f2ac216 refactoring
bad2543 removed debug info
c02e632 added debug info
3213c85 removed comments
a1eabde some imports
8a6346b refactoring of the output postprocessing
470c373 added file that will provide plugin function for output postprocessing
4a02eea refactoring of the output processing
f39643b refactoring of the output handling
85f4cd4 Removing restriction of direct subjob submission
5b6259e changes for the dictionary defining where should output file processing take place - WN or client
2a8bbb5 Fix for bug #91840
7a86d7e Code tidying and fix bug preventing shareref table from finding ShareDirs
abfa64d remove commented code
0cded1e removed not needed code
74b7b59 removed commented code
c4e4b28 removed commented code
476eb4f remove commented code
80138bd remove not needed code
9f71b31 smalll fix
adaa225 (ganga@7127)
3f07dbc removed backend dependent code for post processing on the client, put it in the Job class
3f57221 removed backend dependent code for post processing on the client
f8c3762 removed backend dependent code for post processing on the client
8dd0ed6 removed backend dependent code for post processing on the client
6d30b79 removed backend dependant code for post processing on the client
10550a1 rename postprocess file
f709a38 rename postprocess file
f87354b refactoring
1f32566 refactoring
dd14f01 (ganga@7110)
0ffb0ef added put method
7dc51f2 move the code for uploading to mass storage in the mass storage file object itself
3df7119 refactoring
9492fba implemented the put method
4ce4efa code refactoring
ae3f9a2 small changes
f5807f4 added put method
123880a added methods
2767c47 change in schema
38c775e use the put method of the output files for upload from the client
3acc999 (ganga@7097)
43762de set default values
2812b71 added debugging info
e85e0d6 renaming attribute
1ca3a78 renaming attribute
e69c9a1 removed file
1ea9ce1 removed not needed imports
4fd95b8 changes in schema
b8227e7 file removed
255232b file removed
01172fa file removed
04d55c0 added default output file type - sending to sandbox
4bb5dfa removed not needed imports
fcb0cf9 changes in schema
d0c629c removed not needed config sections
61d2add Bug fix for #91570; ShareDir is hard coded, and not relative to Gangadir
a8d5172 changed the script to work also for Ganga Core manual
11f0dfa fix for savannah item 71129
444bb47 remove subprocess and tarfile from the tarbal in the inputsandbox, fix for savannha item 73323
a023df1 fixed comments
7488456 small fix for output files

git-subtree-dir: src/Ganga
git-subtree-split: c27e25c965eb5aecef736d358c2239ff012cfecc
Update to Ganga 5.8.0