Improve algorithm to retrieve outputs and logs from temporary or permane...

...nt storage.
being able to peek at files in temporary storage (even copy them) before
ASO is done was a requirement. I think the proper fix is to better manage
the file metadata, whatever they are, not to remove the feature.
Or have I misunderstood ?
stefano

On 09/06/2014 09:23 AM, Andres Tanasijczuk wrote:
> In src/python/CRABInterface/HTCondorDataWorkflow.py:
>
>>
>>          statusRes = self.status(workflow, userdn, userproxy)[0]
>>
>> -        transferingIds = [x[1] for x in statusRes['jobList'] if x[0] in ['transferring', 'cooloff', 'held']]
>> -        finishedIds = [x[1] for x in statusRes['jobList'] if x[0] in ['finished', 'failed']]
>> -        return self.getFiles(workflow, howmany, jobids, ['LOG'], transferingIds, finishedIds, tm_user_dn, tm_username, tm_user_role, tm_user_group, saveLogs=tm_save_logs, userproxy=userproxy)
>> +        if savelogs:
>> +            transferingIds = [x[1] for x in statusRes['jobList'] if x[0] in ['transferring', 'cooloff', 'held']]
>
> This is actually useless. The idea of having this list of transferring jobs is because for those
> jobs we could retrieve the files from the temporary storage (since they were not transferred yet).
> But until the transfer doesn't finish we don't have the file metadata available, so the "crab
> getoutput" command can anyway not retrieve the output.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/dmwm/CRABServer/pull/4396/files#r17207444>.
>

-- 
Stefano Belforte - I.N.F.N.     tel   : +39 040 375-6261 (fax: 375-6258)
Area di Ricerca - Padriciano 99 tel mobile: +39 328 010 7327
34012 TRIESTE TS  - Italy       AIM: stefanobelforte