DIRACGrid/DIRAC — Issue #409: Problem in InputDataResolution
Author: phicharp
Created: 2012-01-10 17:26:14+00:00
State: open

As a result of investigations on the crashes of merging jobs that have one file present in two SEs at a site (-DST and -FAILOVER), I found out that the JobWrapper is not looking for active replicas. In addition there is a fix to be made in InputDataResolution in the case a file is on two disk SEs at the same site.

The InputDataResolution class is called with the replicas already resolved in arg['FileCatalog']. This resolution is done in... JobWrapper.py (!):
def __getReplicaMetadata( self, lfns ):
""" Wrapper function to consult catalog for all necessary file metadata
and check the result.
"""
start = time.time()
repsResult = self.rm.getReplicas( lfns )

This is incorrect as it should be getActiveReplicas(). Then the logic is screwed up by the fact that there are two replicas on disk at the site, which is clearly not foreseen, and the bug is at line 142 of InputDataByProtocol:
for se, lfnDict in seFilesDict.items():
pfnList = lfnDict.values()
result = self.rm.getStorageFileAccessUrl( pfnList, se, protocol = requestedProtocol )
where for se='xxx-FAILOVER', lfnDict is empty, therefore getStorageFileAccessUrl is called with an empty list.
It is however not sufficient to protect it, as the SE that is kept is essentially random (it is the one that has the largest number of replicas for the input data...). If only active replicas are selected in the job wrapper, probably a random selection is not worse than any other...