DIRACGrid/DIRAC — Issue #261: remove replicas not registered in FC
Author: elisal
Created: 2011-11-04 09:51:16+00:00
State: open

Hi, I modified the dirac-dms-remove-lfn-replica script, adding an option: FCCheck. By def. it is YES, so it checks the replica's existence in the LFC, and behaves exactly like the current version.
If the option is set to NOLFC, then it does:
-checks in any case if the replica is registered, if yes it doesn't remove anything
-if the replica is NOT registered, then it calls:
StorageElement.getPfnForLfn() to get the surl,
and then it calls ReplicaManager.removeStorageFile( surl, seName )

Examples:
[lxplus423] > srmls srm://srm-lhcb.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/test/roberto/temp/SARA_5.13778
5908 /castor/ads.rl.ac.uk/prod/lhcb/test/roberto/temp/SARA_5.13778
this file exists on storage and it is not registered in LFC:
> dirac-dms-lfn-replicas /lhcb/test/roberto/temp/SARA_5.13778
{'Failed': {'/lhcb/test/roberto/temp/SARA_5.13778': 'No such file or directory'},
'Successful': {}}

then the script with the NOLFC option will remove it:
[hpdesk] > dirac-dms-remove-lfn-replica /lhcb/test/roberto/temp/SARA_5.13778 RAL-DST NOLFC
WARNING: removing physical replica from storage, without removing entry in the FC
ReplicaManager.__executeReplicaStorageElementOperation: Failed to get replicas for file. /lhcb/test/roberto/temp/SARA_5.13778 No such file or directory
ReplicaManager._executeStorageElementFunction: No pfns supplied.
ReplicaManager.__executeReplicaStorageElementOperation: Failed to execute isFile StorageElement operation. ReplicaManager._executeStorageElementFunction: No pfns supplied.
Summary:
Successfully removed: ['srm://srm-lhcb.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/test/roberto/temp/SARA_5.13778']
Failed to remove: []

and in fact the file has been removed:
> srmls srm://srm-lhcb.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/test/roberto/temp/SARA_5.13778
Fri Nov 04 10:20:04 CET 2011: Return status:
- Status code: SRM_FAILURE
- Explanation: Failed for all paths
SRM_INVALID_PATH File/directory 0 /castor/ads.rl.ac.uk/prod/lhcb/test/roberto/temp/SARA_5.13778 does not exist.

on the other side, if I execute the script with the NOLFC option for a replica that IS REGISTERED in LFC, the script will refuse to remove it. E.g.
> dirac-dms-lfn-replicas /lhcb/user/l/lanciott/apiEx.py
{'Failed': {},
'Successful': {'/lhcb/user/l/lanciott/apiEx.py': {'CERN-USER': 'srm://srm-lhcb.cern.ch/castor/cern.ch/grid/lhcb/user/l/lanciott/apiEx.py',
'SARA-USER': 'srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lhcb/user/l/lanciott/apiEx.py'}}}

I try to remove it with the NOLFC option but the script says no, it can't be removed only from storage:
[hpdesk] /home/elisal/dev > dirac-dms-remove-lfn-replica /lhcb/user/l/lanciott/apiEx.py CERN-USER NOLFC
WARNING: removing physical replica from storage, without removing entry in the FC
WARNING: file is registered in FC! it will NOT be removed from storage! {'OK': True, 'Value': {'Successful': {'/lhcb/user/l/lanciott/apiEx.py': True}, 'Failed': {}}}
Summary:
Successfully removed: []
Failed to remove: []

and in fact the replica of CERN-USER is still there :
> dirac-dms-lfn-replicas /lhcb/user/l/lanciott/apiEx.py
{'Failed': {},
'Successful': {'/lhcb/user/l/lanciott/apiEx.py': {'CERN-USER': 'srm://srm-lhcb.cern.ch/castor/cern.ch/grid/lhcb/user/l/lanciott/apiEx.py',
'SARA-USER': 'srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lhcb/user/l/lanciott/apiEx.py'}}}

I think the functionality is fine. I will do some more tests and then if there are no objections I will issue a pull request.
code is here:
https://github.com/elisal/DIRAC/blob/removeUnregisteredRepl/Interfaces/scripts/dirac-dms-remove-lfn-replica.py
cheers