DIRACGrid/DIRAC — Issue #335: Popularity service
Author: elisal
Created: 2011-11-30 11:44:15+00:00
State: open

A popularity service should be implemented in order to have an index of how popular datasets are, and establish a ranking. The objective it to identify the least used data and target them for removal (or reduction of replicas).
CMS and ATLAS experiences have given interesting inputs.
Before starting to work on the task, got a better understanding of the Dirac WMS. Summarized my understanding in a doc. (but I don't know if I can attach it to this task..).
First proposal for the implementation:
-implement a new db in WMS system (PopularityDB) with a table where each entry represent one data access operation for a given file. A first attempt is already on volhcb12.
-in the InputDataAgent.CheckJob method, after the data resolution has been done (i.e. after having called the __resolveInputData and the __checkActiveSEs) insert an entry into the new PopularityDB table, with the LFN, JobId, Site and other relevant data.
This would be just a prototype, to understand what can be the best design for this service.
The new modules for WMS/PopularityDB and the PopularityHandler are on volhcb12, I think port 9164 is free and can be used for this. I will ask Joel if he can open it and then will add the URL in the CS, and test it.
Any feedback is welcome.
thanks