The IceProd framework: Distributed data processing for the IceCube neutrino observatory
IceProd is a lightweight distributed workflow management framework. Uses existing middleware and protocols. Runs at user-level and is easily adaptable to other applications. It has been successful in managing 450k cores across 25 computing centers. Identified areas of improvement including scalability and load balancing.
data management grid computing monitoring distributed computing