The IceProd Framework: Distributed Data Processing for the IceCube Neutrino Observatory
Creators: name: Aartsen, M. G.
name: Abbasi, R.
name: Ackermann, M.
name: Adams, J.
name: Aguilar, J. A.
name: Ahlers, M.
name: Altmann, D.
name: Arguelles, C.
name: Auffenberg, J.
name: Bai, X.
name: Baker, M.
name: Barwick, S. W.
name: Baum, V.
name: Bay, R.
name: Beatty, J. J.
name: Tjus, J. Becker
name: Becker, K. -H.
name: BenZvi, S.
name: Berghaus, P.
name: Berley, D.
name: Bernardini, E.
name: Bernhard, A.
name: Besson, D. Z.
name: Binder, G.
name: Bindig, D.
name: Bissok, M.
name: Blaufuss, E.
name: Blumenthal, J.
name: Boersma, D. J.
name: Bohm, C.
name: Bose, D.
name: Böser, S.
name: Botner, O.
name: Brayeur, L.
name: Bretz, H. -P.
name: Brown, A. M.
name: Bruijn, R.
name: Casey, J.
name: Casier, M.
name: Chirkin, D.
name: Christov, A.
name: Christy, B.
name: Clark, K.
name: Classen, L.
name: Clevermann, F.
name: Coenders, S.
name: Cohen, S.
name: Cowen, D. F.
name: Silva, A. H. Cruz
name: Danninger, M.
name: Daughhetee, J.
name: Davis, J. C.
name: Day, M.
name: De Clercq, C.
name: De Ridder, S.
name: Desiati, P.
name: de Vries, K. D.
name: de With, M.
name: DeYoung, T.
name: Díaz-Vélez, J. C.
name: Dunkman, M.
name: Eagan, R.
name: Eberhardt, B.
name: Eichmann, B.
name: Eisch, J.
name: Euler, S.
name: Evenson, P. A.
name: Fadiran, O.
name: Fazely, A. R.
name: Fedynitch, A.
name: Feintzeig, J.
name: Feusels, T.
name: Filimonov, K.
name: Finley, C.
name: Fischer-Wasels, T.
name: Flis, S.
name: Franckowiak, A.
name: Frantzen, K.
name: Fuchs, T.
name: Gaisser, T. K.
name: Gallagher, J.
name: Gerhardt, L.
name: Gladstone, L.
name: Glüsenkamp, T.
name: Goldschmidt, A.
name: Golup, G.
name: Gonzalez, J. G.
name: Goodman, J. A.
name: Góra, D.
name: Grandmont, D. T.
name: Grant, D.
name: Gretskov, P.
name: Groh, J. C.
name: Groß, A.
name: Ha, C.
name: Ismail, A. Haj
name: Hallen, P.
name: Hallgren, A.
name: Halzen, F.
name: Hanson, K.
name: Hebecker, D.
name: Heereman, D.
name: Heinen, D.
name: Helbing, K.
name: Hellauer, R.
name: Hickford, S.
name: Hill, G. C.
name: Hoffman, K. D.
name: Hoffmann, R.
name: Homeier, A.
name: Hoshina, K.
name: Huang, F.
name: Huelsnitz, W.
name: Hulth, P. O.
name: Hultqvist, K.
name: Hussain, S.
name: Ishihara, A.
name: Jacobi, E.
name: Jacobsen, J.
name: Jagielski, K.
name: Japaridze, G. S.
name: Jero, K.
name: Jlelati, O.
name: Kaminsky, B.
name: Kappes, A.
name: Karg, T.
name: Karle, A.
name: Kauer, M.
name: Kelley, J. L.
name: Kiryluk, J.
name: Kläs, J.
name: Klein, S. R.
name: Köhne, J. -H.
name: Kohnen, G.
name: Kolanoski, H.
name: Köpke, L.
name: Kopper, C.
name: Kopper, S.
name: Koskinen, D. J.
name: Kowalski, M.
name: Krasberg, M.
name: Kriesten, A.
name: Krings, K.
name: Kroll, G.
name: Kunnen, J.
name: Kurahashi, N.
name: Kuwabara, T.
name: Labare, M.
name: Landsman, H.
name: Larson, M. J.
name: Lesiak-Bzdak, M.
name: Leuermann, M.
name: Leute, J.
name: Lünemann, J.
name: Macías, O.
name: Madsen, J.
name: Maggi, G.
name: Maruyama, R.
name: Mase, K.
name: Matis, H. S.
name: McNally, F.
name: Meagher, K.
name: Merck, M.
name: Merino, G.
name: Meures, T.
name: Miarecki, S.
name: Middell, E.
name: Milke, N.
name: Miller, J.
name: Mohrmann, L.
name: Montaruli, T.
name: Morse, R.
name: Nahnhauer, R.
name: Naumann, U.
name: Niederhausen, H.
name: Nowicki, S. C.
name: Nygren, D. R.
name: Obertacke, A.
name: Odrowski, S.
name: Olivas, A.
name: Omairat, A.
name: O'Murchadha, A.
name: Paul, L.
name: Pepper, J. A.
name: Heros, C. Pérez de los
name: Pfendner, C.
name: Pieloth, D.
name: Pinat, E.
name: Posselt, J.
name: Price, P. B.
name: Przybylski, G. T.
name: Quinnan, M.
name: ädel, L. R
name: Rae, I.
name: Rameez, M.
name: Rawlins, K.
name: Redl, P.
name: Reimann, R.
name: Resconi, E.
name: Rhode, W.
name: Ribordy, M.
name: Richman, M.
name: Riedel, B.
name: Rodrigues, J. P.
name: Rott, C.
name: Ruhe, T.
name: Ruzybayev, B.
name: Ryckbosch, D.
name: Saba, S. M.
name: Sander, H. -G.
name: Santander, M.
name: Sarkar, S.
name: Schatto, K.
name: Scheriau, F.
name: Schmidt, T.
name: Schmitz, M.
name: Schoenen, S.
name: Schöneberg, S.
name: Schönwald, A.
name: Schukraft, A.
name: Schulte, L.
name: Schultz, D.
name: Schulz, O.
name: Seckel, D.
name: Sestayo, Y.
name: Seunarine, S.
name: Shanidze, R.
name: Sheremata, C.
name: Smith, M. W. E.
name: Soldin, D.
name: Spiczak, G. M.
name: Spiering, C.
name: Stamatikos, M.
name: Stanev, T.
name: Stanisha, N. A.
name: Stasik, A.
name: Stezelberger, T.
name: Stokstad, R. G.
name: Stößl, A.
name: Strahler, E. A.
name: Ström, R.
name: Strotjohann, N. L.
name: Sullivan, G. W.
name: Taavola, H.
name: Taboada, I.
name: Tamburro, A.
name: Tepe, A.
name: Ter-Antonyan, S.
name: Tešić, G.
name: Tilav, S.
name: Toale, P. A.
name: Tobin, M. N.
name: Toscano, S.
name: Tselengidou, M.
name: Unger, E.
name: Usner, M.
name: Vallecorsa, S.
name: van Eijndhoven, N.
name: Van Overloop, A.
name: van Santen, J.
name: Vehring, M.
name: Voge, M.
name: Vraeghe, M.
name: Walck, C.
name: Waldenmaier, T.
name: Wallraff, M.
name: Weaver, Ch.
name: Wellons, M.
name: Wendt, C.
name: Westerhoff, S.
name: Whitehorn, N.
name: Wiebe, K.
name: Wiebusch, C. H.
name: Williams, D. R.
name: Wissing, H.
name: Wolf, M.
name: Wood, T. R.
name: Woschnagg, K.
name: Xu, D. L.
name: Xu, X. W.
name: Yanez, J. P.
name: Yodh, G.
name: Yoshida, S.
name: Zarzhitsky, P.
name: Ziemann, J.
name: Zierke, S.
name: Zoll, M.
Publisher: name: arXiv
Year: 2013
Subjects: subject: Distributed, Parallel, and Cluster Computing (cs.DC)
subject: FOS: Computer and information sciences
subject: FOS: Computer and information sciences

description: IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, iden- tify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. IceProd is a distributed management system based on Python, XML-RPC and GridFTP. It is driven by a central database in order to coordinate and admin- ister production of simulations and processing of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, Condor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework.