bluuebunny/crossref_metadata_2025 huggingface dataset:
CRIC: a unified information system for WLCG and beyond
Authors: Anisenkov Alexey , Andreeva Julia , Di Girolamo Alessandro , Paparrigopoulos Panos , Vedaee Aresh
Published: September 2019

The Worldwide LHC Computing Grid (WLCG) is an innovative distributed environment which is deployed through the use of grid computing technologiesin order to provide computing and storage resources to the LHC experimentsfor data processing and physics analysis. Following increasing demands of LHC computing needs toward high luminosity era, the experiments are engagdin an ambitious program to extend the capability of WLCG distributed environment, for instance including opportunistically used resources such as High-Performance Computers (HPCs), cloud platforms and volunteer computer. norder to be effectively used by the LHC experiments, all these diverse distributed resources should be described in detail. This implies easy service discovery of shared physical resources, detailed description of service configurations and experiment-specific data structures is needed. In this contribution, we present a high-level information component of a distributed computing environment, the Computing Resource Information Catalogue (CRIC) which aims to facilitate distributed computing operations for the LHC experiments and consolidate WLCG topology information. In addition, CRIC performs data validation and provides coherent view and topology descriptinto the LHC VOs for service discovery and configuration. CRIC represents teevolution of ATLAS Grid Information System (AGIS) into the common experiment independent high-level information framework. CRIC’s mission is to serve not just ATLAS Collaboration needs for the description of the distributed environment but any other virtual organization relying on large scale distributed infrastructure as well as the WLCG on the global scope. The contribution describes CRIC architecture, implementation of data model,collectors, user interfaces, advanced authentication and access control components of the system.
ATLAS utilisation of the Czech national HPC center
Authors: Svatos Michal , Chudoba Jiri , Vokac Petrri
Published: September 2019

The Czech national HPC center IT4Innovations located in Ostrava
provides two HPC systems, Anselm and Salomon. The Salomon HPC is
amongst the hundred most powerful supercomputers on Earth since its commissioning in 2015. Both clusters were tested for usage by the ATLAS experiment for running simulation jobs. Several thousand core hours were allocatedto the project for tests, but the main aim is to use free resources waitigfor large parallel jobs of other users. Multiple strategies for ATLAS job execution were tested on the Salomon and Anselm HPCs. The solution described herein is based on the ATLAS experience with other HPC sites. ARC Compute Element (ARCCE) installed at the grid site in Prague is used for job submission to Salomon. The ATLAS production system submits jobs to the ARC-Evia ARC Control Tower (aCT). The ARC-CE processes job requirements from aTand creates a script for a batch system which is then executed via ssh. Sshfs is used to share scripts and input files between the site and the HPC cluster. The software used to run jobs is rsynced from the site's CVMFS installation to the HPC's scratch space every day to ensure availabiliy of recent software. Using this setting, opportunistic capacity of the Salomon HPC was exploited.
Advanced Analytics service to enhance workflow control at the ATLAS Production System
Authors: Titov Mikhail , Borodin Mikhail , Golubkov Dmitry , Klimentov Alexei
Published: September 2019

Modern workload management systems that are responsible for central data production and processing in High Energy and Nuclear Physics experiments have highly complicated architectures and require a specialized control service for resource and processing components balancing. Such a service represents a comprehensive set of analytical tools, management utilities and monitoring views aimed at providing a deep understanding of internal processes, and is considered as an extension for situational awareness analytic service. Its key points are analysis of task processing, e.g., selection and regulation of key task features that affect its processing the most; modeling of processed data life-cycles for further analysis, e.g., generate guidelines for particular stage of data processing; and forecasting processes with focus on data and tasks states as well as on the management system itself, e.g., to detect the source of any potential malfunction. The prototype of the advanced analytics service will be an essential part of the analytical service of the ATLAS Production System (ProdSys2). Advanced analytics service uses such tools as Time-To-Complete (TTC) estimation towards units of the processing (i.e., tasks and chains of tasks) to control the processing state and to be able to highlight abnormal operations and executions. Obtained metrics are used in decision making processes to regulate the system behaviour and resources consumption.
Limits of the HTCondor Transfer System
Authors: Fajardo Edgar , Würthwein Frank , Jones Richard , Philpott Sandy , Strosahl Kurt
Published: September 2019

In the past, several scaling tests have been performed on the HTCondor batch system regarding its job scheduling capabilities. In this talk we report on a first set of scalability measurements of the file transfer capabilities of the HTCondor batch system. Motivated by the GlueX experiment needs we evaluate the limits and possible use of HTCondor as a solution to transport the output of jobs back to a submitter as a function of the number of concurrent jobs, the size of the output and the distance between a submitting node and computing node. We conclude that HTCondor transfer system satisfies the current GlueX computing demands.
Overview of the ATLAS distributed computing system
Authors: Elmsheuser Johannes , Di Girolamo Alessandro
Published: September 2019

The CERN ATLAS experiment successfully uses a worldwide computing infrastructure to support the physics program during LHC Run 2. The Grid workflow system PanDA routinely manages 250 to 500 thousand concurrently running production and analysis jobs to process simulation and detector data. In total more than 370 PB of data is distributed over more than 150 sites in the WLCG and handled by the ATLAS data management system Rucio. To prepare for the ever growing LHC luminosity in future runs new developments are underway to even more efficiently use opportunistic resources such as HPCs and utilize new technologies. This paper will review and explain the outline and the performance of the ATLAS distributed computing system and give an outlook to new workflow and data management ideas for the beginning of the LHC Run 3. It will be discussed that the ATLAS workflow and data management systems are robust, performant and can easily cope with the higher Run 2 LHC performance. There are presently no scaling issues and each subsystem is able to sustain the large loads.
LHCb and DIRAC strategy towards the LHCb upgrade
Authors: Stagni Federico , Tsaregorodtsev Andrei , Haen Christophe , Charpentier Philippe , Mathe Zoltan , Krzemien Wojciech Jan , Romanovskiy Vladimir
Published: September 2019

The DIRAC project is developing interware to build and operate distributed computing systems. It provides a development framework and a rich set of services for both Workload and Data Management tasks of large scientific communities. DIRAC is adopted by a growing number of collaborations, including LHCb, Belle2, CLIC, and CTA. The LHCb experiment will be upgraded during the second long LHC shutdown (2019-2020). At restart of data taking in Run 3, the instantaneous luminosity will increase by a factor of five. The LHCb computing model also need be upgraded. Oversimplifying, this translates into the need for significantly more computing power and resources, and more storage with respect to what LHCb uses right now. The DIRAC interware will keep being the tool to handle all of LHCb distributed computing resources. Within this contribution, we highlight the ongoing and planned efforts to ensure that DIRAC will be able to provide an optimal usage of its distributed computing resources. This contribution focuses on DIRAC plans for increasing the scalability of the overall system, taking in consideration that the main requirement is keeping a running system working. This requirement translates into the need of studies and developments within the current DIRAC architecture. We believe that scalability is about traffic growth, dataset growth, and maintainability: within this contribution we address all of them, showing the technical solutions we are adopting.
Spanish ATLAS Tier-1 & Tier-2 perspective on computing over the next years
Authors: de la Hoz Santiago González , Acosta-Silva Carlos , Pozo Javier Aparisi , Delfino Manuel , Peso Jose del , Casani Álvaro Fernández , Molina José Flix , Torregrosa Esteban Fullana , Montoro Carlos García , Bahilo Julio Lozano , Montiel Almudena del Rocio , Pages Andreu Pacheco , Martínez Javier Sánchez , Salt José , Vedaee Aresh
Published: September 2019

Since the beginning of the WLCG Project the Spanish ATLAS computing centers have participated with reliable and stable resources as well as personnel for the ATLAS Collaboration. Our contribution to the ATLAS Tier2s and Tier1s computing resources (disk and CPUs) in the last 10 years has been around 4-5%. In 2016 an international advisory committee recommended to revise our contribution according to the participation in the ATLAS experiment. With this scenario, we are optimizing the federation of three sites located in Barcelona, Madrid and Valencia, considering that the ATLAS collaboration has developed workflows and tools to flexibly use all the resources available to the collaboration, where the tiered structure is somehow vanishing. In this contribution, we would like to show the evolution and technical updates in the ATLAS Spanish Federated Tier2 and Tier1. Some developments we are involved in, like the Event Index project, as well as the use of opportunistic resources will be useful to reach our goal. We discuss the foreseen/proposed scenario towards a sustainable computing environment for the Spanish ATLAS community in the HL-LHC period.
System Performance and Cost Modelling in LHC computing
Authors: Biscarat Catherine , Boccali Tommaso , Bonacorsi Daniele , Bozzi Concezio , Costanzo Davide , Duellmann Dirk , Elmsheuser Johannes , Fede Eric , Flix Molina José , Giordano Domenico , Grigoras Costin , Iven Jan , Jouvin Michel , Kemp Yves , Lange David , Meinhard Helge , Michelotto Michele , Douglas Roy Gareth , Sansum Andrew , Sartirana Andrea , Schulz Markus , Sciabà Andrea , Smirnova Oxana , Stewart Graeme , Valassi Andrea , Vernet Renaud , Wenaus Torre , Wuerthwein Frank
Published: September 2019

The increase in the scale of LHC computing expected for Run 3 and even more so for Run 4 (HL-LHC) over the next ten years will certainly require radical changes to the computing models and the data processing of the LHC experiments. Translating the requirements of the physics programmes into computing resource needs is a complicated process and subject to significant uncertainties. For this reason, WLCG has established a working group to develop methodologies and tools intended tocharacterise the LHC workloads, better understand their interaction with the computing infrastructure, calculate their cost in terms of resources and expenditure and assist experiments, sites and the WLCG project in the evaluation of their future choices. This working group started in November 2017 and has about 30 active participants representing experiments and sites. In this contribution we expose the activities, the results achieved and the future directions.
Operational security, threat intelligence & distributed computing: the WLCG Security Operations Center Working Group
Authors: Crooks David , Vâlsan Liviu , Mohammad Kashif , McKee Shawn , Clark Paul , Boutcher Adam , Padée Adam , Wójcik Michał , Giemza Henryk , Kreukniet Bas
Published: September 2019

The modern security landscape for distributed computing in High Energy Physics (HEP) includes a wide range of threats employing different attack vectors. The nature of these threats is such that the most effective method for dealing with them is to work collaboratively, both within the HEP community and with partners further afield - these can, and should, include institutional and campus security teams. In parallel with this work, an appropriate technology stack is essential, incorporating current work on Big Data analytics. The work of the Worldwide LHC Computing Grid (WLCG) Security Operations Center (SOC) Working Group (WG) [1] is to pursue these goals to form a reference design (or guidelines) for WLCG sites of different types. The strategy of the group is to identify necessary components - starting with threat intelligence (MISP [2]) and network intrusion detection (Bro [3]), building a working model over time. We present on the progress of the working group thus far, in particular on the programme of workshops now underway. These workshops give an opportunity to engage with sites to allow the development of advice and procedures for deployment, as well as facilitating wider discussions on how to best work with trust groups at different levels. These trust groups vary in scope but can include institutes, National Grid Infrastructures and the WLCG as a whole.
Towards a responsive CernVM-FS architecture
Authors: Popescu Radu , Blomer Jakob , Ganis Gerardo
Published: September 2019

The CernVM File System (CernVM-FS) provides a scalable and reliable software distribution service implemented as a POSIX read-only filesystem in user space (FUSE). It was originally developed at CERN to assist High Energy Physics (HEP) collaborations in deploying software on the worldwide distributed computing infrastructure for data processing applications. Files are stored remotely as content-addressed blocks on standard web servers and are retrieved and cached on-demand through outgoing HTTP connections only. Repository metadata is recorded in SQLite catalogs, which represent implicit Merkle treeencodings of the repository state. For writing, CernVM-FS follows a publish-subscribe pattern with a single source of new content that is propagated to a large number of readers. This paper focuses on the work to move the CernVM-FS architecturein the direction of a responsive data distribution system. A new distributed publication backend allows scaling out large publication tasks across multiple machines, reducing the time to publish. For the faster propagation of new published content, the addition of a notification system allows clients to subscribe to messages about changes in the repository and to request new root catalogs as soon as they become available. These devel-opments make CernVM-FS more responsive and are particularly relevant for use cases where a short propagation delay from repository down to individual clients is important, such as using CernVM-FS as an AFS replacement for distributing software stacks. Additionally, they permit the implementation of more complex workflows, with producer-consumer pipelines, as for example in the ALICE analysis trains system.
JAliEn: the new ALICE high-performance and high-scalability Grid framework
Authors: Martinez Pedreira M. , Grigoras C. , Yurchenko V.
Published: September 2019

The ALICE experiment will undergo extensive hardware and software upgrades for the LHC Run3. This translates in significant increase of the CPU and storage resources required for data processing, and at the same time the data access rates will grow linearly with the amount of resources. JAliEn (Java ALICE Environment) is the new Grid middleware designed to scale-out horizontally to fulfil the computing needs of the upgrade, and at the same time to modernize all parts of the distributed system software. This paper will present the architecture of the JAliEn framework, the technologies used and performance measurements. This work will also describe the next generation solution that will replace our main database backend, the AliEn File Catalogue. The catalogue is an integral part of the system, containing the metadata of all files written to the distributed Grid storage and also provides powerful search and data manipulation tools. As for JAliEn, the focus has been put onto horizontal scalability, with the aim to handle near exascale data volumes and order of magnitude more workload than the currently used Grid middleware. Lastly, this contribution will present how JAliEn manages the increased complexity of the tasks associated with the new ALICE data processing and analysis framework (ALFA) and multi-core environments.
Factory Monitoring for the 21st Century
Authors: Dost Jeffrey , Fajardo Edgar , Bockelman Brian , Hernández-Cano Leonardo , Hung Caitlin , Kashyap Naveen , Würthwein Frank , Zvada Marian
Published: September 2019

A key aspect of pilot-based grid operations are the GlideinWMS pilot factories. A proper and efficient use of any central block in the grid infrastructure for operations is inevitable, and GlideinWMS factories are no exception. The monitoring package for the GlideinWMS factory was originally developed when the factories were serving a couple of VOs and tens of sites. Today with the factories serving tens of VOs and hundreds of sites around the globe an update of the monitoring is due. Moreover with the new availability of industry open source storage and graphing packages an opportunity remains open.This work presents the changes made to the factory monitoring to leverage different technologies: Elasticsearch, RabbitMQ, Grafana, and InfluxDB to provide a centralized view of the status and work of several GlideinWMS factories located in different continents around the globe.
COMPASS Production System Overview
Authors: Petrosyan Artem
Published: September 2019

Migration of COMPASS data processing to Grid environment has started in 2015 from a small prototype, deployed on a single virtual machine. Since summer of 2017, the system works in production mode, distributing jobs to two traditional Grid sites: CERN and JINR. Now the infrastructure of COMPASS Grid Production System includes 6 virtual machines, each is reserved for one production service: database, PanDA, Auto Pilot Factory, Monitoring, CRIC information system and, finally, production system (ProdSys) management instance, which provides a user interface for production manager and hosts services of automatic processing. Support of COMPASS virtual organization is provided by CERN IT. CRIC is also deployed at CERN Cloud Service. Other ProdSys services are deployed at JINR Cloud Service. There are two storage elements at CERN: EOS for short-term storage and Castor for long-term storage. During last year, along with providing a 24/7 service, the system was instrumented by many features, which allow automating data processing as much as possible. Recently, Blue Waters HPC has become a member of the computing infrastructure of the experiment. Details of implementation, workflow management, and infrastructure overview are presented in this article.
ATLAS Software Installation on Supercomputers
Authors: Undrus Alexander
Published: September 2019

PowerPC and high-performance computers (HPC) are important resources for computing in the ATLAS experiment. The future LHC data processing will require more resources than Grid computing, currently using approximately 100,000 cores at well over 100 sites, can provide. Supercomputers are extremely powerful as they utilize hundreds of thousands of CPUs joined together. However, their architectures have different instruction sets. ATLAS binary software distributions for x86 chipsets do not fit these architectures, as emulation of these chipsets results in huge performance loss. This paper describes the methodology of ATLAS software installation from source code on supercomputers. The installation procedure includes downloading the ATLAS simulation release code with 0.7 million C++ and Python lines as well as the source code of more than 50 external packages, such as ROOT and Geant4, followed by compilation, and rigorous unit and integration testing. The presentation reports the application of this procedure at Titan HPC and Summit PowerPC at Oak Ridge Computing Facility (OLCF).
The Security model of the ALICE next generation Grid framework
Authors: Martinez Pedreira Miguel , Grigoras Costin , Yurchenko Volodymyr , Melnik Storetvedt Maksim
Published: September 2019

JAliEn (Java-AliEn) is the ALICE next generation Grid framework which will be used for the top-level distributed computing resources management during the LHC Run 3 and onward. While preserving an interface familiar to the ALICE users, its performance and scalability are an order of magnitude better than the currently used framework. To implement the JAliEn security model, we have developed the so-called Token Certificates – short lived full Grid certificates, generated by central services automatically or on the client’s request. Token Certificates allow fine-grained control over user/client authorization, e.g. filtering out unauthorized requests based on the client’s type: end user, job agent, jobpayload. These and other parameters (like job ID) are encrypted in the token by the issuing service and cannot be altered.The client-side security implementation is further described in aspects of the interaction between user jobs and job agents. User jobs will use JAliEn tokens for authentication and authorization by the central JAliEn services. These tokens are passed from the job agent through a pipe stream, not stored on disk and thus readily available only to the intended job process. The level of isolation of user payloads is further improved by running them in containers. While JAliEn doesn't rely on X.509 proxies, the backward compatibility is kept to assure interoperability with services that require them.
Towards the integrated ALICE Online-Offline (O2) monitoring subsystem
Authors: Barroso Vasco Chibante , Elia Domenico , Grigoras Costin , Gomez Ramirez Andres , Vino Gioacchino , Wegrzynek Adam
Published: September 2019

ALICE (A Large Ion Collider Experiment) is preparing for a major upgrade of the detector, readout and computing systemsfor LHC Run 3. A new facility called O2 (Online-Offline) will play a major role in data compression and event processing. To efficiently operate the experiment, we are designing a monitoring subsystem, which will provide a complete overview of the O2 overall health, detect performance degradation and component failures. The monitoring subsystem will receive and collect up to 600 kHz of performance metrics. It consists of a custom monitoring library and a server-side, distributed software covering five main functional tasks: parameter collection and processing, storage, visualisation and alarms. To select the most appropriate tools for these tasks, we evaluated three options: “Modular Stack”, Zabbix and the currently used ALICE Grid monitoring tool called MonALISA. The former one consists of a toolkit including collectd, Apache Flume, Apache Spark, InfluxDB, Grafana and Riemann. This paper describes the monitoring subsystem functional architecture. It goes through a complete evaluation of the three considered options, the selection process, risk assessment and justification for the final decision. The in-depth comparison includes functional features and throughput measurement to ensure the required processing and storage performance.
Towards a computing model for the LHCb Upgrade
Authors: Bozzi Concezio , Roiser Stefan
Published: September 2019

The LHCb experiment will be upgraded for data taking in the LHC Run 3. The foreseen trigger output bandwidth trigger ofa few GB/s will result in data sets of tens of PB per year, which need to be efficiently streamed and stored offline for low-latency data analysis. In addition, simulation samples of up to two orders of magnitude larger than those currently simulated are envisaged, with big impact on offline computing and storage resources. This contribution discusses the offline computing model and the required offline resources for the LHCb upgrade, as resulting from the above requirements.
Operation of the ATLAS Distributed Computing
Authors: Barreiro Magino Fernando , Cameron David , Di Girolamo Alessandro , Filipcic Andrej , Glushkov Ivan , Legger Federica , Maeno Tadashi , Walker Rodney
Published: September 2019

ATLAS is one of the four experiments collecting data from the proton-proton collisions at the Large Hadron Collider. The offline processing and storage of the data is handled by a custom heterogenous distributed computing system. This paper summarizes some of the challenges and operations-driven solutions introduced in the system.
BigPanDA: PanDA Workload Management System and its Applications beyond ATLAS
Authors: Svirin Pavlo , De Kaushik , Forti Alessandra , Klimentov Alexei , Larsen Rasmus , Love Peter , Maeno Tadashi , Mashinistov Ruslan , Mukherjee Swagato , Nomerotski Andrei , Oleynik Danila , Panitkin Sergey , Park Hye Yun , Sheldon Erin , Slosar Anze , Wells Jack , Wenaus Torre
Published: September 2019

Modern experiments collect peta-scale volumes of data and utilize vast, geographically distributed computing infrastructure that serves thousands of scientists around the world. Requirements for rapid, near real-time data processing, fast analysis cycles and need to run massive detector simulations to support data analysis pose special premium on efficient use of available computational resources. A sophisticated Workload Management System (WMS) is needed to coordinate the distribution and processing of data and jobs in such environment. The ATLAS experiment at CERN uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. While PanDAcurrently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, it runs around 2 million jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. In 2017 about 1.5 exabytes of data were processed with PanDA.In 2012 BigPanDA project project was started with aim to introduce new types of computing resources into ATLAS computing infrastructure, but also to offering PanDA features to different data-intensive applications for projects and experiments outside of ATLAS and High-Energy and Nuclear Physics. In this article we will present accomplishments and discuss possible directions for future work.
CMS Computing Resources: Meeting the demands of the high-luminosity LHC physics program
Authors: Lange David , Bloom Kenneth , Boccali Tommaso , Gutsche Oliver , Vaandering Eric
Published: September 2019

The high-luminosity program has seen numerous extrapolations of its needed computing resources that each indicate the need for substantial changes if the desired HL-LHC physics program is to be supported within the current level of computing resource budgets. Drivers include large increases in event complexity (leading to increased processing time and analysis data size) and trigger rates needed (5-10 fold increases) for the HL-LHC program. The CMS experiment has recently undertaken an effort to merge the ideas behind short-term and long-term resource models in order to make easier and more reliable extrapolations to future needs. Near term computing resource estimation requirements depend on numerous parameters: LHC uptime and beam intensities; detector and online trigger performance; software performance; analysis data requirements; data access, management, and retention policies; site characteristics; and network performance. Longer term modeling is affected by the same characteristics, but with much larger uncertainties that must be considered to understand the most interesting handles for increasing the "physics per computing dollar" of the HL-LHC. In this presentation, we discuss the current status of long term modeling of the CMS computing resource needs for HL-LHC with emphasis on techniques for extrapolations, uncertainty quantification, and model results. We illustrate potential ways that high-luminosity CMS could accomplish its desired physics program within today's computing budgets.
ATLAS Distributed Computing: Its Central Services core
Authors: Lee Christopher Jon , Di Girolamo Alessandro , Elmsheuser Johannes , Buzykaev Alexey , Obreshkov Emil , Glushkov Ivan , Sun Shaojun
Published: September 2019

The ATLAS Distributed Computing (ADC) Project is responsible for the off-line processing of data produced by the ATLAS experiment at the Large Hadron Collider (LHC) at CERN. It facilitates data and workload management for ATLAS computing on the Worldwide LHC Computing Grid (WLCG). ADC Central Services operations (CSOPS) is a vital part of ADC, responsible for the deployment and configuration of services needed by ATLAS computing and operation of those services on CERN IT infrastructure, providing knowledge of CERN IT services to ATLAS service managers and developers, and supporting them in case of issues. Currently this entails the management of 43 different OpenStack projects, with more than 5000 cores allocated for these virtual machines, as well as overseeing the distribution of 29 petabytes of storage space in EOS for ATLAS. As the LHC begins to get ready for the next long shut-down, which will bring in many new upgrades to allow for more data to be captured by the on-line systems, CSOPS must not only continue to support the existing services, but plan ahead for the expected increase in data, users, and services that will be required. This paper attempts to explain the current state of CSOPS as well as the strategies in place to maintain the service functionality in the long term.
Dataclient: a simple interface for scientific data transfers hiding x.509 complexities
Authors: Ciaschini Vincenzo , Morganti Lucia , Tenti Matteo , Pellegrino Carmelo
Published: September 2019

Since the current data infrastructure of the HEP experiments is based on GridFTP, most computing centres have adapted and based their own access to the data on the X.509. This is an issue for smaller experiments that do not have the resources to train their researchers in the complexities of X.509 certificates and that would prefer an approach based on username/password.

On the other hand, asking computing centres to support different access strategies is not so straightforward, as this would require a significant investment of effort and manpower.

At CNAF-INFN Tier1 we tackled this problem by creating a layer on top of the gridftp client/server, that completely hides the X.509 infrastructure under an authentication/authorization process based on the Kerberos realm of our centre, and therefore based on username/password.

We called this Dataclient. In this article we will describe both the principles that drove its design and its general architecture, together with the measures undertaken to simplify the user experience and maintenance burden.
Broadcasting dynamic metadata content to external web pages using AMI (ATLAS Metadata Interface) embeddable components
Authors: Lambert Fabian , Odier Jérôme , Fulachier Jérôme
Published: September 2019

AMI (ATLAS Metadata Interface) is a generic ecosystem for metadata aggregation, transformation and cataloguing. Often, it is interesting to share up-to-date metadata with other content services such as wikis. Here, the cross-domain solution implemented in the AMI Web Framework is described: a system of embeddable controls, communicating with the central AMI service and based on the AJAX and CORS technologies. The main available controls and their basic usage are also described.
XRootD plug-in based solutions for site specific require-ments
Authors: Knedlik Jan , Kramp Paul , Schwarz Kilian , Kollegger Thorsten
Published: September 2019

XRootD† has been established as a standard for WAN data access in HEP and HENP. Site specific features, like those existing at GSI, have historically been hard to implement with native methods. XRootD allows a custom replacement of basic functionality for native XRootD functions through the use of plug-ins. XRootD clients allow this since version 4.0. In this contribution, our XRootD based developments motivated by the use in the current ALICE Tier 2 Centre at GSI and the upcoming ALICE Analysis Facility will be shown. Among other things, an XRootD redirector plug-in which redirects local clients directly to a shared file system, as well as the needed changes to the XRootD base code, which are publicly available since XRootD version 4.8.0, will be presented. Furthermore, a prototype for an XRootD based disk caching system for opportunistic resources has been developed.
Advancing throughput of HEP analysis work-flows using caching concepts
Authors: Caspart Rene , Fischer Max , Giffels Manuel , Heidecker Christoph , Kühn Eileen , Quast Günter , Sauter Martin , Schnepf Matthias J. , von Cube R. Florian
Published: September 2019

High throughput and short turnaround cycles are core requirements for efficient processing of data-intense end-user analyses in High Energy Physics (HEP). Together with the tremendously increasing amount of data to be processed, this leads to enormous challenges for HEP storage systems, networks and the data distribution to computing resources for end-user analyses. Bringing data close to the computing resource is a very promising approach to solve throughput limitations and improve the overall performance. However, achieving data locality by placing multiple conventional caches inside a distributed computing infrastructure leads to redundant data placement and inefficient usage of the limited cache volume. The solution is a coordinated placement of critical data on computing resources, which enables matching each process of an analysis work-flow to its most suitable worker node in terms of data locality and, thus, reduces the overall processing time. This coordinated distributed caching concept was realized at KIT by developing the coordination service NaviX that connects an XRootD cache proxy infrastructure with an HTCondor batch system. We give an overview about the coordinated distributed caching concept and experiences collected on prototype system based on NaviX.
Xcache in the ATLAS Distributed Computing Environment
Authors: Hanushevsky Andrew , Ito Hironori , Lassnig Mario , Popescu Radu , De Silva Asoka , Simon Michal , Gardner Robert , Garonne Vincent , De Stefano John , Vukotic Ilija , Yang Wei
Published: September 2019

Inherited from the flexible architecture of Xrootd, Xcache allows a wide range of customization through configurations and plugin modules. This paper describes several completed and ongoing R&D efforts of using Xcache in the LHC ATLAS distributed computing environment, in particular, using Xcache with the ATLAS data management system Rucio for easy-to-use and to improve cache hit rate, to replace Squid and improve distribution of large files in CVMFS, to adapt the HPC environment and the data lake model for efficient data distribution and access for the HPCs.
CERN Tape Archive — from development to production deployment
Authors: Davis Michael C. , Bahyl Vladímir , Cancio Germán , Cano Eric , Leduc Julien , Murray Steven
Published: September 2019

The first production version of the CERN Tape Archive (CTA) software is planned to be released during 2019. CTA is designed to replace CASTOR as the CERN tape archive solution, to face the scalability and performance challenges arriving with LHC Run–3.

In this paper, we describe the main commonalities and differences between CTA and CASTOR. We outline the functional enhancements and integration steps required to add the CTA tape back-end to an EOS disk storage system. We present and discuss the different deployment and migration scenarios for replacing the five CASTOR instances at CERN, including a description of how the File Transfer Service (FTS) will interface with EOS and CTA.
Scaling the EOS namespace – new developments, and performance optimizations
Authors: Bitzes Georgios , Sindrilaru Elvin Alin , Joachim Peters Andreas
Published: September 2019

EOS is the distributed storage solution being developed and deployed at CERN with the primary goal of fulfilling the data needs of the LHC and its various experiments. Being in production since 2011, EOS currently manages around 256 petabytes of raw disk space and 3.4 billion files across several instances. Nowadays, EOS is increasingly being used as a distributed filesystem and file sharing platform, which poses scalability challenges on its legacy namespace subsystem, tasked with keeping track of all file and directory metadata on a
particular instance.

In this paper we discuss said challenges, and present our solution which has recently entered production. We made several architectural improvements to the overall system design, the most important of which was introducing QuarkDB, a highly-available datastore capable of serving as the metadata backend for EOS,
tailored to the needs of the namespace.

We also describe our efforts in providing comparable latency and performance to the legacy in-memory implementation, both when reading through the use of extensive caching and prefetching, and when writing through the use of latencyhiding techniques involving a persistent, back-pressured local queue for batching writes towards the QuarkDB backend.
A new mechanism to use the Conditions Database REST API to serve the ATLAS detector description
Authors: Formica Andrea , Bianchi Riccardo Maria , De Salvo Alessandro
Published: September 2019

Efficient and fast access to the detector description of the ATLAS experiment is needed for many tasks, at different steps of the data chain: from detector development to reconstruction, from simulation to data visualization. Until now, the detector description was only accessible through dedicated services integrated into the experiment’s software framework, or by the use of external applications. In this work, we explore the possibility of using a web access-based conditions database to store and serve the detector description, aiming at a simplification of the software architecture of the experiment and a reduction in the number of software packages to be maintained.
Storage events: distributed users, federation and beyond
Authors: Paul Millar A. , Adeyemi Olufemi , Garonne Vincent , Litvintsev Dmitry , Mkrtchyan Tigran , Rossi Albert , Sahakyan Marina , Starek Jürgen
Published: September 2019

For federated storage to work well, some knowledge from each storage system must exist outside that system, regardless of the use case. This is needed to allow coordinated activity; e.g., executing analysis jobs on worker nodes with good accessibility to the data.

Currently, this is achieved by clients notifying central services of activity; e.g., a client notifies a replica catalogue after an upload. Unfortunately, this forces end users to use bespoke clients. It also forces clients to wait for asynchronous activities to finish.

dCache provides an alternative approach: storage events. In this approach the storage systems (rather than the clients) become the coordinating service, notifying interested parties of key events.

At DESY, we are investigating storage events along with Apache OpenWhisk and Kubernetes to build a "serverless" cloud, similar to AWS Lambda or Google Cloud Functions, for photon science use cases.

Storage events are more generally useful: catalogues are notified whenever data is uploaded or delete, tape becomes more efficient because analysis can start immediately after the data is on disk, caches can be "smart" fetching new datasets preemptively.

In this paper we will present work within dCache to support a new event-based interface, with which these and other use cases become more efficient.
The ATLAS Wide-Range Database and Application Monitoring
Authors: Vasileva Petya , Formica Andrea , Dimitrov Gancho
Published: September 2019

In HEP experiments at LHC the database applications often become complex, reflecting the increasingly demanding requirements of the researchers. The ATLAS experiment has several Oracle DB clusters with over 216 database schemes each with its own set of database objects. To effectively monitor them, we designed a modern and portable application with exceptionally good characteristics. Some of them include: A concise view of the most important DB metrics; a list of top SQL statements based on CPU, executions, block reads, etc.; volume growth plots per schema and DB object type; a database jobs section with signalization for failures; and in-depth analysis in case of row-lock contention or DB sessions.

This contribution also describes the technical aspects of the implementation. The project can be separated into three independent layers. The first layer consists in highly-optimized database objects hiding all complicated calculations. The second layer represents a server providing REST access to the underlying database backend. The third layer is a JavaScript/AngularJS web interface. In addition, we will summarize the continuous integration cycle of the application, which uses GitLab-ci pipelines for basic testing, containerization and deployment on the CERN Openshift infrastructure.
An http data-federation ecosystem with caching functionality using DPM and Dynafed
Authors: Michelino Davide , Pardi Silvio , Russo Guido , Spisso Bernardino
Published: September 2019

The implementation of cache systems in the computing model of HEP experiments enables to accelerate access to hot data sets by scientists, opening new scenarios of data distribution and enable to exploit the paradigm of storage-less sites. In this work, we present a study for the creation of an http data-federation ecosystem with caching functionality. We created plug-in integrated in the logic of a DPM Storage, able to reproduce a cache behaviour, taking advantage from the new feature introduced in the last version of Disk Pool Manager, called volatile-pool. Then we used Dynafed as lightweight federation system to aggregate a set of standard Grid Storage together with the caching system. With the designed setup, clients asking for a file present on the Data-Grid are automatically redirected to the cache, if the cache is the closest storage, thanks to the action of the geo-plugin run by Dynafed. As proof of the concept, we tested the whole system in a controlled environment within the Belle II computing infrastructure using a set of files located in production Storage Elements. Preliminary results demonstrate the proper functionality of the logic and encourage continuing the work.
Using ZFS to manage Grid storage and improve middleware resilience
Authors: Currie Robert , Li Teng , Washbrook Andrew
Published: September 2019

ZFS is a powerful storage management technology combining filesystem, volume management and software RAID technology into a single solution. The WLCG Tier-2 computing at Edinburgh was an early adopter of ZFS on Linux, with this technology being used to manage all of our storage systems including servers with aging components. Our experiences of ZFS deployment have been shared with the Grid storage community which has led to additional sites adopting this technology.

ZFS is highly configurable therefore allowing systems to be tuned to give best performance under diverse workloads. This paper highlights our experiences in tuning our systems for best performance when combining ZFS with DPM storage management. This has resulted in reduced system load and better data throughput. This configuration also provides the high redundancy required for managing older storage servers. We also demonstrate how ZFS can be combined with Distributed Replicated Block Device (DRBD) technology to provide a performant and resilient hypervisor solution to host multiple production Grid services.
Advancements in data management services for distributed e-infrastructures: the eXtreme-DataCloud project
Authors: Cesini Daniele , Donvito Giacinto , Costantini Alessandro , Aguilar Gomez Fernando , Duma Doina Cristina , Fuhrmann Patrick , Dutka Lukasz , Viljolen Matthew , Battaglia Serena , Poireau Vincent , Dell’Agnello Luca , Keeble Oliver , Lemrani Rachid , Ohmann Christian , Marco de Lucas Jesus
Published: September 2019

The development of data management services capable to cope with very large data resources is a key challenge to allow the future einfrastructures to address the needs of the next generation extreme scale scientific experiments. To face this challenge, in November 2017 the H2020 eXtreme DataCloud - XDC project has been launched. Lasting for 27 months and combining the expertise of eight large European research
organisations, the project aims at developing scalable technologies for
federating storage resources and managing data in highly distributed
computing environments. The targeted platforms are the current and next
generation e-Infrastructures deployed in Europe, such as the European
Open Science Cloud (EOSC), the European Grid Infrastructure (EGI), and
the Worldwide LHC Computing Grid (WLCG). The project is use-case
driven with a multidisciplinary approach, addressing requirements from
research communities belonging to a wide range of scientific domains:
High Energy Physics, Astronomy, Photon and Life Science, Medical
research. XDC is aimed at implementing data management scalable
services, combining already established data management and
orchestration tools, to address the following high level topics: policy
driven data management based on Quality-of-Service, Data Life-cycle
management, smart placement of data with caching mechanisms to reduce
access latency, meta-data with no predefined schema handling, execution
of pre-processing applications during ingestion, data management and
protection of sensitive data in distributed e-infrastructures, intelligent data
placement based on access patterns. This contribution introduces the
project, presents the foreseen overall architecture and the developments
that are being carried on to implement the requested functionalities.
Bootstrapping a New LHC Data Transfer Ecosystem
Authors: Bockelman Brian , Hanushevsky Andrew , Keeble Oliver , Lassnig Mario , Millar Paul , Weitzel Derek , Yang Wei
Published: September 2019

GridFTP transfers and the corresponding Grid Security Infrastructure
(GSI)-based authentication and authorization system have been data transfer
pillars of the Worldwide LHC Computing Grid (WLCG) for more than a
decade. However, in 2017, the end of support for the Globus Toolkit - the reference
platform for these technologies - was announced. This has reinvigorated
and expanded efforts to replace these pillars. We present an end-to-end alternate
utilizing HTTP-based WebDAV as the transfer protocol, and bearer tokens for
distributed authorization.

This alternate ecosystem, integrating significant pre-existing work and ideas in
the area, adheres to common industry standards to the fullest extent possible,
with minimal agreed-upon extensions or common interpretations of the core
protocols. The bearer token approach allows resource providers to delegate authorization
decisions to the LHC experiments for experiment-dedicated storage
areas.

This demonstration touches the entirety of the stack - from multiple storage
element implementations to FTS3 to the Rucio data management system. We
show how the traditional production and user workflows can be reworked utilizing
bearer tokens, eliminating the need for GSI proxy certificates for storage
interactions.
A data caching model for Tier 2 WLCG computing centres using XCache
Authors: Li Teng , Currie Robert , Washbrook Andrew
Published: September 2019

As the WLCG continues to consolidate the data storage resources
to reduce maintaining costs, more storage elements will be decommissioned
or federated in the future. In both cases data streaming would become a
common method for data accessing in many LHC Tier 2 and Tier 3
computing centres which makes it vital to optimise the network usage (i.e.
latency reduction and bandwidth utilisation). Consequently, the data cache
service sits closely to the computing resources becomes a desirable tool. In
this paper we discuss how to setup a XCache server at an example WLCG
Tier 2 site (Edinburgh) and use the ATLAS analysis queue as a testbed to
study the data caching performance. The results could be used to guide the
future development to make the caching server more effective.