bluuebunny/crossref_metadata_2025 huggingface dataset:
Research and Exploit of Resource Sharing Strategy at IHEP
Authors: JIANG Xiaowei , Shi Jingyan , Zou Jiaheng , Hu Qingbao , Du Ran , Sun Zhenyu
Published: September 2019

At IHEP (Institute of High Energy Physics, Chinese Academy of Sciences), computing resources are contributed by different experiments including BES, JUNO, DYW, HXMT, etc. The resources were divided into different partitions to satisfy the dedicated experiment data processing requirements. IHEP had a local Torqu&Maui cluster with 50 queues serving for above 10 experiments. The separated resource partitions leaded to imbalance resource load. In a typical situation, BES resource partition was quite busy without free slot but still with lots of jobs in idle, while JUNO resources are free and wasted seriously.

After moving resources from Torque&Maui to HTCondor in 2016, job scheduling efficiency has been improved a lot. In order to balance the imbalance resource load, we designed an efficient sharing strategy to improve the overall resourceutilization. We created an unified pool shared by all experiments. For each experiment, resources are divided into two parts: dedicated resource and sharing resource. The slots in dedicated resource only run jobs from its own experiment, and the slots in sharing resource are shared by jobs from all experiments. Default ratio of dedicated resource to sharing resource is 1:4. To maximize the sharing effectiveness, the ratio is dynamically adjusted between 0:5 and 4:1 based on the number of jobs submitted by each experiment.

We have developed a central control system to decide how many resources can be allocated to each experiment group. This system is implemented at two sides: server side and client side. A management database is built at server side, which is storing resource, group and experiment information. Once the sharing ratio needs to be adjusted, resource group will be changed and updated into database. The resource group information is published to the server buffer in real-time. The client periodically pulls resource group information from server buffer via https protocol And resource scheduling configuration at client side is changed based on the resource group information. With this method, share ratio can be modified and deployed dynamically.

Sharing strategy is implemented with HTCondor. ClassAd mechanism and accounting-group in HTCondor facilitate to utilizethe sharing strategy at IHEP computing cluster. With the sharing strategy, resource usage has been improved dramatically.
Modeling and Simulation of Load Balancing Strategies for Computing in High Energy Physics
Authors: Caspart René , Firnkes Patrick , Giffels Manuel , Koziolek Anne , Quast Günter , Reussner Ralf , Stemmer-Grabow Maximilian
Published: September 2019

The amount of data to be processed by experiments in high energy physics (HEP) will increase tremendously in the coming years. To cope with this increasing load, most efficient usage of the resources is mandatory. Furthermore, the computing resources for user jobs in HEP will be increasingly distributed and heterogeneous, resulting in more difficult scheduling due to the increasing complexity of the system. We aim to create a simulation for the WLCG helping the HEP community to solve both challenges: a more efficient utilization of the grid and coping with the rising complexity of the system. There is currently no simulation in existence which helps the operators of the grid to make the correct decisions while optimizing the load balancing strategy. This paper presents a proof of concept in which the computing jobs at the Tier 1 center GridKa are modeled and simulated. To model the computing jobs we extended the Palladio simulator with a mechanism to simulate load balancing strategies. Furthermore, we implemented an automated model parameter analysis and model creation.

Finally, the simulation results are validated using real-word performance data. Our results suggest that simulating larger parts of the grid is feasible and can help to optimize the utilization of the grid.
OSG and GPUs: A tale of two use cases
Authors: Fajardo Edgar , Rynge Mats , Weitzel Derek , Merino Gonzalo , Schultz David , Brik Vladimir , Skarlupka Heath
Published: September 2019

With the increase of power and reduction of cost of GPU accelerated processors a corresponding interest in their uses in the scientific domain has spurred. OSG users are no different and they have shown an interest in accessing GPU resourcesvia their usual workload infrastructures. Grid sites that have these kinds of resources also want to make them grid available. In this talk, we discuss the software and infrastructure challenges and limitations of the OSG implementations to make GPU’s widely accessible over the grid. Two use cases are considered for this. First: IceCube, a big VO with a well-curated software stack taking advantage of GPUs with OpenCL. Second, a more general approach to supporting the grid use of industry and academia maintained machine learning libraries like Tensorflow, and Keras on the grid using Singularity.
WISE Information Security for Collaborating e-Infrastructures
Authors: Short Hannah , Kelsey David , Wartel Romain , Groep David , Kaila Urpo , Niederberger Ralph , Harris Nicole
Published: September 2019

As most are fully aware, cybersecurity attacks are an ever-growing problem as larger parts of our lives take place on-line. Distributed digital infrastructures are no exception and action must be taken to both reduce the security risk and tohandle security incidents when they inevitably happen. These activities are carried out by the various research infrastructures and it has become very clear in recent years that collaboration with others both helps to improve the security and to work more efficiently. The Wise Information Security for Collaborating e-Infrastructures (WISE) community provides a trusted framework where security experts can share information on topics such as risk management, experiences about certification processes and threat intelligence. With participants from multiple large scale Infrastructures, WISE focuses on standards, guidelines and practices, and promotes the protection of critical infrastructure. To date WISE has published two documents; a risk management template and a second version of the SCI framework, endorsed by multiple large-scale infrastructures. In 2018 WISE began work on new areas of relevance to the High Energy Physics community, including a focus on operational security and incident response for interoperating infrastructures. We present an overview of the available WISE recommendations, future work and how WISE brings benefits to the High Energy Physics community.
SPT-3G Computing
Authors: Riedel Benedikt , Bryant Lincoln , Carlstrom John , Crawford Thomas , Gardner Robert W. , Harrington Nicholas , Huang Nicholas , Rahlin Alexandra , Stephen Judith , Whitehorn Nathan
Published: September 2019

SPT-3G, the third generation camera on the South Pole Telescope (SPT), was deployed in the 2016-2017 Austral summer season. The SPT is a 10-meter telescope located at the geographic South Pole and designed for observations in the millimeter-wave and submillimeter-wave regions of the electromagnetic spectrum. The SPT is primarily used to study the cosmic microwave background (CMB). The upgraded camera produces an order of magnitude more data than the previous generations of SPT cameras. The telescope is expected to collect a petabyte (PB) of data over course of five years, which is a significantly larger data volume than any other CMB telescope in operation. The increase in data rate required radical changes to the SPT computing model both at the South Pole and University of Chicago. This paper will describe the overall integration of distributed storage and compute resources into a common interface, deployment of on-site data reduction and storage infrastructure, and the usage of the Open Science Grid (OSG) by the SPT collaboration.
The Cherenkov Telescope Array production system for data-processing and Monte Carlo simulation
Authors: Arrabito Luisa , Bernlöhr Konrad , Bregeon Johan , Cumani Paolo , Hassan Tarek , Haupt Andreas , Maier Gernot , Moralejo Abelardo , Neyroud Nadine , Stagni Federico , Tsaregorodtsev Andrei
Published: September 2019

The Cherenkov Telescope Array (CTA) is the next-generation instrument in the field of very high energy gamma-ray astronomy. It will be composed of two arrays of Imaging Atmospheric Cherenkov Telescopes, located at La Palma (Spain) and Paranal (Chile). The construction of CTA has just started with the installation of the first telescope on site at La Palma and the first data expected by the end of 2018. The scientific operations should begin in 2022 for a duration of about 30 years. The overall amount of data produced during these operations is around 27 PB per year. The associated computing power for data processing and Monte Carlo (MC) simulations is of the order of hundreds of millions of CPU HS06 hours per year. In order to cope with these high computing requirements, we have developed a production system prototype based on the DIRAC framework, that we have intensively exploited during the past 6 years to handle massive MC simulations on the grid for the CTA design and prototyping phases. CTA workflows are composed of several inter-dependent steps, which we used to handle separately within our production system. In order to fully automatize the whole workflows execution, we have partially revised the production system by further enhancing the data-driven behavior and by extending the use of meta-data to link together the different steps of a workflow. In this contribution we present the application of the production system to the last years MC campaigns as well as the recent production system evolution, intended to obtain a fully data-driven and automatized workflow execution for efficient processing of real telescope data.
PanDA and RADICAL-Pilot Integration: Enabling the Pilot Paradigm on HPC Resources
Authors: Merzky Andre , Svirin Pavlo , Turilli Matteo
Published: September 2019

PanDA executes millions of ATLAS jobs a month on Grid systems with more than 300,000 cores. Currently, PanDA is compatible only with few high-performance computing (HPC) resources due to different edge services and operational policies; does not implement the pilot paradigm on HPC; and does not dynamically optimize resource allocation among queues. We integrated the PanDA Harvester service and the RADICAL-Pilot (RP) system to overcome these limitations and enable the execution of ATLAS, Molecular Dy-namics and other workloads on HPC resources. This paper offer two main con-tributions: (1) introducing PanDA Harvester and RADICAL-Pilot, two systems independent developed to support high-throughput computing (HTC) on high-performance computing (HPC) infrastructures; (2) describing the integration between these two systems to produce a middleware component with unique functionalities, including the concurrent execution of heterogeneous workloads on the Titan OLCF machine. We integrated Harvester and RP by prototyping a Next Generation Executor (NGE) to expose RP capabilities and manage the execution of PanDA workloads. In this way, we minimized the reengineering of the two systems, allowing their integration while being in production.
Development and operational experience of the web based application to collect, manage, and release the alignment and calibration configurations for data processing at CMS
Authors: Md Hasbuddin , Kumar Arun , Govi Giacomo , Franzoni Giovanni , Pernie Luca , Herland Sverre
Published: September 2019

The alignment and calibration workflows at the Compact Muon Solenoid (CMS) experiment are fundamental to provide a high quality physics data and to maintain the design performance of the experiment. To facilitate the operational efforts required by the experiment, the alignment and calibration team has developed and deployed a set of web-based applications to search, navigate and prepare a consistent set of calibrations to be consumed in reconstruction of data for physics, accessible through the Condition DB Browser. The Condition DB Browser hosts also various data management tools, including a vi-sualization tool that allows to easily inspect alignment an calibration contents, an user-defined notification agent for delivering updates on modification to the database, a logging service for the user and the automatic online-to-offline condition uploads. In this paper we report on the operational experience of this web application from 2017 data taking, with focus on new features and tools incorporated during this period.
HSM and backup services at INFN-CNAF
Authors: Cavalli Alessandro , Cesini Daniele , Fattibene Enrico , Prosperini Andrea , Sapunenko Vladimir
Published: September 2019

IBM Spectrum Protect (ISP) software, one of the leader solutions in data protection, contributes to the data management infrastructure operated at CNAF, the central computing and storage facility of INFN (Istituto Nazionale di Fisica Nucleare – Italian National Institute for Nuclear Physics). It is used to manage about 55 Petabytes of scientific data produced by LHC (Large Hadron Collider at CERN) and other experiments in which INFN is involved, stored on tape resources as the highest latency storage tier within HSM (Hierarchical Space Management) environment. To accomplish this task, ISP works together with IBM Spectrum Scale (formerly GPFS - General Parallel File System) and GEMSS (Grid Enabled Mass Storage System), an in-house developed software layer that manages migration and recall queues. Moreover, we perform backup/archive operation of main IT services running at CNAF, such as mail servers, configurations, repositories, documents, logs, etc. In this paper we present the current configuration of the HSM infrastructure and the backup and recovery service, with particular attention to issues related to the increasing amount of scientific data to manage, expected for the next years.
Capability-Based Authorization for HEP
Authors: Weitzel Derek , Bockelman Brian , Basney Jim , Tannenbaum Todd , Miller Zach , Gaynor Jeff
Published: September 2019

Outside the HEP computing ecosystem, it is vanishingly rare to encounter user X509 certificate authentication (and proxy certificates are even more rare). The web never widely adopted the user certificate model, but increasingly sees the need for federated identity services and distributed authorization. For example, Dropbox, Google and Box instead use bearer tokens issued via the OAuth2 protocol to authorize actions on their services. Thus, the HEP ecosystem has the opportunity to reuse recent work in industry that now covers our needs. We present a token-based ecosystem for authorization tailored for use by CMS.

We base the tokens on the SciTokens profile for the standardized JSON Web Token (JWT) format. The token embeds a signed description of what capabilities the VO grants the bearer; the site-level service can verify the VO’s signature without contacting a central service.

In this paper, we describe the modifications done to enable token-based authorization in various software packages used by CMS, including XRootD, CVMFS, and HTCondor. We describe the token-issuing workflows that would be used to get tokens to running jobs in order to authorize data access and file stageout, and explain the advantages for hosted web services. Finally, we outline what the transition would look like for an experiment like CMS.
The challenges of mining logging data in ATLAS
Authors: Gallas Elizabeth J , Ozturk Nurcan
Published: September 2019

Processing ATLAS event data requires a wide variety of auxiliary information from geometry, trigger, and conditions database systems. This information is used to dictate the course of processing and refine the measurement of particle trajectories and energies to construct a complete and accurate picture of the remnants of particle collisions. Such processing occurs on a worldwide computing grid, necessitating wide-area access to this information.

Event processing tasks may deploy thousands of jobs. Each job calls for a unique set of information from the databases via SQL queries to dedicated squids in the ATLAS Frontier system, a system designed to pass queries to the database only if that result has not already been cached from another request. Many queries passing through Frontier are logged in an Elastic Search cluster along with pointers to the associated tasks and jobs, various metrics, and states at the time of execution. PanDA, which deploys the jobs, stores various configuration files as well as many log files after each job completes. Information is stored at each stage, but no system contains all information needed to draw a complete picture. This presentation describes the challenges of mining information from these sources to compile a view of database usage by jobs and tasks as well as assemble a global picture of the coherence and competition of tasks in resource usage to identify inefficiencies and bottlenecks within the overall system.
The Open High Throughput Computing Content Delivery Network
Authors: Dykstra Dave , Bockelman Brian , Blomer Jakob , Field Laurence
Published: September 2019

LHC experiments make extensive use of web proxy caches, especially for software distribution via the CernVM File System and for conditions data via the Frontier Distributed Database Caching system. Since many jobs read the same data, cache hit rates are high and hence most of the traffic flows efficiently over Local Area Networks. However, it is not always possible to have local web caches, particularly for opportunistic cases where
experiments have little control over site services. The Open High Throughput Computing (HTC) Content Delivery Network (CDN), openhtc.io, aims to address this by using web proxy caches from a commercial CDN provider. Cloudflare provides a simple interface for registering DNS aliases of any web server and does reverse proxy web caching on those aliases. The openhtc.io domain is hosted on Cloudflare's free tier CDN which has no bandwidth limit and makes use of data centers throughout the world, so the average performance for clients is much improved compared to reading from CERN or a Tier 1. The load on WLCG servers is also significantly reduced. WLCG Web Proxy Auto Discovery is used to select local web caches when they are available and otherwise select openhtc.io caching. This paper describes the Open HTC CDN in detail and provides initial results from its use for LHC@Home and USCMS opportunistic computing.
Using a dynamic data federation for running Belle-II simulation applications in a distributed cloud environment
Authors: Ebert Marcus , Berghaus Frank , Casteels Kevin , Driemel Colson , Leavett-Brown Colin , Fernandez Galindo Fernando , Paterson Michael , Seuster Rolf , Sobie Randall , Tafirout Reda , Taylor Ryan
Published: September 2019

The dynamic data federation software Dynafed, developed by CERN IT, provides a federated storage cluster on demand using the HTTP protocol with WebDAV extensions. Traditional storage sites which support an experiment can be added to Dynafed without requiring any changes to the site. Dynafed also supports direct access to cloud storage such as S3 and Azure. We report on the usage of Dynafed to support Belle-II production jobs running on a distributed cloud system utilizing clouds across North America. Cloudscheduler, developed by the University of Victoria HEP Research Computing group , federates Openstack, OpenNebula, Amazon, Google, and Microsoft cloud compute resources and provides them as a unified Grid site which on average runs about 3500 Belle-II production jobs in parallel. The input data for those jobs is accessible through a single endpoint, our Dynafed instance. This Dynafed instance unifies storage resources provided by Amazon S3, Ceph, and Minio object stores as endpoints, as well as storage provided by traditional DPM and dCache sites. We report on our long term experience with this setup, the implementation of a grid-mapfile based X509 authentication/authorization for Belle-II access, and we show how a federated cluster can be used by Belle-II through gfalFS.
Towards an Event Streaming Service for ATLAS data processing
Authors: Brino Alex , Di Girolamo Alessandro , Guan Wen , Lassnig Mario , Maeno Tadashi , Magini Nicolò , Nilsson Paul , Tsulaia Vakhtang , Walker Rodney , Wenaus Torre
Published: September 2019

The ATLAS experiment at the LHC is gradually transitioning from the traditional file-based processing model to dynamic workflow management at the event level with the ATLAS Event Service (AES). The AES assigns finegrained processing jobs to workers and streams out the data in quasi-real time, ensuring fully efficient utilization of all resources, including the most volatile. The next major step in this evolution is the possibility to intelligently stream the input data itself to workers. The Event Streaming Service (ESS) is now in development to asynchronously deliver only the input data required for processing when it is needed, protecting the application payload fromWAN latency without creating expensive long-term replicas. In the current prototype implementation, ESS processes run on compute nodes in parallel to the payload, reading the input event ranges remotely over the network, and replicating them in small input files that are passed to the application. In this contribution, we present the performance of the ESS prototype for different types of workflows in comparison to tasks accessing remote data directly. Based on the experience gained with the current prototype, we are now moving to the development of a server-side component of the ESS. The service can evolve progressively into a powerful Content Delivery Network-like capability for data streaming, ultimately enabling the delivery of ‘virtual data’ generated on demand.
dCache - storage for advanced scientific use cases and beyond
Authors: Mkrtchyan Tigran , Adeyemi Olufemi , Fuhrmann Patrick , Garonne Vincent , Litvintsev Dmitry , Millar Paul , Rossi Albert , Sahakyan Marina , Starek Jürgen , Yasar Sibel
Published: September 2019

The dCache project provides open source storage software
deployed internationally to satisfy ever more demanding scientific storage
requirements. Its multifaceted approach provides an integrated way of
supporting different use cases with the same storage, from high throughput
data ingest, through wide access and easy integration with existing
systems. In supporting new communities, such as photon science and
microbiology, dCache is evolving to provide new features and access to
new technologies. In this paper, we describe some of these recent features
that facilitate the use of storage to maximise the gain from stored data,
including quality-of-service management, support for distributed and
federated systems, and improvements with support for parallel NFS
(pNFS).
dCache - joining the noWORM storage club
Authors: Mkrtchyan Tigran , Adeyemi Olufemi , Fuhrmann Patrick , Garonne Vincent , Litvintsev Dmitry , Millar Paul , Rossi Albert , sahakyan Marina , Millar Paul , Starek Jürgen , Yasar Sibel
Published: September 2019

For over a decade, dCache.ORG has provided robust software, called dCache, that is used at more than 80 universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments and many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from all-in-one Raspberry-Pi up to hundreds of nodes in multi-petabyte infrastructures. The life cycle of scientific data is well defined - collected, processed, archived and finally deleted, when it’s not needed anymore. Moreover, during all those stages the data is never modified: either the original data is used, or new derived data is produced. With this knowledge, dCache was designed to handle immutable files as efficiently as possible. Data replication, HSM connectivity and data-server independent operations are only possible due to the immutable nature of stored data. Nowadays many commercial vendors provide such write-once-read-many or WORM storage systems, as they become more and more demanded with grown demand of audio, photo and video content in the web. On the other hand by providing standard NFSv4.1 interface dCache is often used as a general-purpose file-system, especially by new communities, like photon scientists or microbiologists. Although many users are aware of data immutability, some applications and use cases still require in-place updates of stored files. To satisfy new requirements some fundamental changes have to be applied to dCache’s core design. However, new developments must not compromise any aspect of existing functionality. In this presentation we will show new developments in dCache to turn it into a regular file system. We will discuss the challenges to build a distributed storage system, ‘life’ with POSIX compliance, handling of multiple replicas and backward compatibility by providing WORM and noWORM capabilities within the same storage system.
Evolution of the open-source data management system Ru-cio for LHC Run-3 and beyond ATLAS
Authors: Barisits Martin , Beermann Thomas , Bogado Joaquin , Garonne Vincent , Javurek Tomas , Lassnig Mario , Manzi Andrea , Martelli Edoardo , Serfon Cedric , Wegner Tobias , Zhao Xin
Published: September 2019

Rucio, the distributed data management system of the ATLAS experiment
already manages more than 400 Petabytes of physics data on the grid. Rucio
was incrementally improved throughout LHC Run-2 and is currently being
prepared for the HL-LHC era of the experiment. Next to these improvements
the system is currently evolving into a full-scale generic data management system
for application beyond ATLAS, or even beyond high-energy physics. This
contribution focuses on the development roadmap of Rucio for LHC Run-3,
such as event level data management, generic meta-data support and increased
usage of networks and tapes. At the same time Rucio is evolving beyond the
original ATLAS requirements. This includes additional authentication mechanisms,
generic database compatibility, deployment and packaging of the software
stack in containers, and a project paradigm shift to a full-scale open source
project..
Continuous Performance Benchmarking Framework for ROOT
Authors: Shadura Oksana , Vassilev Vassil , Bockelman Brian Paul
Published: September 2019

Foundational software libraries such as ROOT are under intense pressure to
avoid software regression, including performance regressions. Continuous performance
benchmarking, as a part of continuous integration and other code
quality testing, is an industry best-practice to understand how the performance
of a software product evolves. We present a framework, built from industry best
practices and tools, to help to understand ROOT code performance and monitor
the efficiency of the code for several processor architectures. It additionally
allows historical performance measurements for ROOT I/O, vectorization and
parallelization sub-systems.
A new approach for ATLAS Athena job configuration
Authors: Lampl Walter
Published: September 2019

The offline software framework of the ATLAS experiment (Athena) consists of many small components of various types like Algorithm, Tool or Service. To assemble these components into an executable application for event processing, a dedicated configuration step is necessary. The configuration of a particular job depends on the work-flow (simulation, reconstruction, high-level trigger, overlay, calibration, analysis ...) and the input data (real or simulated data, beam-energy, ...).

The configuration step is done by executing python code. The resulting config-uration depends on optionally pre-set flags as well as meta-data about the data to be processed. For the python configuration code, there is almost no structure enforced, leaving the full power of python to the user.

While this approach did work, it also proved to be error prone and complicated to use. It also leads to jobs containing more components that they actually need. For LHC Run 3 a more robust system is envisioned. It is still based on python but enforces a structure and emphasis modularity. This contribution briefly reports about the configuration system used during LHC Run 1 and Run 2 and details the prototype of an improved system to be used in Run 3 and beyond.
Porting the LHCb Stack from x86 (Intel) to aarch64 (ARM) and ppc64le (PowerPC)
Authors: Promberger Laura , Clemencic Marco , Couturier Ben , Iartza Aritz Brosa , Neufeld Niko
Published: September 2019

LHCb is undergoing major changes in its data selection and processing chain for the upcoming LHC Run 3 starting in 2021. With this in sight several initiatives have been launched to optimise the software stack. This contribution discusses porting the LHCb Stack from x86_64 architecture to both architectures aarch64 and ppc64le with the goal to evaluate the performance and the cost of the computing infrastructure for the High Level Trigger (HLT). This requires porting a stack with more than five million lines of code and finding working versions of external libraries provided by LCG. Across all software packages the biggest challenge is the growing use of vectorisation - as many vectorisation libraries are specialised on x86 architecture and do not have any support for other architectures. In spite of these challenges we have successfully ported the LHCb High Level Trigger code to aarch64 and ppc64le. This contribution discusses the status and plans for the porting of the software as well as the LHCb approach for tackling code vectorisation in a platform independent way.
The ATLAS multithreaded offline framework
Authors: Kama Sami , Leggett Charles , Snyder Scott , Tsulaia Vakho
Published: September 2019

In preparation for Run 3 of the LHC, scheduled to start in 2021, the ATLAS experiment is revising its offline software so as to better take advantage of machines with many cores. A major part of this effort is migrating the software to run as a fully multithreaded application, as this has been shown to significantly improve the memory scaling behavior. This note outlines changes made to the software framework to support this migration.
Building, testing and distributing common software for the LHC experiments
Authors: Cervantes Villanueva Javier , Ganis Gerardo , Konstantinov Dmitri , Latyshev Grigorii , Mato Vila Pere , Mendez Lorenzo Patricia , Pacholek Rafal , Razumov Ivan
Published: September 2019

Building, testing and deploying of coherent large software stacks is very challenging, in particular when they consist of the diverse set of packages required by the LHC experiments, the CERN Beams Department and data analysis services such as SWAN. These software stacks include several packages (Grid middleware, Monte Carlo generators, Machine Learning tools, Python modules) all available for a large number of compilers, operating systems and hardware architectures.

To address this challenge, we developed an infrastructure around a tool called lcgcmake. Dedicated modules are responsible for building the packages, con-trolling the dependencies in a reliable and scalable way. The distribution relies on a robust and automatic system, responsible for building and testing the packages, installing them on CernVM-FS and packaging the binaries in RPMs and tarballs. This system is orchestrated through Jenkins on build machines provided by the CERN Openstack facility. The results are published through user-friendly web pages.

In this paper we will present an overview of these infrastructure tools and policies. We also discuss the role of this effort within the HEP Software Foundation (HSF). Finally we will discuss the evolution of the infrastructure towards container (Docker) technologies and the future directions and challenges of the project.
Evolution of the VISPA-project
Authors: Erdmann Martin , Fischer Benjamin , Geiger Lukas , Geiser Erik , Noll Dennis Daniel Nick , Rath Yannik Alexander , Rieger Marcel , Schlüter Felix , Schmidt David Josef , Urban Martin , von Cube Ralf Florian
Published: September 2019

VISPA (Visual Physics Analysis) is a web-platform that enables users to work on any secure shell (SSH) reachable resource using just their webbrowser. It is used successfully in research and education for HEP data analysis. The emerging JupyterLab is an ideal choice for a comprehensive, browser-based, and extensible work environment and we seek to unify it with the efforts of the VISPA-project. The primary objective is to provide the user with the freedom to access any external resources at their disposal, while maintaining a smooth integration of preconfigured ones including their access permissions. Additionally, specialized HEP tools, such as native format data browsers (ROOT, PXL), are being migrated from VISPA- to JupyterLab-extensions as well. We present these concepts and their implementation progress.
RDMA-accelerated data transport in ALFA
Authors: Klein Dennis , Rybalchenko Alexey , Al-Turany Mohammad , Kollegger Thorsten
Published: September 2019

ALFA is a modern software platform for simulation, reconstruction and analysis of particle physics experiments. The FairMQ library in ALFA provides building blocks for distributed processing pipelines in anticipation of high data rates in next-generation, trigger-less FAIR and LHC RUN3 ALICE experiments. Modern data transport technologies are integrated through FairMQ by implementing an abstract message queuing based transport interface. Current implementations are based on ZeroMQ, nanomsg and shared memory and can be selected at run-time. In order to achieve highest inter-node data throughput on high bandwidth network fabrics (e.g. Infiniband), we propose a new FairMQ transport implementation based on the libfabric technology.
A modular software framework for test-beam data analysis. The TbGaudi package
Authors: Rachwał Bartłomiej , Szumlak Tomasz
Published: September 2019

For purposes of test beam off-line data analysis a modular software framework has been developed and described in theses proceedings. The TbGaudi package allows to handle the test beam data for a set of runs, as well as the set of different devices under test in one-go, and finally obtain an integrated workflow to present the results.

All code is written in C++ programming language. A class based design makes it flexible to add any new features of the device under investigation following a plug-in scheme. Currently, the toolkit handles different types of analysis such as charge collection efficiency, track resolution and eta correction, implemented for non-uniform irradiated sensors.

The framework is applied for the LHCb upgrade studies for the VELO and UT groups.
Sustainability of the Merlin++ particle tracking code
Authors: Rowan Scott , Tygier Sam , Cai Yuanfang , Venters Colin C. , Appleby Robert B. , Barlow Roger J.
Published: September 2019

Merlin++ is a C++ particle accelerator and particle tracking library originally developed at DESY for use in International Linear Collider (ILC) simulations. Merlin++ has more recently been adapted for High-Luminosity Large Hadron Collider (HL-LHC) collimation studies, utilizing advanced scattering physics. However, as is all too common in long-standing high-energy physics software, recent developments have focused on functional additions rather than code design and maintainability. This has resulted in usability issues for users and developers alike. The following presents recent improvements in adhering to modern software sustainability practices to address these issues. Quantifiable improvements in code complexity and maintainability are presented via appropriate test metrics and the evolution of the software architecture is analyzed. Experiences and conclusions of applying modern sustainability methodology to longstanding scientific software are discussed.
A plugin-based approach to data analysis for the AMS experiment on the ISS
Authors: Formato Valerio
Published: September 2019

In many HEP experiments a typical data analysis workflow requires each user to read the experiment data in order to extract meaningful information and produce relevant plots for the considered analysis. Multiple users accessing the same data result in a redundant access to the data itself, which could be factorized effectively improving the CPU efficiency of the analysis jobs and relieving stress from the storage infrastructure. To address this issue we present a modular and lightweight solution where the users code is embedded in different "analysis plugins" which are then collected and loaded at runtime for execution, where the data is read only once and shared between all the different plugins. This solution was developed for one of the data analysis groups within the AMS collaboration but is easily extendable to all kinds of analyses and workloads that need I/O access on AMS data or custom data formats and can even adapted with little effort to another HEP experiment data. This framework could then be easily embedded into a "analysis train" and we will discuss a possible implementation and different ways to optimise CPU efficiency and execution time.
The ATLAS Metadata Interface (AMI) 2.0 metadata ecosystem: new design principles and features
Authors: Odier Jérôme , Lambert Fabian , Fulachier Jérôme
Published: September 2019

ATLAS Metadata Interface (AMI) is a generic ecosystem for metadata aggregation, transformation and cataloging. Benefiting from 18 years of feedback in the LHC context, the second major version was recently released. This paper describes the design choices and their benefits for providing high-level metadata-dedicated features. In particular, the Metadata Querying Language (MQL) - a domain-specific language allowing to query databases without knowing the relation between entities - and on the AMI Web framework are described.
Machine learning techniques for jet flavour identification at CMS
Authors: Verzetti Mauro
Published: September 2019

Jet flavour identification is a fundamental component of the physics program of the LHC-based experiments. The presence of multiple flavours to be identified leads to a multiclass classification problem. We present results from a realistic simulation of the CMS detector, one of two multi-purpose detectors at the LHC, and the respective performance measured on data. Our tagger, named DeepJet, relies heavily on applying convolutions on lower level physics objects, like individual particles. This approach allows the usage of an unprecedented amount of information with respect to what is found in the literature. DeepJet stands out as the first proposal that can be applied to multi-classification for all jet flavours. We demonstrate significant improvements by the new approach on the classification capabilities of the CMS experiment in simulation in several of the tested classes. At high momentum improvements of nearly 90% less false positives at a standard operation point are reached.
Scaling studies for deep learning in Liquid Argon Time Projection Chamber event classification
Authors: Strube Jan , Bhattacharya Kolahal , Church Eric , Daily Jeff , Malachi Schram , Charles Siegel , Kevin Wierman
Published: September 2019

Measurements in Liquid Argon Time Projection Chamber neutrino detectors feature large, high fidelity event images. Deep learning techniques have been extremely successful in classification tasks of photographs, but their application to these event images is challenging, due to the large size of the events, more two orders of magnitude larger than images found in classical challenges like MNIST or ImageNet. This leads to extremely long training cycles, which slow down the exploration of new network architectures and hyperpa-rameter scans to improve the classification performance. We present studies of scaling an LArTPC classification problem on multiple architectures, spanning multiple nodes. The studies are carried out in simulated events in the Micro-BooNE detector.
Decay Chain Reconstruction at Belle II
Authors: Tenchini Francesco , Krohn Jo-Frederik
Published: September 2019

The Belle II detector underwent commissioning in 2018. Featuring a state of the art vertex detector with innovative pixel sensors, it will record collisions of e+e-beams from the SuperKEKB accelerator which is slated to provide luminosities 40 times higher than KEKB.

This will bring new opportunities and challenges for physicists seeking to perform precision measurements, as increasingly precise vertex position reconstruction and background rejection power is needed to take advantage of the unprecedented statistics provided.

We discuss the vertex reconstruction techniques available that make such measurements possible and present an approach to particle decay reconstruction based on the global fit of the full decay chain, implemented using a Kalman Filter. This technique is especially well suited to the treatment of processes with neutral decays. It allows for improved background rejection, access to a larger number of decay modes and a flexible framework for consistent treatment of measurement uncertainties.
Large-Scale Distributed Training Applied to Generative Adversarial Networks for Calorimeter Simulation
Authors: Vlimant Jean-Roch , Pantaleo Felice , Pierini Maurizio , Loncar Vladimir , Vallecorsa Sofia , Anderson Dustin , Nguyen Thong , Zlokapa Alexander
Published: September 2019

In recent years, several studies have demonstrated the benefit of using deep learning to solve typical tasks related to high energy physics data taking and analysis. In particular, generative adversarial networks are a good candidate to supplement the simulation of the detector response in a collider environment. Training of neural network models has been made tractable with the improvement of optimization methods and the advent of GP-GPU well adapted to tackle the highly-parallelizable task of training neural nets. Despite these advancements, training of large models over large data sets can take days to weeks. Even more so, finding the best model architecture and settings can take many expensive trials. To get the best out of this new technology, it is important to scale up the available network-training resources and, consequently, to provide tools for optimal large-scale distributed training. In this context, our development of a new training workflow, which scales on multi-node/multi-GPU architectures with an eye to deployment on high performance computing machines is described. We describe the integration of hyper parameter optimization with a distributed training framework using Message Passing Interface, for models defined in keras [12] or pytorch [13]. We present results on the speedup of training generative adversarial networks trained on a data set composed of the energy deposition from electron, photons, charged and neutral hadrons in a
fine grained digital calorimeter.
Exploring End-to-end Deep Learning Applications for Event Classification at CMS
Authors: Andrews Michael , Paulini Manfred , Gleyzer Sergei , Poczos Barnabas
Published: September 2019

An essential part of new physics searches at the Large Hadron Collider (LHC) at CERN involves event classification, or distinguishing potential signal events from those coming from background processes. Current machine learning techniques accomplish this using traditional hand-engineered features like particle 4-momenta, motivated by our understanding of particle decay phenomenology. While such techniques have proven useful for simple decays, they are highly dependent on our ability to model all aspects of the phenomenology and detector response. Meanwhile, powerful deep learning algorithms are capable of not only training on high-level features, but of performing feature extraction. In computer vision, convolutional neural networks have become the state-of-the-art for many applications. Motivated by their success, we apply deep learning algorithms to low-level detector data from the 2012 CMS Simulated Open Data to directly learn useful features, in what we call, end-to-end event classification. We demonstrate the power of this approach in the context of a physics search and offer solutions to some of the inherent challenges, such as image construction, image sparsity, combining multiple sub-detectors, and de-correlating the classifier from the search observable, among others.
Integrating a dynamic data federation into the ATLAS distributed data management system
Authors: Berghaus Frank , Wegner Tobias , Lassnig Mario , Ebert Marcus , Serfon Cedric , Galindo Fernando , Seuster Rolf , Garonne Vincent , Tafirout Reda , Sobie Randall
Published: September 2019

Input data for applications that run in cloud computing centres can be stored at remote repositories, typically with multiple copies of the most popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. In this approach, the closest copy of the data is used based on geographical or other information. Currently, we are using the dynamic data federation, Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standard interfaces, such as Amazon S3, Microsoft Azure and HTTP with WebDAV extensions. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate. We have set up an instance of Dynafed and integrated it into the ATLAS distributed data management system, Rucio. We report on the challenges faced during the installation and integration.
Merging OpenStack-based private clouds: the case of CloudVeneto.it
Authors: Andreetto Paolo , Chiarello Fabrizio , Costa Fulvia , Crescente Alberto , Fantinel Sergio , Fanzago Federica , Konomi Ervin , Mazzon Paolo Emilio , Menguzzato Matteo , Segatta Matteo , Sella Gianpietro , Sgaravatto Massimo , Traldi Sergio , Verlato Marco , Zangrando Lisa
Published: September 2019

The Cloud Area Padovana, deployed in 2014, is a scientific IaaS cloud, spread between two different sites: the INFN Padova Unit and the INFN Legnaro National Labs. It provides about 1100 logical cores and 50 TB of storage. The entire computing facility, owned by INFN, satisfies the computational and storage demands of more than 100 users belonging to about 30 research projects, mainly related to HEP and nuclear physics. The Padova data centre also has hosted and operated since 2015 an independent IaaS cloud managing network, storage and computing resources owned by 10 departments of the University of Padova, supporting a broader range of scientific and engineering disciplines. This infrastructure provides about 480 logical cores and 90 TB of storage and supports more than 40 research projects. These two clouds share only a limited set of ICT services and tools (mainly for configuration, monitoring and accounting), whereas their daily operations and maintenance are carried out separately by INFN and University personnel. At the end of 2017 we planned to merge the two infrastructures in order to optimise the use of resources (both human and ICT) and to avoid useless duplication of services. We discuss here how we plan to implement this integration, resulting in a single cloud infrastructure named CloudVeneto.it.
Lightweight WLCG Sites
Authors: Sharma Mayank , Litmaath Maarten , Silva Junior Eraldo , Santana Renato
Published: September 2019

This article describes a new framework, called SIMPLE, for settingup and maintaining classic WLCG sites with minimal operational efforts and insights needed into the WLCG middleware. The framework provides a single common interface to install and configure any of its supported grid services, such as Compute Elements, Batch Systems, Worker Nodes and miscellaneous middleware packages. It leverages modern container orchestration tools like Kubernetes, Docker Swarm, and confiuration management tools like Puppet, Ansible, to automate deployment of the WLCG services on behalf of a site admin. The framework is modular and extensible by design. Therefore, it is easy to add support for more grid services as well as infrastructure automation tools to accommodate diverse scenarios at different sites. We provide insight into the design of the framework and our efforts towards development, release and deployment of its first implementation featuring CREAM E, TORQUE Batch System and TORQUE based Worker Nodes.