bluuebunny/crossref_metadata_2025 huggingface dataset:
The DIRAC interware: current, upcoming and planned capabilities and technologies
Authors: Stagni Federico , Tsaregorodtsev Andrei , Sailer André , Haen Christophe
Published: November 2020

Efficient access to distributed computing and storage resources is mandatory for the success of current and future High Energy and Nuclear Physics Experiments. DIRAC is an interware to build and operate distributed computing systems. It provides a development framework and a rich set of services for the Workload, Data and Production Management tasks of large scientific communities. A single DIRAC installation provides a complete solution for the distributed computing of one, or more than one collaboration. The DIRAC Workload Management System (WMS) provides a transparent, uniform interface for managing computing resources. The DIRAC Data Management System (DMS) offers all the necessary tools to ensure data handling operations: it supports transparent access to storage resources based on multiple technologies, and is easily expandable. Distributed Data management can be performed, also using third party services, and operations are resilient with respect to failures. DIRAC is highly customizable and can be easily extended. For these reasons, a vast and heterogeneous set of scientific collaborations have adopted DIRAC as the base for their computing models. Users from different experiments can interact with the system in different ways, depending on their specific tasks, expertise level and previous experience using command line tools, python APIs or Web Portals. The requirements of the diverse DIRAC user communities and hosting infrastructures triggered multiple developments to improve the system usability: examples include the adoption of industry standard authorization and authentication infrastructure solutions, the management of diverse computing resources (cloud, HPC, GPGPU, etc.), the handling of high-intensity work and data flows, but also advanced monitoring and accounting using no-SQL based solutions and message queues. This contribution will highlight DIRAC’s current, upcoming and planned capabilities and technologies.
Migrating INFN-T1 from CREAM-CE/LSF to HTCondor-CE/HTCondor
Authors: Dal Pra S , Fornari F , Michelotto D , Chierici A
Published: November 2020

The INFN Tier-1 datacentre provides computing resources to several HEP and Astrophysics experiments. These are organized in Virtual Organizations submitting jobs to our computing facilities through Computing Elements, acting as Grid interfaces to the Local Resource Manager. We are phasing-out our current LRMS (IBM/Platform LSF 9.1.3) and CEs (CREAM) set to adopt HTCondor as a replacement for LSF and HTCondor-CE in place of CREAM. A small instance has been set up to practice with the cluster management and evaluate the feasibility of our migration plans to a new LRMS and CE set. A second cluster instance has been setup to work on production. A number of management tools have been adapted or rewritten in order to integrate the new system with the existing infrastructure. Two different accounting solution for the HTCondor-CE have been implemented, and the more reliable one have been adopted. A python tool has been written to disentangle the management of HTCondor machines from our puppet instance, and to enable a quicker configuration of the cluster nodes. The monitoring tools tied to the old system are being adapted to also work on the new one. Finally, the most relevant setup steps have been documented in a public wiki page and a support mailing has been created to help other INFN sites willing to migrate their LRMS and CE to HTCondor. This document reports about our experience with HTCondor-CE on top of HTCondor and the integration of this system into our infrastructure.
The Belle II Raw Data Management System
Authors: Hernández Villanueva Michel , Ueda Ikuo
Published: November 2020

The Belle II experiment, a major upgrade of the previouse+e−asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way to be easily accessed and analyzed by more than 1000 collaborators. In particular, the Belle II Raw Data Management system has been developed with an aim to upload output files onto grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II Distributed Data Management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control) and consists of a database, services, client and monitoring tools, and several agents that treat the data automatically. The first year of data taken with the Belle II full detector has been managed by the Belle II Raw Data Management system successfully. The design, current status, and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.
XRootD and Object Store: A new paradigm
Authors: Ellis Katy , Brew Chris , Patargias George , Adye Tim , Appleyard Rob , Dewhurst Alastair , Johnson Ian
Published: November 2020

The XRootD software framework is essential for data access at WLCG sites. The WLCG community is exploring and expanding XRootD functionality. This presents a particular challenge at the RAL Tier-1 as the Echo storage service is a Ceph based Erasure Coded object store. External access to Echo uses gateway machines which run GridFTP and caching servers. Local jobs access Echo via caches on every worker node, but it is clear there are inefficiencies in the system. Remote jobs also access data via XRootD on Echo. For CMS jobs this is via the AAA service. ATLAS, who are consolidating their storage at fewer sites, are increasingly accessing job input data remotely. This paper describes the continuing work to optimise both local and remote data access by testing different caching methods.
The ATLAS EventIndex for LHC Run 3
Authors: Barberis Dario , Aleksandrov Igor , Alexandrov Evgeny , Baranowski Zbigniew , Dimitrov Gancho , Fernández Casaní Álvaro , Gallas Elizabeth J. , García Montoro Carlos , González de la Hoz Santiago , Hrivnac Julius , Kazymov Andrei , Mineev Mikhail , Prokoshin Fedor , Rybkin Grigori , Sánchez Javier , Salt Cairols José , Villaplana Perez Miguel
Published: November 2020

The ATLAS EventIndex was designed in 2012-2013 to provide a global event catalogue and limited event-level metadata for ATLAS analysis groups and users during the LHC Run 2 (2015-2018). It provides a good and reliable service for the initial use cases (mainly event picking) and several additional ones, such as production consistency checks, duplicate event detection and measurements of the overlaps of trigger chains and derivation datasets. The LHC Run 3, starting in 2021, will see increased data-taking and simulation production rates, with which the current infrastructure would still cope but may be stretched to its limits by the end of Run 3. This proceeding describes the implementation of a new core storage service that will be able to provide at least the same functionality as the current one for increased data ingestion and search rates, and with increasing volumes of stored data. It is based on a set of HBase tables, with schemas derived from the current Oracle implementation, coupled to Apache Phoenix for data access; in this way we will add to the advantages of a BigData based storage system the possibility of SQL as well as NoSQL data access, allowing to re-use most of the existing code for metadata integration.
Named Data Networking based File Access for XRootD
Authors: Iordache Cǎtǎlin , Liu Ran , Balcas Justas , Šrivinskas Raimondas , Wu Yuanhao , Fan Chengyu , Shannigrahi Susmit , Newman Harvey , Yeh Edmund
Published: November 2020

We present the design and implementation of a Named Data Networking (NDN) based Open Storage System plug-in for XRootD. This is an important step towards integrating NDN, a leading future internet architecture, with the existing data management systems in CMS. This work outlines the first results of data transfer tests using internal as well as external 100 Gbps testbeds, and compares the NDN-based implementation with existing solutions.
Token-based authorization in StoRM WebDAV
Authors: Ceccanti Andrea , Vianello Enrico , Michelotto Diego
Published: November 2020

At the end of May 2017 the Globus Alliance announced that the open-source Globus Toolkit (GT) would be no longer supported by the Globus team at the University of Chicago. This announcement had an obvious impact on WLCG, given the central role of the Globus Security Infrastructure (GSI) and GridFTP in the WLCG data management framework, so discussions started in the appropriate forums on the search for alternatives. At the same time, support for token-based authentication and authorization has emerged as a key requirement for storage elements powering WLCG data centers. In this contribution, we describe the work done to enable token-based authentication and authorization in the StoRM WebDAV service, describing and highlighting the differences between support for external OpenID connect providers, groupbased and capability-based authorization schemes, and locally-issued authorization tokens. We discuss how StoRM WebDAV token-based authorization is being exploited in several contexts, from WLCG DOMA activities to other scientific experiments hosted at the INFN Tier-1 data center. In this contribution, we also describe the methodology used to compare Globus GridFTP and StoRM WebDAV and we present initial results confirming how HTTP represent a viable alternative to GridFTP for data transfers also performance-wise.
Development of the Dataset Searcher Webapp for finding data on the Belle II computing grid
Authors: Smith Kim , Dossett David , Sevior Martin
Published: November 2020

In any large scale scientific experiment involving enormous quantities of data it is crucial that everyone involved has quick and easy access to all the relevant datasets for their research. By the end of the run time of the Belle II experiment there will be a projected 50 ab−1 of integrated luminosity making it no exception. Until now the only method for locating data of interest was by looking up hand written tables that needed to be regularly updated. In this paper, a new webapp built on the DIRAC software framework will be presented which aims to be the new standard for not only locating data but also storing all its associated metadata.
The Quest to solve the HL-LHC data access puzzle
Authors: Espinal X. , Jezequel S. , Schulz M. , Sciabà A. , Vukotic I. , Wuerthwein F.
Published: November 2020

HL-LHC will confront the WLCG community with enormous data storage, management and access challenges. These are as much technical as economical. In the WLCG-DOMA Access working group, members of the experiments and site managers have explored different models for data access and storage strategies to reduce cost and complexity, taking into account the boundary conditions given by our community.Several of these scenarios have been evaluated quantitatively, such as the Data Lake model and incremental improvements of the current computing model with respect to resource needs, costs and operational complexity.To better understand these models in depth, analysis of traces of current data accesses and simulations of the impact of new concepts have been carried out. In parallel, evaluations of the required technologies took place. These were done in testbed and production environments at small and large scale.We will give an overview of the activities and results of the working group, describe the models and summarise the results of the technology evaluation focusing on the impact of storage consolidation in the form of Data Lakes, where the use of streaming caches has emerged as a successful approach to reduce the impact of latency and bandwidth limitation.We will describe the experience and evaluation of these approaches in different environments and usage scenarios. In addition we will present the results of the analysis and modelling efforts based on data access traces of the experiments.
Transitioning CMS to Rucio Data Managment
Authors: Vaandering Eric
Published: November 2020

Following a thorough review in 2018, the CMS experiment at the CERN LHC decided to adopt Rucio as its new data management system. Rucio is emerging as a community software project and will replace an aging CMSonly system before the start-up of LHC Run 3 in 2021. Rucio was chosen after an evaluation determined that Rucio could meet the technical and scale needs of CMS. The data management system for CMS needs to manage the current data sample of approximately 200 PB of data with 1 PB of transfers per day. The data management system must also have a development path suitable for LHC Run 4 (2026) data rates, which are expected to be 50 times larger.

This contribution details the ongoing CMS adoption process as we replace our legacy system with Rucio, focusing on the challenges of integrating Rucio into an established set of experimental tools and procedures. This will include the migration of metadata, the construction of interfaces to the rest of the CMS computing tools, scale tests, operations, monitoring, and the plan to gradually turn over primary responsibility for the management of data to Rucio. A description of CMS user data management with Rucio will also be provided.
XRootD Third Party Copy for the WLCG and HLLHC
Authors: Adye T , Bockelman B , Ellis K , Freyermuth O , Furano F , Ganis G , Hanushevsky A , Ito H , Johnson I , Keeble O , Litvintsev D , Manzi A , Millar P , Mkrtchyan T , Patargias G , Rossi A , Severini H , Simon M , Sindrilaru E , Yang W
Published: November 2020

A Third Party Copy (TPC) mechanism has existed in the pure XRootD storage environment for many years. However, using the XRootD TPC in the WLCG environment presents additional challenges due to the diversity of the storage systems involved such as EOS, dCache, DPM and ECHO, requiring that we carefully navigate the unique constraints imposed by these storage systems and their site-specific environments through customized configuration and software development. To support multitenant setups seen at many WLCG sites, X509 based authentication and authorization in XRootD was significantly improved to meet both security and functionality requirements. This paper presents architecture of the pull based TPC with optional X509 credential delegation, and how it is implemented in native XRootD and dCache. The paper discusses technical requirements, challenges, design choices and implementation details in the WLCG storage systems, as well as in FTS/gfal2. It also outlines XRootD’s plan to support newer TPC and security models such as token based authorization.
Data-centric Graphical User Interface of the ATLAS Event Index Service
Authors: Hřivnáč Julius , Alexandrov Evgeny , Alexandrov Igor , Baranowski Zbigniew , Barberis Dario , Dimitrov Gancho , Fernandez Casani Alvaro , Gallas Elizabeth , García Montoro Carlos , Gonzalez de la Hoz Santiago , Kazymov Andrei , Mineev Mikhail , Prokoshin Fedor , Rybkin Grigori , Sanchez Javier , Salt Jose , Villaplana Perez Miguel
Published: November 2020

The Event Index service of the ATLAS experiment at the LHC keeps references to all real and simulated events. Hadoop Map files and HBase tables are used to store the Event Index data, a subset of data is also stored in the Oracle database. Several user interfaces are currently used to access and search the data, from a simple command line interface, through a programmable API, to sophisticated graphical web services. It provides a dynamic graph-like overview of all available data (and data collections). Data are shown together with their relations, like paternity or overlaps. Each data entity then gives users a set of actions available for the referenced data. Some actions are provided directly by the Event Index system, others are just interfaces to different ATLAS services. In many cases, specialized views are offered for detailed data inspection, such as histograms, Venn diagrams, etc.

This paper documents the current status of the service, its features and performance. The future system evolution to the new Event Index architecture based on the Apache Phoenix is also described as well as possible extension to a more general framework for giving a new, more intuitive access to experiment data.
Deploying and administrating the ATLAS Metadata Interface (AMI) 2.0 ecosystem
Authors: Odier Jérôme , Fulachier Jérôme , Lambert Fabian
Published: November 2020

ATLAS Metadata Interface (AMI) is a generic software ecosystem for metadata aggregation, transformation and cataloguing. Benefiting from about 20 years of feedback in the LHC context, the second major version was released in 2018. This paper describes how to install and administrate AMI version 2. A particular focus is given to the registration of existing databases in AMI, the adding of additional metadata and, finally, the generation of high level HTML 5 search interfaces using a dedicated wizard.
Recent developments in histogram libraries
Authors: Dembinski Hans Peter , Pivarski Jim , Schreiner Henry
Published: November 2020

Boost.Histogram, a header-only C++14 library that provides multidimensional histograms and profiles, became available in Boost 1.70. It is extensible, fast, and uses modern C++ features. Using template metaprogramming, the most efficient code path for any given configuration is automatically selected. The library includes key features designed for the particle physics community, such as optional under- and overflow bins, weighted increments, reductions, growing axes, thread-safe filling, and memory-efficient counters with high-dynamic range.

Python bindings for Boost.Histogram are being developed in the Scikit-HEP project to provide a fast, easy-to-install package as a backend for other Python libraries and for advanced users to manipulate histograms. Versatile and efficient histogram filling, effective manipulation, multithreading support, and other features make this a powerful tool. This library has also driven package distribution efforts in Scikit-HEP, allowing binary packages hosted on PyPI to be available for a very wide variety of platforms.

Two other libraries fill out the remainder of the Scikit-HEP Python histogramming effort. Aghast is a library designed to provide conversions between different forms of histograms, enabling interaction between histogram libraries, often without an extra copy in memory. This enables a user to make a histogram in one library and then save it in another form, such as saving a Boost.Histogram in ROOT. And Hist is a library providing friendly, analyst-targeted syntax and shortcuts for quick manipulations and fast plotting using these two libraries.
BSM: Bundled Software Manager and the application for CEPC
Authors: Zhao Xianghu , Ruan Manqi , Li Gang , Zhang Xiaomei
Published: November 2020

The Circular Electron Positron Collider (CEPC) is designed as a future Higgs Factory. Like most high energy physics experiments, the offline software consists of many packages. BSM (Bundled Software Manager) is thus created in order to simplify the deployment and usage of software which has many packages and dependencies. BSM utilizes Git as the software release repository. The details of software are defined in the Git repository including installation instructions, environment, dependencies, etc. Commands are supported for various shells. Json output and python API are also available for advanced development. The installation of each package could be configured separately and extended with customized handlers. BSM manages the environment variables and the version switching is easy. It also has fine environment control on a single package. Users can also define their own packages easily and these packages will be managed by BSM with simple configuration. CEPC software has already set up the deployment procedure with BSM. And BSM is also designed with flexibility to create different applications other than CEPC software. It is suitable for the projects including a lot of packages and it is safe for different BSM applications to coexist with each other under proper configuration.
Modularization of the LHCb software environment and preparation for heterogeneous resources
Authors: Clemencic Marco , Couturier Ben
Published: November 2020

LHCb software runs in very diﬀerent computing environments: the trigger farm at CERN, on the LHC Computing Grid (LCG), on shared clusters or on software developer’s desktops. . . The old model assumes the availability of CVMFS and relies on custom scripts (a.k.a LbScripts) to configure the environment to build and run the software. It lacks flexibility and does not allow, for example running in container and it is very diﬃcult to extend them to configure and run on new environments. This paper describes the steps taken to modularize those tools to allow for easier development and deployment (as standard Python packages), but also added integration with container technology to better support non standard environments.
Offline Software Management of the AMS experiment
Authors: Choutko Vitali , Eogorov Alexander , Eline Alexandre , Shan Baosong
Published: November 2020

The Alpha Magnetic Spectrometer (AMS) is a particle physics experiment installed and operating aboard the International Space Station (ISS) from May 2011 and expected to last through 2028 and beyond. The AMS offline software is used for data reconstruction, Monte-Carlo simulation and physics analysis. This paper presents how we manage the offline software, including the version control, automatic integration, automatic deployment, and the distribution.
A multi-purpose user interface for the iFDAQ of the COMPASS experiment
Authors: Květoň Antonín , Bodlák Martin , Frolov Vladimir , Huber Stefan , Jarý Vladimír , Konorov Igor , Nový Josef , Steffen Dominik , Šubrt Ondřej , Virius Miroslav
Published: November 2020

In HEP experiments, remote access to control systems is one of the fundamental pillars of efficient operations. At the same time, development of user interfaces with emphasis on usability can be one of the most labor-intensive software tasks to be undertaken in the life cycle of an experiment. While desirable, the development and maintenance of a large variety of interfaces (e.g., desktop control interface, web monitoring interface, development API...) is often simply not feasible, as far as manpower is concerned. We present a solution employed in the control software of the iFDAQ of the COMPASS experiment at CERN. Being a mix of a command-line and terminal tool, this interface can fulfill the roles of a dynamic monitoring interface, a control interface, and a scripting API simultaneously. Furthermore, it can easily be used as a remote access tool for operations experts, needing nearly no setup user-side and being compatible with smartphones. We also discuss the methodology and results of a concrete use case – automated run control for performance tests of the iFDAQ readout software.
SpackDev: Multi-Package Development with Spack
Authors: Green Chris , Amundson James , Garren Lynn , Gartung Patrick , Sexton-Kennedy Elizabeth
Published: November 2020

High Energy Physics (HEP) software environments often involve ∼ hundreds of external packages and libraries, and similar numbers of experiment-specific, science-critical packages—many under continuous development. Managing coherent releases of the external software stack is challenging enough, but managing the highly-collaborative—and distributed—development of a large body of code against such a stack adds even more complexity and room for error.

Spack is a popular Python-based package management tool with a specific focus on the needs of High Performance Computing (HPC) systems and system administrators whose strength is orchestrating the discrete download, build, testing, and installation of pre-packaged or tagged third-party software against similarly stable dependencies. As such it is becoming increasingly popular within HEP as that community makes increasing use of HPC facilities, and as efforts to develop future HPC systems utilize Spack to provide scientific software on those platforms [1].

SpackDev is a system to facilitate the simultaneous development of interconnected sets of packages. Intended to handle packages without restriction to one internal build system, SpackDev is integrated with Spack as a command extension in order to leverage features such as dependency calculations and build system configuration, and is generally applicable outside HEP. We describe SpackDev’s features and development over the last two years, initial experience using SpackDev in the context of the LArSoft liquid argon detector toolkit, and work remaining before it can be considered a fully-functional multi-package build system for HEP experiments utilizing Spack.
Evolution of the LHCb Continuous Integration system
Authors: Currie Robert , Mataev Rosen , Clemencic Marco
Published: November 2020

The physics software stack of LHCb is based on Gaudi and is comprised of about 20 interdependent projects, managed across multiple GitLab repositories. At present, the continuous integration (CI) system used for regular building and testing of this software is implemented using Jenkins and runs on a cluster of about 300 cores.

LHCb CI pipelines are python-based and relatively modern with some degree of modularity, i.e. the separation of test jobs from build jobs. However, these still suffer from obsoleted design choices that prevent improvements to scalability and reporting. In particular, the resource use and speed have not been thoroughly optimized due to the predominant use of the system for nightly builds, where a feedback time of 8 hours is acceptable. We describe recent work on speeding up pipelines by aggressively splitting and parallelizing checkout, build and test jobs and caching their artifacts. The current state of automatic code quality integration, such as coverage reports, is shown.

This paper presents how feedback time from change (merge request) submission to build and test reports is reduced from “next day” to a few hours by dedicated on-demand pipelines. Custom GitLab integration allows easy triggering of pipelines, including linked changes to multiple projects, and provides immediate feedback as soon as ready. Reporting includes a comparison to tests on a unique stable reference build, dynamically chosen for every set of changes under testing. This work enables isolated testing of changes that integrates well into the development workflow, leaving nightly testing primarily for integration tests.
The use of Convolutional Neural Networks for signal-background classification in Particle Physics experiments
Authors: Ayyar Venkitesh , Bhimji Wahid , Gerhardt Lisa , Robertson Sally , Ronaghi Zahra
Published: November 2020

The success of Convolutional Neural Networks (CNNs) in image classification has prompted efforts to study their use for classifying image data obtained in Particle Physics experiments. Here, we discuss our efforts to apply CNNs to 2D and 3D image data from particle physics experiments to classify signal from background.

In this work we present an extensive convolutional neural architecture search, achieving high accuracy for signal/background discrimination for a HEP classification use-case based on simulated data from the Ice Cube neutrino observatory and an ATLAS-like detector. We demonstrate among other things that we can achieve the same accuracy as complex ResNet architectures with CNNs with less parameters, and present comparisons of computational requirements, training and inference times.
Study of the influence of initial-state fluctuations on hydrodynamic simulations
Authors: Słodkowski Marcin , Gawryszewski Patryk , Setniewski Dominik
Published: November 2020

In this work, we are focusing on assessing the contribution of the initial-state fluctuations of heavy ion collision in the hydrodynamic simulations. We are trying to answer the question of whether the hydrodynamic simulation retains the same level of fluctuation in the final-state as for the initial stage. In another scenario, the hydrodynamic simulations of the fluctuation drowns in the final distribution of expanding matter. For this purpose, we prepared sufficient relativistic hydrodynamic program to study A+A interaction which allows analysing initial-state fluctuations in the bulk nuclear matter. For such an assumption, it is better to use high spatial resolution. Therefore, we applied the (3+1) dimensional Cartesian coordinate system. We implemented our program using parallel computing on graphics cards processors - Graphics Processing Unit (GPU). Simulations were carried out with various levels of fluctuation in initial conditions using the average method of events coming from UrQMD models. Energy density distributions were analysed and the contribution of fluctuations in initial conditions was assessed in the hydrodynamic simulation.
Coffea Columnar Object Framework For Effective Analysis
Authors: Smith Nicholas , Gray Lindsey , Cremonesi Matteo , Jayatilaka Bo , Gutsche Oliver , Hall Allison , Pedro Kevin , Acosta Maria , Melo Andrew , Belforte Stefano , Pivarski Jim
Published: November 2020

The coffea framework provides a new approach to High-Energy Physics analysis, via columnar operations, that improves time-to-insight, scalability, portability, and reproducibility of analysis. It is implemented with the Python programming language, the scientific python package ecosystem, and commodity big data technologies. To achieve this suite of improvements across many use cases, coffea takes a factorized approach, separating the analysis implementation and data delivery scheme. All analysis operations are implemented using the NumPy or awkward-array packages which are wrapped to yield user code whose purpose is quickly intuited. Various data delivery schemes are wrapped into a common front-end which accepts user inputs and code, and returns user defined outputs. We will discuss our experience in implementing analysis of CMS data using the coffea framework along with a discussion of the user experience and future directions.
Evolution of the ATLAS analysis model for Run-3 and prospects for HL-LHC
Authors: Elmsheuser Johannes , Anastopoulos Christos , Boyd Jamie , Catmore James , Gray Heather , Krasznahorkay Attila , McFayden Josh , Meyer Christopher John , Sfyrla Anna , Strandberg Jonas , Suruliz Kerim , Theveneaux-Pelzer Timothee
Published: November 2020

With an increased dataset obtained during the Run-2 of the LHC at CERN, the even larger forthcoming Run-3 data and the expected increase of the dataset by more than one order of magnitude for the HL-LHC, the ATLAS experiment is reaching the limits of the current data production model in terms of disk storage resources. The anticipated availability of an improved fast simulation will enable ATLAS to produce significantly larger Monte Carlo samples with the available CPU, which will then be limited by insuﬃcient disk resources. The ATLAS Analysis Model Study Group for Run-3 was setup at the end of Run-2. Its tasks have been to analyse the eﬃciency and suitability of the current analysis model and to propose significant improvements. The group has considered options allowing ATLAS to save, for the same sample of data and simulated events, at least 30% disk space overall, and has given recommendations on how significantly larger savings could be realised for the HL-LHC. Furthermore, suggestions were made to harmonise the current stage of analysis across the collaboration. The group has now completed its work: key recommendations will be the new small sized analysis formats DAOD_PHYS and DAOD_PHYSLITE and the increased usage of a tape carousel mode in the centralised production of these formats. This proceeding reviews the recommended ATLAS analysis model for Run-3 and the status of its implementation. It also provides an outlook to the HL-LHC analysis.
ML Track Fitting in Nuclear Physics
Authors: Britton Thomas , Lawrence David , Gavalian Gagik
Published: November 2020

Charged particle tracking represents the largest consumer of CPU resources in high data volume Nuclear Physics (NP) experiments. An effort is underway to develop machine learning (ML) networks that will reduce the resources required for charged particle tracking. Tracking in NP experiments represent some unique challenges compared to high energy physics (HEP). In particular, track finding typically represents only a small fraction of the overall tracking problem in NP. This presentation will outline the differences and similarities between NP and HEP charged particle tracking and areas where ML learning may provide a benefit. The status of the specific effort taking place at Jefferson Lab will also be shown.
Likelihood preservation and statistical reproduction of searches for new physics
Authors: Feickert Matthew , Heinrich Lukas , Stark Giordon
Published: November 2020

Likelihoods associated with statistical fits in searches for new physics are beginning to be published by LHC experiments on HEPData. The first of these is the search for bottom-squark pair production by ATLAS. These likelihoods adhere to a specification first defined by the HistFactory p.d.f. template. This is per-se independent of its implementation in ROOT and it is useful to be able to run statistical analysis outside of the ROOT and RooStats/RooFit framework. We introduce a JSON schema that fully describes the HistFactory statistical model and is sufficient to reproduce key results from published ATLAS analyses. Using two independent implementations of the model, one in ROOT and one in pure Python, we reproduce the sbottom multi-blimits using the published likelihoods on HEPData underscoring the implementation independence and long-term viability of the archived data.
Machine Learning with ROOT/TMVA
Authors: Albertsson Kim , An Sitong , Gleyzer Sergei , Moneta Lorenzo , Niermann Joana , Wunsch Stefan , Zampieri Luca , Zapata Mesa Omar Andres
Published: November 2020

ROOT provides, through TMVA, machine learning tools for data analysis at HEP experiments and beyond. We present recently included features in TMVA and the strategy for future developments in the diversified machine learning landscape. Focus is put on fast machine learning inference, which enables analysts to deploy their machine learning models rapidly on large scale datasets. The new developments are paired with newly designed C++ and Python interfaces supporting modern C++ paradigms and full interoperability in the Python ecosystem. We present as well a new deep learning implementation for convolutional neural network using the cuDNN library for GPU. We show benchmarking results in term of training time and inference time, when comparing with other machine learning libraries such as Keras/Tensorflow.
zfit: scalable pythonic fitting
Authors: Eschle Jonas , Navarro Puig Albert , Silva Coutinho Rafael , Serra Nicola
Published: November 2020

Statistical modeling and fitting is a key element in most HEP analyses. This task is usually performed in the C++ based framework ROOT/RooFit. Recently the HEP community started shifting more to the Python language, which the tools above are only loose integrated into, and a lack of stable, native Python based toolkits became clear. We presented zfit, a project that aims at building a fitting ecosystem by providing a carefully designed, stable API and a workflow for libraries to communicate together with an implementation fully integrated into the Python ecosystem. It is built on top of one of the state-of-theart industry tools, TensorFlow, which is used the main computational backend. zfit provides data loading, extensive model building capabilities, loss creation, minimization and certain error estimation. Each part is also provided with convenient base classes built for customizability and extendability.
Constraining effective field theories with machine learning
Authors: Brehmer Johann , Cranmer Kyle , Espejo Irina , Held Alexander , Kling Felix , Louppe Gilles , Pavez Juan
Published: November 2020

An important part of the Large Hadron Collider (LHC) legacy will be precise limits on indirect effects of new physics, framed for instance in terms of an effective field theory. These measurements often involve many theory parameters and observables, which makes them challenging for traditional analysis methods. We discuss the underlying problem of “likelihood-free” inference and present powerful new analysis techniques that combine physics insights, statistical methods, and the power of machine learning. We have developed MadMiner, a new Python package that makes it straightforward to apply these techniques. In example LHC problems we show that the new approach lets us put stronger constraints on theory parameters than established methods, demonstrating its potential to improve the new physics reach of the LHC legacy measurements. While we present techniques optimized for particle physics, the likelihood-free inference formulation is much more general, and these ideas are part of a broader movement that is changing scientific inference in fields as diverse as cosmology, genetics, and epidemiology.
Optimising HEP parameter fits via Monte Carlo weight derivative regression
Authors: Valassi Andrea
Published: November 2020

HEP event selection is traditionally considered a binary classification problem, involving the dichotomous categories of signal and background. In distribution fits for particle masses or couplings, however, signal events are not all equivalent, as the signal differential cross section has different sensitivities to the measured parameter in different regions of phase space. In this paper, I describe a mathematical framework for the evaluation and optimization of HEP parameter fits, where this sensitivity is defined on an event-by-event basis, and for MC events it is modeled in terms of their MC weight derivatives with respect to the measured parameter. Minimising the statistical error on a measurement implies the need to resolve (i.e. separate) events with different sensitivities, which ultimately represents a non-dichotomous classification problem. Since MC weight derivatives are not available for real data, the practical strategy I suggest consists in training a regressor of weight derivatives against MC events, and then using it as an optimal partitioning variable for 1-dimensional fits of data events. This CHEP2019 paper is an extension of the study presented at CHEP2018: in particular, event-by-event sensitivities allow the exact computation of the “FIP” ratio between the Fisher information obtained from an analysis and the maximum information that could possibly be obtained with an ideal detector. Using this expression, I discuss the relationship between FIP and two metrics commonly used in Meteorology (Brier score and MSE), and the importance of “sharpness” both in HEP and in that domain. I finally point out that HEP distribution fits should be optimized and evaluated using probabilistic metrics (like FIP or MSE), whereas ranking metrics (like AUC) or threshold metrics (like accuracy) are of limited relevance for these specific problems.
Partial wave analysis with OpenAcc
Authors: Xiao Yanjia , Ji Xiaobin , Liu Beijiang , Xiong Xi’an
Published: November 2020

Partial wave analysis(PWA) is an important tool in hadron physics. Large data sets from the experiments in high precision frontier require high computational power. To utilize GPU cluster and the resource of super computers with various types of accelerator, we implement a software framework for partial wave analysis using OpenAcc, OpenAccPWA. OpenAccPWA provides convenient approaches for exposing parallelism in the code and excellent support for the large amount of existing CPU-based codes of partial wave amplitudes. It can avoid heavy workload of code migration from CPU to GPU. This proceeding will briefly introduce the software framework and performance of OpenAccPWA.
Striped Data Analysis Framework
Authors: Gutsche Oliver , Mandrichenko Igor
Published: November 2020

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.
Consolidating the interactive analysis and Grid infrastructure at DESY
Authors: Beyer Christoph , Finnern Thomas , Flemming Martin , Gellrich Andreas , Hartmann Thomas , Kemp Yves , Lewendel Birgit , Reppin Johannes , Sever Krunoslav , Sternberger Sven , Voss Christian
Published: November 2020

Within WLCG, the DESY site in Hamburg is one of the largest Tier-2 sites with about 18500 CPU cores for Grid workloads. Additionally, about 8000 CPU cores are available for interactive user analyses in the National Analysis Factory [NAF]. After migrating these two batch systems onto a common HTCondor based set-up during the previous four years, we recapitulate the lessons learned during the transition especially since both use cases differ in their workloads. For Grid jobs start-up latencies are negligible and the primary focus is on an optimal utilization of the resources. Complementary, users of the NAF expect a high responsiveness of the batch system as well as the storage for interactive analyses. In this document, we will also give an outlook to future developments and concepts for the DESY high-throughput computing. In the ongoing evolution of the HTC batch system, we are exploring how to integrate anonymous jobs with the batch system as back-end for Function-as-a-Service workflows as well as an option for dynamic expansions to remote computing resources.
Standalone containers with ATLAS offline software
Authors: Vogel Marcelo , Borodin Mikhail , Forti Alessandra , Heinrich Lukas
Published: November 2020

This paper describes the deployment of the offline software of the ATLAS experiment at LHC in containers for use in production workflows such as simulation and reconstruction. To achieve this goal we are using Docker and Singularity, which are both lightweight virtualization technologies that can encapsulate software packages inside complete file systems. The deployment of offline releases via containers removes the interdependence between the runtime environment needed for job execution and the configuration of the computing nodes at the sites. Docker or Singularity would provide a uniform runtime environment for the grid, HPCs and for a variety of opportunistic resources. Additionally, releases may be supplemented with a detector’s conditions data, thus removing the need for network connectivity at computing nodes, which is normally quite restricted for HPCs. In preparation to achieve this goal, we have built Docker and Singularity images containing single full releases of ATLAS software for running detector simulation and reconstruction jobs in runtime environments without a network connection. Unlike similar efforts to produce containers by packing all possible dependencies of every possible workflow into heavy images (≈ 200GB), our approach is to include only what is needed for specific workflows and to manage dependencies efficiently via software package managers. This approach leads to more stable packaged releases where the dependencies are clear and the resulting images have more portable sizes ( 16GB). In an effort to cover a wider variety of workflows, we are deploying images that can be used in raw data reconstruction. This is particularly challenging due to the high database resource consumption during the access to the experiment’s conditions payload when processing data. We describe here a prototype pipeline in which images are provisioned only with the conditions payload necessary to satisfy the jobs’ requirements. This database-on-demand approach would keep images slim, portable and capable of supporting various workflows in a standalone fashion in environments with no network connectivity.
A fully unprivileged CernVM-FS
Authors: Blomer Jakob , Dykstra Dave , Ganis Gerardo , Mosciatti Simone , Priessnitz Jan
Published: November 2020

The CernVM File System provides the software and container distribution backbone for most High Energy and Nuclear Physics experiments. It is implemented as a file system in user-space (Fuse) module, which permits its execution without any elevated privileges. Yet, mounting the file system in the first place is handled by a privileged suid helper program that is installed by the Fuse package on most systems. The privileged nature of the mount system call is a serious hindrance to running CernVM-FS on opportunistic resource and supercomputers. Fortunately, recent developments in the Linux kernel and in the Fuse user-space libraries enabled fully unprivileged mounting for Fuse file systems (as of RHEL 8), or at least outsourcing the privileged mount system call to a custom, external process. This opens the door to several, very appealing new ways to use CernVM-FS, such as a generally usable “super pilot” consisting of the pilot code bundled with Singularity and CernVM-FS, or the on-demand instantiation of unprivileged, ephemeral containers to publish new CernVM-FS content from anywhere. In this contribution, we discuss the integration of these new Linux features with CernVM-FS and show some of its most promising, new applications.