bluuebunny/crossref_metadata_2025 huggingface dataset:
CERN AppStore: Development of a multi-platform application management system for BYOD devices at CERN
Authors: Bató Tamás , Bukowiec Sebastian , Kwiatek Michal
Published: November 2020

The number of BYOD continuously grows at CERN. Additionally, it is desirable to move from a centrally managed model to a distributed model where users are responsible for their own devices. Following this strategy, the new tools have to be provided to distribute and - in case of licensed software - also track applications used by CERN users. The available open source and commercial solutions were analyzed and none of them proved to be a good fit for CERN use cases. Therefore, it was decided to develop a system that could integrate various open source solutions and provide desired functionality for multiple platforms, both mobile and desktop. This paper presents the architecture and design decisions made to achieve a platform-independent, modern, maintainable and extensible system for software distribution at CERN.
Analysis Tools for the VyPR Performance Analysis Framework for Python
Authors: Dawes Joshua Heneage , Han Marta , Reger Giles , Franzoni Giovanni , Pfeiffer Andreas
Published: November 2020

VyPR (http://pyvypr.github.io/home/) is a framework being developed with the aim of automating as much as possible the performance analysis of Python programs. To achieve this, it uses an analysis-by-specification approach; developers specify the performance requirements of their programs (without any modifications of the source code) and such requirements are checked at runtime. VyPR then provides tools which allow developers to perform detailed analyses of the performance of their code. Such analyses can include determining the common paths taken to reach badly performing parts of code, deciding whether a single path through code led to variations in time taken by future observations, and more.

This paper describes the developments that have taken place in the past year on VyPR’s analysis tools to yield a Python shell-based analysis library, and a web-based application. It concludes by demonstrating the use of the analysis tools on the CMS Experiment’s Conditions Upload service.
Modern Software Stack Building for HEP
Authors: Stewart Graeme A. , Morgan Benjamin , Cervantes Villanueva Javier , Willett Hobbs A.
Published: November 2020

High-Energy Physics has evolved a rich set of software packages that need to work harmoniously to carry out the key software tasks needed by experiments. The problem of consistently building and deploying these packages as a coherent software stack is one that is shared across the HEP community. To that end the HEP Software Foundation Packaging Working Group has worked to identify common solutions that can be used across experiments, with an emphasis on consistent, reproducible builds and easy deployment into CernVM-FS or containers via CI systems. We based our approach on well-identified use cases and requirements from many experiments. In this paper we summarise the work of the group in the last year and how we have explored various approaches based on package managers from industry and the scientific computing community. We give details about a solution based on the Spack package manager which has been used to build the software required by the SuperNEMO and FCC experiments and trialled for a multi-experiment software stack, Key4hep. We shall discuss changes that needed to be made to Spack to satisfy all our requirements. We show how support for a build environment for software developers is provided.
ALFA: A framework for building distributed applications
Authors: Al-Turany Mohammad , Rybalchenko Alexey , Klein Dennis , Kretz Matthias , Kresan Dmytro , Karabowicz Radoslaw , Lebedev Andrey , Manafov Anar , Kollegger Thorsten , Uhlig Florian
Published: November 2020

The ALFA framework is a joint development between ALICE Online-Offline and FairRoot teams. ALFA has a distributed architecture, i.e. a collection of highly maintainable, testable, loosely coupled, independently deployable processes. ALFA allows the developer to focus on building singlefunction modules with well-defined interfaces and operations. The communication between the independent processes is handled by FairMQ transport layer. FairMQ offers multiple implementations of its abstract data transport interface, it integrates some popular data transport technologies like ZeroMQ and nanomsg. Furthermore it also provides shared memory and RDMA transport (based on libfabric) for high throughput, low latency applications. Moreover, FairMQ allows the single process to use multiple and different transports at the same time. FairMQ based processes can be controlled and orchestrated via different systems by implementing the corresponding plugin. However, ALFA delivers also the Dynamic Deployment System (DDS) as an independent set of utilities and interfaces, providing a dynamic distribution of different user processes on any Resource Management System (RMS) or a laptop. ALFA is already being tested and used by different experiments in different stages of data processing as it offers an easy integration of heterogeneous hardware and software.
Code health in EOS: Improving test infrastructure and overall service quality
Authors: Sindrilaru Elvin Alin , Bitzes Georgios , Luchetti Fabio , Patrascoiu Mihai
Published: November 2020

During the last few years, the EOS distributed storage system at CERN has seen a steady increase in use, both in terms of traﬃc volume as well as sheer amount of stored data. This has brought the unwelcome side effect of stretching the EOS software stack to its design constraints, resulting in frequent user-facing issues and occasional downtime of critical services. In this paper, we discuss the challenges of adapting the software to meet the increasing demands, while at the same time preserving functionality without breaking existing features or introducing new bugs. We document our eﬀorts in modernizing and stabilizing the codebase, through the refactoring of legacy code, introduction of widespread unit testing, as well as leveraging Kubernetes to build a comprehensive test orchestration framework capable of stressing every aspect of an EOS installation, with the goal of discovering bottlenecks and instabilities before they reach production.
Optimizing Provisioning of LCG Software Stacks with Kubernetes
Authors: Bachmann Richard , Ganis Gerardo , Konstantinov Dmitri , Razumov Ivan , Heinz Johannes Martin
Published: November 2020

The building, testing and deployment of coherent large software stacks is very challenging, in particular when they consist of the diverse set of packages required by the LHC*** experiments, the CERN Beams department and data analysis services such as SWAN. These software stacks comprise a large number of packages (Monte Carlo generators, machine learning tools, Python modules, HEP**** specific software), all available for several compilers, operating systems and hardware architectures. Along with several releases per year, development builds are provided each night to allow for quick updates and testing of development versions of packages such as ROOT, Geant4, etc. It also provides the possibility to test new compilers and new configurations.

Timely provisioning of these development and release stacks requires a large amount of computing resources. A dedicated infrastructure, based on the Jenkins continuous integration system, has been developed to this purpose. Resources are taken from the CERN OpenStack cloud; Puppet configurations are used to control the environment on virtual machines, which are either used directly as resource nodes or as hosts for Docker containers. Containers are used more and more to optimize the usage of our resources and ensure a consistent build environment while providing quick access to new Linux flavours and specific configurations.

In order to add build resources on demand more easily, we investigated the integration of a CERN provided Kubernetes cluster into the existing infrastructure. In this contribution we present the status of this prototype, focusing on the new challenges faced, such as the integration of these ephemeral build nodes into CERN’s IT infrastructure, job priority control, and debugging of job failures.
Winventory: microservices architecture case study
Authors: Bukowiec Sebastian , Gomulak Pawel Tadeusz
Published: November 2020

In the CERN laboratory, users have access to a large number of different licensed software assets. The landscape of such assets is very heterogeneous including Windows operating systems, office tools and specialized technical and engineering software. In order to improve management of the licensed software and to better understand the needs of the users, it was decided to develop a Winventory application. The Winventory is a tool that gathers and presents statistics of software assets on CERN Windows machines and facilitates interaction with their individual users. The system was built based on microservices architecture pattern, an increasingly popular approach to web application development. The microservices architecture pattern separates the application into multiple independently deployable units that can be individually developed, tested and deployed. This paper presents the microservices architecture and design choices made in order to achieve a modern, maintainable and extensible system for managing licensed software at CERN.
Raythena: a vertically integrated scheduler for ATLAS applications on heterogeneous distributed resources
Authors: Muškinja Miha , Calafiura Paolo , Leggett Charles , Shapoval Illya , Tsulaia Vakho
Published: November 2020

The ATLAS experiment has successfully integrated HighPerformance Computing resources (HPCs) in its production system. Unlike the current generation of HPC systems, and the LHC computing grid, the next generation of supercomputers is expected to be extremely heterogeneous in nature: different systems will have radically different architectures, and most of them will provide partitions optimized for different kinds of workloads. In this work we explore the applicability of concepts and tools realized in Ray (the high-performance distributed execution framework targeting large-scale machine learning applications) to ATLAS event throughput optimization on heterogeneous distributed resources, ranging from traditional grid clusters to Exascale computers. We present a prototype of Raythena, a Ray-based implementation of the ATLAS Event Service (AES), a fine-grained event processing workflow aimed at improving the efficiency of ATLAS workflows on opportunistic resources, specifically HPCs. The AES is implemented as an event processing task farm that distributes packets of events to several worker processes running on multiple nodes. Each worker in the task farm runs an event-processing application (Athena) as a daemon. The whole system is orchestrated by Ray, which assigns work in a distributed, possibly heterogeneous, environment.

For all its flexibility, the AES implementation is currently comprised of multiple separate layers that communicate through ad-hoc command-line and filebased interfaces. The goal of Raythena is to integrate these layers through a feature-rich, efficient application framework. Besides increasing usability and robustness, a vertically integrated scheduler will enable us to explore advanced concepts such as dynamically shaping of workflows to exploit currently available resources, particularly on heterogeneous systems.
A Faster, More Intuitive RooFit
Authors: Hageböck Stephan
Published: November 2020

RooFit and RooStats, the toolkits for statistical modelling in ROOT, are used in most searches and measurements at the Large Hadron Collider as well as atBfactories. Larger datasets to be collected at e.g. the HighLuminosity LHC will enable measurements with higher precision, but will require faster data processing to keep fitting times stable. In this work, a simplification of RooFit’s interfaces and a redesign of its internal dataflow is presented. Interfaces are being extended to look and feel more STL-like to be more accessible both from C++ and Python to improve interoperability and ease of use, while maintaining compatibility with old code. The redesign of the dataflow improves cache locality and data loading, and can be used to process batches of data with vectorised SIMD computations. This reduces the time for computing unbinned likelihoods by a factor four to 16. This will allow to fit larger datasets of the future in the same time or faster than today’s fits.
Fast Inference for Machine Learning in ROOT/TMVA
Authors: Albertsson Kim , An Sitong , Moneta Lorenzo , Wunsch Stefan , Zampieri Luca
Published: November 2020

ROOT provides, through TMVA, machine learning tools for data analysis at HEP experiments and beyond. However, with the rapidly evolving ecosystem for machine learning, the focus of TMVA is shifting. We present the new developments and strategy of TMVA, which will allow the analyst to integrate seamlessly, and effectively, different workflows in the diversified machine-learning landscape. Focus is put on a fast machine learning inference system, which will enable analysts to deploy their machine learning models rapidly on large scale datasets. We present the technical details of a fast inference system for decision tree algorithms, included in the next ROOT release (6.20). We further present development status and proposal for a fast inference interface and code generator for ONNX-based Deep Learning models.
Identification of new long-lived particles using deep neural networks
Authors: Komm Matthias
Published: November 2020

We present the development of a deep neural network for identifying generic displaced jets arising from the decays of exotic long-lived particles in data recorded by the CMS detector at the CERN LHC. Various jet features including detailed information about each clustered particle candidate as well as reconstructed secondary vertices are refined through the use of 1-dimensional convolution layers before being combined with high-level engineered features and passed through a series of fully-connected layers. The proper lifetime of the long-lived particle,cτ0, is treated as a parameter of the neural network model, which allows for hypothesis testing over several orders of magnitude ranging fromcτ0=1 µm to 10 m. Domain adaptation by backward propagation is performed to construct domain-independent features at an intermediate layer of the network to mitiage difference between simulation and data. The training is performed by streaming ROOT trees containingO(100M) jets directly into the TensorFlow queue system, which allows for a flexible selection of input features and asynchronous preprocessing. The application of the tagger is showcased in a search for long-lived gluinos as predicted by split supersymmetric models demonstrating significant gains in sensitivity over a reference analysis.
The FAST-HEP toolset: Using YAML to make tables out of trees
Authors: Krikler Benjamin Edward , Davignon Olivier , Kreczko Lukasz , Linacre Jacob
Published: November 2020

The Faster Analysis Software Taskforce (FAST) is a small, European group of HEP researchers that have been investigating and developing modern software approaches to improve HEP analyses. We present here an overview of the key product of this effort: a set of packages that allows a complete implementation of an analysis using almost exclusively YAML files. Serving as an analysis description language (ADL), this toolset builds on top of the evolving technologies from the Scikit-HEP and IRIS-HEP projects as well as industry-standard libraries such as Pandas and Matplotlib. Data processing starts with event-level data (the trees) and can proceed by adding variables, selecting events, performing complex user-defined operations and binning data, as defined in the YAML description. The resulting outputs (the tables) are stored as Pandas dataframes which can be programmatically manipulated and converted to plots or inputs for fitting frameworks. No longer just a proof-of-principle, these tools are now being used in CMS analyses, the LUX-ZEPLIN experiment, and by students on several other experiments. In this talk we will showcase these tools through examples, highlighting how they address the different experiments’ needs, and compare them to other similar approaches.
Data Analysis using ALICE Run 3 Framework
Authors: Eulisse Giulio , Alkin Anton , Grosse-Oetringhaus Jan Fiete , Hristov Peter , Innocenti Gian Michele , Kabus Maja Jadwiga
Published: November 2020

The ALICE Experiment is currently undergoing a major upgrade program, both in terms of hardware and software, to prepare for the LHC Run 3. A new Software Framework is being developed in collaboration with the FAIR experiments at GSI to cope with the 100-fold increase in the number of recorded events. We present our progress to adapt such a framework for the end user physics data analysis. In particular, the design and technology choices are highlighted. How Apache Arrow is adopted as the platform for the in-memory analysis data layout is discussed. The benefits of this solution are illustrated, these include: efficient and parallel data processing, interoperability with a large number of analysis tools and ecosystems, integration with the modern ROOT declarative analysis framework RDataFrame.
Erratic server behavior detection using machine learning on streams of monitoring data
Authors: Adam Martin , Magnoni Luca , Pilát Martin , Adamová Dagmar
Published: November 2020

With the explosion of the number of distributed applications, a new dynamic server environment emerged grouping servers into clusters, utilization of which depends on the current demand for the application. To provide reliable and smooth services it is crucial to detect and fix possible erratic behavior of individual servers in these clusters. Use of standard techniques for this purpose requires manual work and delivers sub-optimal results. Using only application agnostic monitoring metrics our machine learning based method analyzes the recent performance of the inspected server as well as the state of the rest of the cluster, thus checking not only the behavior of the single server, but the load on the whole distributed application as well. We have implemented our method in a Spark job running in the CERN MONIT infrastructure. In this contribution we present results of testing multiple machine learning algorithms and pre-processing techniques to identify the servers erratic behavior. We also discuss the challenges of deploying our new method into production.
Achieving metric oriented load balancing
Authors: Canilho Paulo , Reguero Ignacio , Saiz Pablo
Published: November 2020

The Load Balance Service at CERN handles more than 500 aliases, distributed over more than 2000 nodes. After being in production for more than fifteen years, it has been going through a major redesign over the last two years. Last year, the server part was reimplemented in the Go Programming Language, taking advantage of its concurrency features to improve the scaling of the system. This year, the client side has been the main focus. This article describes the client side and its configuration, including the tools used at CERN. All the components used by the Load Balance Service are open source, and they can be deployed at other places.
Large Elasticsearch cluster management
Authors: Saiz Pablo , Schwickerath Ulrich
Published: November 2020

The Centralised Elasticsearch Service at CERN runs the infrastructure to provide Elasticsearch clusters for more than 100 different use cases.

This contribution presents how the infrastructure is managed, covering the resource distribution, instance creation, cluster monitoring and user support. The contribution will present the components that have been identified as critical in order to share resources and minimise the amount of clusters and machines needed to run the service. In particular, all the automation for the instance configuration, including index template management, backups and visualisation settings, will be explained in detail.
The NOTED software tool-set improves efficient network utilization for Rucio data transfers via FTS
Authors: Busse-Grawitz Coralie , Martelli Edoardo , Lassnig Mario , Manzi Andrea , Keeble Oliver , Cass Tony
Published: November 2020

We present NOTED, a project to better exploit WAN bandwidth for Rucio and FTS data transfers. We describe the developed software tool-box and a successful demonstration of automated load-balancing in the production network. The first component is the Transfer Broker: It identifies large data transfers for which network reconfiguration is both possible and beneficial, and translates the transfer information into parameters that can be used by network controllers. It will make this information available via a public interface. The second component is a demonstration of a Network Controller. It periodically reads the parameters provided by the Transfer Broker, and so decides which actions to apply to improve the network utilisation for a given transfer.
Pre-Commercial Procurement: R&D as a Service for the European Open Science Cloud
Authors: Devouassoux Marion , Jones Bob , Fernandes João
Published: November 2020

The economical and contractual aspects linked to the production use of commercial cloud services are often overlooked in research environments in Europe, preventing researchers from reaping the full benefits of these services. Since 2016, CERN, in collaboration with leading European research institutes, has launched several projects to address this problem. The preparation and execution phases of these projects have revealed key lessons in procuring commercial cloud resources for research environments in Europe. These lessons can help shape the European Open Science Cloud (EOSC) [1].
High-availability on-site deployment to heterogeneous architectures for Project 8 and ADMX
Authors: LaRoque Benjamin
Published: November 2020

Project 8 is applying a novel spectroscopy technique to make a precision measurement of the tritium beta-decay spectrum, resulting in either a measurement of or further constraint on the effective mass of the electron antineutrino. ADMX is operating an axion haloscope to scan the mass-coupling parameter space in search of dark matter axions. Both collaborations are executing medium-scale experiments, where stable operations last for three to nine months and the same system is used for development and testing between periods of operation. It is also increasingly common to use low-cost computing elements, such as the Raspberry Pi, to integrate computing and control with custom instrumentation and hardware. This leads to situations where it is necessary to support software deployment to heterogeneous architectures on rapid development cycles while maintaining high availability. Here we present the use of docker containers to standardize packaging and execution of control software for both experiments and the use of kubernetes for management and monitoring of container deployment in an active research and development environment. We also discuss the advantages over more traditional approaches employed by experiments at this scale, such as detached user execution or custom control shell scripts.
Designing a New Infrastructure for ATLAS Online Web Services
Authors: Ballestrero Sergio , Brasolin Franco , Sanchez Pineda Arturo , Scannicchio Diana Alessandra , Twomey Matthew Shaun
Published: November 2020

Within the ATLAS detector, the Trigger and Data Acquisition system is responsible for the online processing of data streamed from the detector during collisions at the Large Hadron Collider (LHC) at CERN. The online farm is composed of ~4000 servers processing the data read out from ~100 million detector channels through multiple trigger levels. The capability to monitor the ongoing data taking and all the involved applications is essential to debug and intervene promptly to ensure efficient data taking. The base of the current web service architecture was designed a few years ago, at the beginning of the ATLAS operation (Run 1). It was intended to serve primarily static content from a Network-attached Storage, and privileging strict security, using separate web servers for internal (ATLAS Technical and Control Network - ATCN) and external (CERN General Purpose Network and public internet) access. During these years, it has become necessary to add to the static content an increasing number of dynamic web-based User Interfaces, as they provided new functionalities and replaced legacy desktop UIs. These are typically served by applications on VMs inside ATCN and made accessible externally via chained reverse HTTP proxies. As the trend towards Web UIs continues, the current design has shown its limits, and its increasing complexity became an issue for maintenance and growth. It is, therefore, necessary to review the overall web services architecture for ATLAS, taking into account the current and future needs of the upcoming LHC Run 3. In this paper, we present our investigation and roadmap to re-design the web services system to better operate and monitor the ATLAS detector, while maintaining the security of critical services, such as Detector Control System, and maintaining the separation of remote monitoring and on-site control according to ATLAS policies.
External Resources: Clouds and HPCs for the expansion of the ATLAS production system at the Tokyo regional analysis center
Authors: Kaneda Michiru
Published: November 2020

The Tokyo regional analysis center at the International Center for Elementary Particle Physics, the University of Tokyo, is one of the Tier 2 sites for the ATLAS experiment in the Worldwide LHC Computing Grid. The current system has 10,752 CPU cores and 16 PB disk storage. CERN plans the HighLuminosity LHC starting from 2026, which increases the peak luminosity by 5 times compared to the present value at the LHC. A shortage of computing resources for the high-luminosity LHC is expected with the current system. Therefore new ideas to expand the system are necessary. To expand the ATLAS production system at the Tokyo regional analysis center, R&D using external resources has been launched. One kind of external resources is a commercial cloud resource, such as Google Cloud Platform. Another resource is the HighPerformance Computer (HPC) at the University of Tokyo. In this contribution, the current status of the R&D, the systems for these resources, and comparisons of the cost are reported.
Increasing interoperability for research clouds: CS3APIs for connecting sync&share storage, applications and science environments
Authors: González Labrador Hugo , Mościcki Jakub T. , Lamanna Massimo , Pace Alberto
Published: November 2020

Cloud Services for Synchronization and Sharing (CS3) [14] have become increasing popular in the European Education and Research landscape in the last years. Services such as CERNBox, SWITCHdrive, SURFdrive, PSNCBox, Sciebo, CloudStor and many more have become indispensable in everyday work for scientists, engineers, educators and other users in public research and education sector.

CS3 services are currently too fragmented and lack interoperability. To fix this problem and interconnect storage-, applicationand research services a set of interoperable interfaces, CS3APIs [10], has been developed.

CS3APIs enable creation of easily-accessible and integrated science environments, facilitating cross-institutional research activities and avoiding fragmented silos based on ad-hoc solutions.

In this paper we introduce the CS3APIs and its reference implementation, Reva [16].
Real-time HEP analysis with funcX, a high-performance platform for function as a service
Authors: Woodard Anna Elizabeth , Trisovic Ana , Li Zhuozhao , Babuji Yadu , Chard Ryan , Skluzacek Tyler , Blaiszik Ben , Katz Daniel S. , Foster Ian , Chard Kyle
Published: November 2020

We explore how the function as a service paradigm can be used to address the computing challenges in experimental high-energy physics at CERN. As a case study, we use funcX—a high-performance function as a service platform that enables intuitive, flexible, efficient, and scalable remote function execution on existing infrastructure—to parallelize an analysis operating on columnar data to aggregate histograms of analysis products of interest in real-time. We demonstrate efficient execution of such analyses on heterogeneous resources.
Virgo and Gravitational-Wave Computing in Europe
Authors: Bagnasco Stefano
Published: November 2020

Advanced Virgo is an interferometer for the detection of gravitational waves at the European Gravitational Observatory in Italy. Along with the two Advanced LIGO interferometers in the US, Advanced Virgo is being used to collect data from astrophysical sources such as compact binary coalescences and is currently running the third observational period, collecting gravitational wave event candidates at a rate of more than once per week. Data from the interferometer are processed by running search pipelines for several expected signals, from coalescing compact binaries to continuous waves and burst events. Furthermore, detector characterisation studies are run. Some of the processing needs to be done with low latency, to be able to provide triggers for other observatories and make multi-messenger observations possible. Deep searches are run offline on external computing centres. Thus, data needs also to be reliably and promptly distributed from the EGO site to computer centres in Europe and the US for further analysis and archival storage. Two of the defining characteristics of Virgo computing are the heterogeneity of the activities and the need to interoperate with LIGO. A very wide array of analysis pipelines differing in scientific target, implementation details and running environment assumptions have to be allowed to run ubiquitously and uniformly on dedicated resources and, in perspective, on heterogeneous infrastructures. The current status, possible strategies and outlook of Virgo computing are discussed.
Integrating Interactive Jupyter Notebooks at the BNL SDCC
Authors: Rind Ofer , Strecker-Kellogg William , Allan Daniel , Benjamin Douglas , Karasawa Mizuki , Li Kristy
Published: November 2020

At the SDCC we are deploying a Jupyterhub infrastructure to enable scientists from multiple disciplines to access our diverse compute and storage resources. One major design goal was to avoid rolling out yet another compute backend, but rather to leverage our pre-existing resources via our batch systems (HTCondor and Slurm). Challenges faced include creating a frontend that allows users to choose what HPC resources they have access to as well as selecting containers or environments, delegating authentication to a MFA-enabled proxy, and automating deployment of multiple hub instances. This paper covers the design and features of our Jupyterhub service.
Trends in computing technologies and markets: The HEPiX TechWatch WG
Authors: Sciaba Andrea , Meinhard Helge , Cancio Germán , Gasthuber Martin , Hollowell Christopher , Michelotto Michele , Martelli Edoardo , Muralidharan Servesh , Panzer-Steindel Bernd , Seuster Rolf , Wegner Peter , Yen Eric , Misawa Shigeki
Published: November 2020

Driven by the need to carefully plan and optimise the resources for the next data taking periods of Big Science projects, such as CERN’s Large Hadron Collider and others, sites started a common activity, the HEPiX Technology Watch Working Group, tasked with tracking the evolution of technologies and markets of concern to the data centres. The talk will give an overview of general and semiconductor markets, server markets, CPUs and accelerators, memories, storage and networks; it will highlight important areas of uncertainties and risks.
User documentation and training at Belle II*
Authors: Cunliffe Sam , Komarov Ilya , Kuhr Thomas , Ritter Martin , Tenchini Francesco
Published: November 2020

Belle II is a rapidly growing collaboration with members from one hundred and nineteen institutes spread around the globe. The software development team of the experiment, as well as the software users, are very much decentralised. Together with the active development of the software, such decentralisation makes the adoption of the latest software releases by users an essential, but quite challenging task. To ensure the relevance of the documentation, we adopted the policy of in-code documentation and configured a website that allows us to tie the documentation to given releases. To prevent tutorials from becoming outdated, we covered them by unit-tests. For the user support, we use a question and answer service that not only reduces repetition of the same questions but also turned out to be a place for discussions among the experts. A prototype of a metasearch engine for the different sources of documentation has been developed. For training of the new users, we organise centralised StarterKit workshops attached to the collaboration meetings. The materials of the workshops are later used for self-education and organisation of local training sessions.
Operating the Belle II Collaborative Services and Tools
Authors: Gellrich Andreas , Knittel Daniel , Kuhr Thomas
Published: November 2020

In 2016 the Belle II collaboration reviewed and consolidated their collaborative services and tools. Services and tools were migrated into the existing IT infrastructure at DESY. Missing components were added, workflows adapted, and a membership management system was developed. Since the Belle II collaboration was already alive and working, the collaborative work could not be completely started from scratch. The introduction of new technologies - in particular up-to-date security mechanisms - required users to get used to new systems, workflows, and look-and-feels. In this contribution we review the migration process and thoroughly discussed experiences which we gained while supporting the daily work of the Belle II collaboration.
A Software Institute for Data-Intensive Sciences, Joining Computer Science Academia and Natural Science Research
Authors: Bird Ian , Campana Simone , Mato Vila Pere , Roiser Stefan , Schulz Markus , Stewart Graeme A. , Valassi Andrea
Published: November 2020

With the ever-increasing size of scientific collaborations and complexity of scientific instruments, the software needed to acquire, process and analyze the gathered data is increasing in both complexity and size. Unfortunately the role and career path of scientists and engineers working on software R&D and developing scientific software are neither clearly established nor defined in many fields of natural science. In addition, the exchange of information between scientific software development and computer science departments at universities or computing schools is scattered and fragmented into individual initiatives. To address the above issues we propose a new effort on a European level, which concentrates on strengthening the role of software developers in natural sciences, acts as a hub for the exchange of ideas among different stake-holders in computer science and scientific software and forms a lobbying forum for software engineering in natural sciences on an international level. This contribution discusses in detail the motivation, role and interplay with other initiatives of a “Software Institute for Data-Intensive Sciences”, which is currently being discussed between research institutes, universities and funding agencies in Europe. In addition to the current status, an outlook on future prospects of this initiative will be given.
The release of the 13 TeV ATLAS Open Data: using open education resources effectively
Authors: Serkin Leonid
Published: November 2020

The ATLAS Collaboration is releasing a new set of proton–proton collision data to the public for educational purposes. The data was collected by the ATLAS detector at the Large Hadron Collider at a centre-of-mass energy √s = 13 TeV during the year 2016 and corresponds to an integrated luminosity of 10 fb−1. This dataset is accompanied by simulated events describing several Standard Model processes, as well as hypothetical Beyond Standard Model signal processes. Associated computing tools are provided to make the analysis of the dataset easily accessible. In the following, we summarise the properties of the 13 TeV ATLAS Open Data set and the available analysis tools. Several examples intended as a starting point for further analysis work by users are shown. The general aim of the dataset and tools released is to provide user-friendly and straightforward interactive interfaces to replicate the procedures used by high-energy-physics researchers and enable users to experience the analysis of particle-physics data in educational environments.
Integrating LHCb workflows on HPC resources: status and strategies
Authors: Stagni Federico , Valassi Andrea , Romanovskiy Vladimir
Published: November 2020

High Performance Computing (HPC) supercomputers are expected to play an increasingly important role in HEP computing in the coming years. While HPC resources are not necessarily the optimal fit for HEP workflows, computing time at HPC centers on an opportunistic basis has already been available to the LHC experiments for some time, and it is also possible that part of the pledged computing resources will be offered as CPU time allocations at HPC centers in the future. The integration of the experiment workflows to make the most efficient use of HPC resources is therefore essential. This paper describes the work that has been necessary to integrate LHCb workflows at a specific HPC site, the Marconi-A2 system at CINECA in Italy, where LHCb benefited from a joint PRACE (Partnership for Advanced Computing in Europe) allocation with the other Large Hadron Collider (LHC) experiments. This has required addressing two types of challenges: on the software application workloads, for optimising their performance on a many-core hardware architecture that differs significantly from those traditionally used in WLCG (Worldwide LHC Computing Grid), by reducing memory footprint using a multi-process approach; and in the distributed computing area, for submitting these workloads using more than one logical processor per job, which had never been done yet in LHCb.
Single-Pass Covariance Matrix Calculation on a Hybrid FPGA/CPU Platform
Authors: Arnold Lukas On , Owaida Muhsen
Published: November 2020

Covariance matrices are used for a wide range of applications in particle physics, including Kálmán filter for tracking purposes or Primary Component Analysis for dimensionality reduction. Based on a novel decomposition of the covariance matrix, a design that requires only one pass of data for calculating the covariance matrix is presented. Two computation engines are used depending on parallelizability of the necessary computation steps. The design is implemented onto a hybrid FPGA/CPU system and yields speed-up of up to 5 orders of magnitude compared to previous FPGA implementation.
Large-scale HPC deployment of Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN)
Authors: Hildreth Michael , Paolo Hurtado Anampa Kenyi , Kankel Cody , Hampton Scott , Brenner Paul , Johnson Irena , Simko Tibor
Published: November 2020

The NSF-funded Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN) project aims to develop and deploy artificial intelligence (AI) and likelihood-free inference (LFI) techniques and software using scalable cyberinfrastructure (CI) built on top of existing CI elements. Specifically, the project has extended the CERN-based REANA framework, a cloud-based data analysis platform deployed on top of Kubernetes clusters that was originally designed to enable analysis reusability and reproducibility. REANA is capable of orchestrating extremely complicated multi-step workflows, and uses Kubernetes clusters both for scheduling and distributing container-based workloads across a cluster of available machines, as well as instantiating and monitoring the concrete workloads themselves. This work describes the challenges and development efforts involved in extending REANA and the components that were developed in order to enable large scale deployment on High Performance Computing (HPC) resources. Using the Virtual Clusters for Community Computation (VC3) infrastructure as a starting point, we implemented REANA to work with a number of differing workload managers, including both high performance and high throughput, while simultaneously removing REANA’s dependence on Kubernetes support at the workers level.
Reconstruction of track candidates at the LHC crossing rate using FPGAs
Authors: Tuci Giulia , Punzi Giovanni
Published: November 2020

In 2021 the LHCb experiment will be upgraded, and the DAQ system will be based on full reconstruction of events, at the full LHC crossing rate. This requires an entirely new system, capable of reading out, building and reconstructing events at an average rate of 30 MHz. In facing this challenge, the system could take advantage of a fast pre-processing of data on dedicated FPGAs. The results of an R&D on these technologies, developed in the context of the LHCb Upgrade I, are presented in this document. In particular, the details and potential benefits of an approach based on producing in real-time sorted collections of hits in the VELO detector (pre-tracks) are discussed. These pre-processed data can then be used as seeds by the High Level Trigger (HLT) farm to find tracks for the Level 1 trigger with much lower computational effort than possible by starting from the raw detector data, thus freeing an important fraction of the power of the CPU farm for higher level processing tasks.
The Acts project: track reconstruction software for HL-LHC and beyond
Authors: Gessinger Paul , Grasland Hadrien , Gray Heather , Kiehn Moritz , Klimpel Fabian , Langenberg Robert , Salzburger Andreas , Schlag Bastian , Zhang Jin , Ai Xiaocong
Published: November 2020

The reconstruction of trajectories of the charged particles in the tracking detectors of high energy physics (HEP) experiments is one of the most difficult and complex tasks of event reconstruction at particle colliders. As pattern recognition algorithms exhibit combinatorial scaling to high track multiplicities, they become the largest contributor to the CPU consumption within event reconstruction, particularly at current and future hadron colliders such as the LHC, HL-LHC and FCC-hh. Current algorithms provide an extremely high standard of physics and computing performance and have been tested on billions of simulated and recorded data events. However, most algorithms date back to more than 20 years ago and maintaining them has become increasingly challenging. In addition, they are challenging to adapt to modern programming paradigms and parallel architectures.

Acts is based on the well-tested and highly functioning components of LHC track reconstruction algorithms, implemented with modern software concepts and inherently designed for parallel architectures. Multithreading becomes increasingly important to balance the memory usage per CPU core. However, a fully multithreaded event processing framework blurs the clear border between events, which has in the past often been used as a clearly defined validity boundary for event conditions. Acts is equipped with a full contextual conditions concept that allows to run concurrent track reconstruction even in case of multiple detector alignments, conditions or varying magnetic field being processed at the same time. It provides an experiment and, in particular, framework-independent software toolkit and light-weight, highly optimised event data model for track reconstruction. Particular care is given to thread safety and data locality. It is designed as a toolbox that allows to implement and extend widely known pattern recognition algorithms, and in addition suitable for algorithm templating and R&D. Acts has been used as the fast simulation engine for the Tracking Machine Learning (TrackML) Challenge, and will provide reference implementation of several submitted solution programs of the two phases of the challenge.
HPC Systems in the Next Decade – What to Expect, When, Where
Authors: Pleiter Dirk
Published: November 2020

HPC systems have seen impressive growth in terms of performance over a period of many years. Soon the next milestone is expected to be reached with the deployment of exascale systems in 2021. In this paper, we provide an overview of the exascale challenges from a computer architecture’s perspective and explore technological and other constraints. The analysis of upcoming architectural options and emerging technologies allow for setting expectations for application developers, which will have to cope with heterogeneous architectures, increasingly diverse compute technologies as well as deeper memory and storage hierarchies. Finally, needs resulting from changing science and engineering workflows will be discussed, which need to be addressed by making HPC systems available as part of more open e-infrastructures that provide also other compute and storage services.
Computing and Software at Belle II
Authors: Dossett David
Published: November 2020

In 2019 the Belle II detector began collecting data from e+e− collisions at the SuperKEKB electron-positron collider. Belle II aims to collect a data sample of 50 ab−1, 50 times larger than the previous generation of B-Factories. In order to process this data and release it to physics analysts there are many interconnected pieces of software and computing resources that must be utilised effectively. In this paper, the current Belle II computing systems, software, and data production activities are summarised.
Orano achève la vitrification des solutions de produits de fission des combustibles UNGG
Authors: Renaut Christophe , Leveel Éric , Chanu Thierry , Darvey Fabrice , Renouf Sylvain , Charles Mélanie
Published: November 2020

L’usine UP2-400, première usine du site Orano la Hague, a réalisé le traitement des combustibles des réacteurs Uranium naturel graphite gaz (UNGG) de type UMo (alliage molybdène) et MoSnAI (alliage molybdène, étain et aluminium), de 1966 à 1985, ainsi que le traitement des combustibles usés des réacteurs de la filière eau légère (UOx). Du fait de ces caractéristiques chimiques, Orano a mis au point la vitrification des produits de fission en creuset froid. Le point sur cette haute technologie.
Repetition frequency locking of a terahertz quantum cascade laser emitting at 4.2 THz
Authors: Guan Wen , Li Ziping , Zhou Kang , Wan Wenjian , Liao Xiaoyu , Zhao Yiran , Yang Sijia , Cao J. C. , Li Hua
Published: November 2020

The electrically-pumped terahertz quantum cascade laser (QCL) is characterized by high power emission, compact, broad frequency coverage, and so on, which shows abilities for frequency comb operations. Although free-running QCLs can work as frequency combs by designing the laser structure with small group velocity dispersions and/or inserting mirrors to compensate laser intrinsic dispersions, the ideal comb operation can only be obtained by firmly locking the repetition frequency and carrier frequency of a laser. In this work, we have reported a repetition frequency locking of a terahertz QCL emitting around 4.2 THz. When the 6-mm-long laser is operated in continuous wave mode without any locking techniques, the repetition frequency is measured to be ~6.15 GHz with a linewidth of hundred kilohertz. Once a phase lock loop (PLL) is applied to dynamically control the drive current of the QCL, we have demonstrated a successful repetition frequency locking of the laser with a signal to noise ratio of 80 dB. This technique can be employed for the frequency comb and dual-comb operations of terahertz QCLs for high-resolution applications.