bluuebunny/crossref_metadata_2025 huggingface dataset:
The U.S. CMS HL-LHC R&D Strategic Plan
Authors: Gutsche Oliver , Bose Tulika , Votava Margaret , Mason David , Melo Andrew , Liu Mia , Hufnagel Dirk , Gray Lindsey , Hildreth Mike , Holzman Burt , Lannon Kevin , Sehrish Saba , Sperka David , Letts James , Bauerdick Lothar , Bloom Kenneth
Published: May 2024

The HL-LHC run is anticipated to start at the end of this decade and will pose a significant challenge for the scale of the HEP software and computing infrastructure. The mission of the U.S. CMS Software & Computing Operations Program is to develop and operate the software and computing resources necessary to process CMS data expeditiously and to enable U.S. physicists to fully participate in the physics of CMS. We have developed a strategic plan to prioritize R&D efforts to reach this goal for the HL-LHC. This plan includes four grand challenges: modernizing physics software and improving algorithms, building infrastructure for exabyte-scale datasets, transforming the scientific data analysis process and transitioning from R&D to operations. We are involved in a variety of R&D projects that fall within these grand challenges. In this talk, we will introduce our four grand challenges and outline the R&D program of the U.S. CMS Software & Computing Operations Program.
Building a user-oriented notification system at CERN
Authors: Antunes Carina , Semedo Jose , Wagner Andreas , Ormancey Emmanuel , Carpente Caetan , Jakovljevic Igor
Published: May 2024

CERN, as many large organizations, heavily relies on a variety of communication means for different use-cases and teams. The large amount of notifications received every day leads to adverse consequences. For the users receiving them, it is challenging to control and keep track of where, how and when some information was received. At the same time for those sending and maintaining the tools that deliver notifications, it is difficult to choose which targets to adopt (email, messaging system and platforms, electronic devices, etc.).

The CERN Notifications system aims at consolidating communication by providing a central place where notifications are created, maintained and distributed. CERN Notifications allows not only optimising the flow for the multiple people and teams which are responsible for sending, but also empowers the target users by respecting their preferences: how, where and when they receive their notifications.

This paper describes the design and architecture of the CERN Notifications system and its components, how it was designed with a flexible and highly modular architecture which allows adding further device targets with little effort. Furthermore, it presents implementation details and the decisions behind those. And last but not least it describes the features that empower users to choose how to consume information sent to them.
Enhancing data consistency in ATLAS and CERN HR databases through automated synchronization
Authors: de Aragão Aleksandravicius Gabriel , Loureiro Cruz Ana Clara , Niklaus Moreira da Rocha Rodrigues Carolina , Lemos Lúcidi Pinhão Gabriela , Goes Afonso Pedro Henrique , Coura Torres Rodrigo , Seixas José Manoel
Published: May 2024

As the largest particle physics laboratory in the world, it is no surprise that CERN has a vast network of thousands of collaborators spread globally. Among CERN’s experiments, ATLAS stands out with over 6,000 active members and 300 associate institutes. This extensive community must go through the standard registering and maintenance procedures within CERN’s human resources database, called Foundation. Simultaneously, the Glance system, among other specific functions, also aims to fulfill the same objectives within the context of different CERN experiments, such as ATLAS, LHCb and ALICE. It achieves that by having its own database. Since members need to exist in the two databases, a lot of data was being duplicated. Manual updates by the ATLAS secretariat became necessary to maintain consistency over members and institutes data, such as names, employment information and authorship status. Today, the Glance system undergoes a transformative process to redefine its relationship with Foundation. The goal is to eliminate duplication of data by establishing a single source of truth. At the same time, the automation of a series of internal processes will be made to ensure synchronization between the two databases at all times, thus removing the need for manual intervention from the ATLAS secretariat. This requires some restructuring to Glance’s database, updates to the code and overall implementation of new tools that facilitate seamless communication with Foundation.
Iterative and incremental development of the ATLAS Publication Tracking System
Authors: Loureiro Cruz Ana Clara , Niklaus Moreira Da Rocha Rodrigues Carolina , de Aragão Aleksandravicius Gabriel , Lemos Lúcidi Pinhão Gabriela , Goes Afonso Pedro Henrique , Coura Torres Rodrigo , Seixas José Manoel
Published: May 2024

The ATLAS experiment is a particle physics experiment situated at the Large Hadron Collider (LHC) at CERN. It involves almost 6000 members from approximately 300 institutes spread all over the globe and more than 100 papers published every year. This dynamic environment brings some challenges such as how to ensure publication deadlines, communication between the groups involved, and the continuity of workflows. The solution found for those challenges was automation, which was achieved through the Glance project, more specifically through the Glance Analysis systems, developed to support the analysis and publications life cycle in 2011. Now, after twelve years, in order to satisfy the experiments’ most recent needs, the systems need code refactoring and database remodelling. The goal is to have only one system to accommodate all the analysis and publications workflows, the so-called ATLAS Publication Tracking system, an evolution of the current Analysis systems. This project includes a database remodelling that reflects the hierarchical relation between analyses and publications; a code base that supports non-standard workflows; the expansion of the current API so all the authorized ATLAS members can access ATLAS publication data programmatically; a service-oriented architecture for integration with external software, such as GitLab; the creation of an automatic test environment, which assures the quality of the systems on each update. The ATLAS Publication Tracking system is a long-term project being developed with an iterative and incremental approach, which ensures that the most valuable tools are implemented with priority while allowing a smooth transition between the old systems and the new one.
Large scale dynamic web deployment for CERN, experience and outlook
Authors: Vineet Reddy Rajula , Valdemar Yde Joachim , Borges Aurindo Barros Francisco , Antunes Carina , Wagner Andreas
Published: May 2024

CERN hosts more than 1200 websites essential for the mission of the Organization, both for internal and external collaboration and communication, as well as public outreach. The complexity and scale of CERN’s online presence is very diverse with some websites, like home.cern, accommodating more than one million unique visitors in a day. However, regardless of their diversity, all websites are created using the Drupal content management system (CMS), and are self-hosted directly in the CERN Data Center on a dedicated infrastructure that runs on Kubernetes. Workflows like provisioning, deleting, cloning, upgrading, and similar are fully automated and managed by a customized Kubernetes controller. By leveraging the custom controller, the infrastructure has proven highly reliant with minimal, manual intervention necessary. In order to further automate deployments and improve governance, a customized version of Drupal called the CERN Drupal Distribution is implemented. Supported by end-to-end integration tests and automated browser simulation, this setup enables the propagation of security and feature updates seamlessly to all websites without any downtime.

This paper outlines the architecture which allows building, testing, and distributing updates to a large number of websites without any downtime. Furthermore, it presents experiences and learnings from managing such a service at CERN with limited resources.
End-to-End Analysis Automation over Distributed Resources with Luigi Analysis Workflows
Authors: Rieger Marcel
Published: May 2024

In particle physics, workflow management systems are primarily used as tailored solutions in dedicated areas such as Monte Carlo production. However, physicists performing data analyses are usually required to steer their individual, complex workflows manually, frequently involving job submission in several stages and interaction with distributed storage systems by hand. This process is not only time-consuming and error-prone, but also leads to undocumented relations between particular workloads, rendering the steering of an analysis a serious challenge. This article presents the Luigi Analysis Workflow (Law) Python package which is based on the open-source pipelining tool Luigi, originally developed by Spotify. It establishes a generic design pattern for analyses of arbitrary scale and complexity, and shifts the focus from executing to defining the analysis logic. Law provides the building blocks to seamlessly integrate with interchangeable remote resources without, however, limiting itself to a specific choice of infrastructure. In particular, it introduces the concept of complete separation between analysis algorithms on the one hand, and run locations, storage locations, and software environments on the other hand. To cope with the sophisticated demands of end-to-end HEP analyses, Law supports job execution on WLCG infrastructure (ARC, gLite, CMS-CRAB) as well as on local computing clusters (HTCondor, Slurm, LSF), remote file access via various protocols using the Grid File Access Library (GFAL2), and an environment sandboxing mechanism with support for sub-shells and virtual environments, as well as Docker and Singularity containers. Moreover, the novel approach ultimately aims for analysis preservation out-of-the-box. Law is developed opensource and independent of any experiment or the language of executed code, and its user-base increased steadily over the past years.
Meld Exploring the Feasibility of a Framework-less Framework
Authors: Knoepfel Kyle
Published: May 2024

HEP data-processing frameworks are essential ingredients in getting from raw data to physics results. But they are often tricky to use well, and they present a significant learning barrier for the beginning HEP physicist. In addition, existing frameworks typically support rigid, collider-based data models, which do not map well to neutrino-physics experiments like DUNE. Neutrino physicists thus expend significant effort working around framework limitations instead of using a framework that directly supports their needs.

Presented here is Meld, a Fermilab R&D project, which intends to address these limitations. By leveraging modern C++ capabilities, state-of-the-art concurrency libraries, and a flexible data model, it is possible for beginning (and seasoned) HEP physicists to execute framework programs easily and efficiently, with minimal coupling to framework-specific constructs. Meld aims to directly support the frameworks needs of neutrino experiments like DUNE as well as the more common collider-based experiments.
Publication of the Belle II Software
Authors: Kuhr Thomas , Bianchi Fabizio , De Pietro Giacomo , Jaffe David , Meier Frank , Sevior Martin , Urquijo Philipp
Published: May 2024

The Belle II software was developed by a few hundred individual contributors over several years. Following the rising desire of making it publicly available, the collaboration established open source software policies and procedures. The political and technical challenges and their solutions at Belle II are discussed in this article. With the publication of the Belle II software, basf2, on GitHub and Zenodo in 2021 an important milestone towards open science was reached.
Train to Sustain
Authors: Malik Sudhir , Lieret Kilian , Elmer Peter , Hernandez Villanueva Michel , Roiser Stefan
Published: May 2024

The HSF/IRIS-HEP Software Training group provides software training skills to new researchers in High Energy Physics (HEP) and related communities. These skills are essential to produce high-quality and sustainable software needed to do the research. Given the thousands of users in the community, sustainability, though challenging, is the centerpiece of its approach. The training modules are open source and collaborative. Different tools and platforms, like GitHub, enable technical continuity, collaboration and nurture the sense to develop software that is reproducible and reusable. This contribution describes these efforts and its broader impacts.
Version control and DevOps for accelerator and experiments: Experience and outlook
Authors: Trobo Ismael P. , Evangelou Konstantinos , Lossent Alexandre , Wagner Andreas
Published: May 2024

GitLab has been in operation at CERN since 2012. It is a selfservice code hosting application based on Git that provides collaboration and code review features, becoming one of the key infrastructures at CERN. It is extensively utilised at CERN, with more than 17 000 active users, hosting more than 120 000 projects and triggering more than 5 000 jobs per hour. During the initial stages, a custom-made solution was deployed. However, with the exponential increase of projects, workflows, and continuous integrations, the GitLab infrastructure became hard and complex to scale and maintain. The recent migration, which involved adopting a new supported Cloud Hybrid infrastructure, has enabled CERN to align its GitLab infrastructure with industry standards and best practices. This has resulted in a significantly more robust and high-performing infrastructure, leading to notable benefits throughout the entire deployment process. This paper will address how this deployment process, on the road to success, has presented a series of challenges and pitfalls that have been faced during this complex migration process.
A multidimensional, event-by-event, statistical weighting procedure for signal to background separation
Authors: Baldwin Zachary
Published: May 2024

Numerous analyses performed in nuclear and particle physics are in search of signals that are contaminated by irreducible background that cannot be suppressed using event-selection criteria. These background events can lead to unphysical or biased results when extracting physical observables and need to be taken into account. Exploring a data set across multiple dimensions allows us to characterize the phase space of a desired reaction through a set of coordinates. For a subset of these coordinates, known as reference coordinates, signal and background follow different distributions with known functional forms with potential unknown parameters. The Quality Factor approach uses the space defined by the remaining non-reference phase space coordinates to determine the k-nearest neighbors of an event. The distribution of these neighbors in the reference coordinates undergoes a fit with the sum of the signal and background model functions, employing techniques like the unbinned maximum likelihood method, to extract the signal fraction, or Q-factor. This quality factor, which is defined for each event, is equal to the probability that it originates from the signal of interest. In this document, we will give a brief overview of this procedure and illustrate examples using Monte Carlo simulations and data from the GlueX experiment at Jefferson Lab.
HDF5 Experience in DUNE
Authors: Chowdhury Barnali
Published: May 2024

The Deep Underground Neutrino Experiment (DUNE) has so far represented data using a combination of custom data formats and those based on ROOT I/O. Recently, DUNE has begun using the Hierarchical Data Format (HDF5) for some of its data storage applications. HDF5 provides high-performance, low-overhead I/O in DUNE’s data acquisition (DAQ) environment. DUNE will use HDF5 to record raw data from the ProtoDUNE-II Horizontal Drift (HD), ProtoDUNE-II Vertical Drift (VD) and a number of test stands. Dedicated I/O modules have been developed to read the HDF5 data from these detectors into the offline framework for reconstruction directly and via XRootD. HDF5 is also very commonly used on High Performance Computers (HPCs) and is well-suited for use in AI/ML applications. The DUNE software stack contains modules that export data from an offline job in HDF5 format, so that they can be processed by external AI/ML software. The collaboration is also developing strategies to incorporate HDF5 in the detector simulation chains.
Event Generator Tuning Incorporating Systematic Uncertainty
Authors: Schroff Jaffae , Ju Xiangyang
Published: May 2024

Event generators play an important role in all physics programs at the Large Hadron Collider and beyond. Dedicated efforts are required to tune the parameters of event generators to accurately describe data. There are many tuning methods ranging from expert-based manual tuning to surrogate function-based semi-automatic tuning, to machine learning-based re-weighting. Although they scale differently with the number of generator parameters and the number of experimental observables, these methods are effective in finding optimal generator parameters. However, none of these tuning methods includes the Monte Carlo (MC) systematic uncertainties. That makes the tuning results sensitive to systematic variations. In this work, we introduce a novel method to incorporate the MC systematic uncertainties into the tuning procedure and to quantitatively evaluate uncertainties associated with the tuned parameters. Tested with a dummy example, the method results in a χ2 distribution that is centered around one, the optimal generator parameters are closer to the true parameters, and the estimated uncertainties are more accurate.
A grid site reimagined: Building a fully cloud-native ATLAS Tier 2 on Kubernetes
Authors: Taylor Ryan Paul , Albert Jeffrey Ryan , Barreiro Megino Fernando Harald
Published: May 2024

The University of Victoria (UVic) operates an Infrastructure-asa-Service scientific cloud for Canadian researchers, and a Tier 2 site for the ATLAS experiment at CERN as part of the Worldwide LHC Computing Grid (WLCG). At first, these were two distinctly separate systems, but over time we have taken steps to migrate the Tier 2 grid services to the cloud. This process has been significantly facilitated by basing our approach on Kubernetes, a versatile, robust, and very widely adopted automation platform for orchestrating containerized applications. Previous work exploited the batch capabilities of Kubernetes to run grid computing jobs and replace the conventional grid computing elements by interfacing with the Harvester workload management system of the ATLAS experiment. However, the required functionality of a Tier 2 site encompasses more than just batch computing. Likewise, the capabilities of Kubernetes extend far beyond running batch jobs, and include for example scheduling recurring tasks and hosting long-running externally-accessible services in a resilient way. We are now undertaking the more complex and challenging endeavour of adapting and migrating all remaining services of the Tier 2 site — such as APEL accounting and Squid caching proxies, and in particular the grid storage element — to cloud-native deployments on Kubernetes. We aim to enable fully comprehensive deployment of a complete ATLAS Tier 2 site on a Kubernetes cluster via Helm charts, which will benefit the community by providing a streamlined and replicable way to install and configure an ATLAS site. We also describe our experience running a high-performance self-managed Kubernetes ATLAS Tier 2 cluster at the scale of 8 000 CPU cores for the last two years, and compare with the conventional setup of grid services.
Building a Flexible and Resource-Light Monitoring Platform for a WLCG-Tier2
Authors: Currie Robert , Yuan Wenlong
Published: May 2024

Software development projects at Edinburgh identified a desire to build and manage our own monitoring platform. This better allows us to support the developing and varied physics and computing interests of our Experimental Particle Physics group. This production platform enables oversight of international experimental data management, local software development projects and active monitoring of lab facilities within our research group.

Larger sites such as CERN have access to many resources to support generalpurpose centralised monitoring solutions such as MONIT. At a WLCG Tier2 we only have access to a fraction of these resources and manpower. Recycling nodes from grid storage and borrowed capacity from our Tier2 Hypervisors has enabled us to build a reliable monitoring infrastructure. This also contributes back to our Tier2 management improving our operational and security monitoring.

Shared experiences from larger sites gave us a head-start in building our own service monitoring (FluentD) and multi-protocol (AMQP/STOMP/UDP datagram) messaging frameworks atop both our Elasticsearch and OpenSearch clusters. This has been built with minimal hardware and software complexity, maximising maintainability, and reducing manpower costs. A secondary design goal has also been the ability to migrate and upgrade individual components with minimal service interruption. To achieve this, we made heavy use of different layers of containerisation (Podman/Docker), virtualization and NGINX web proxies.

This presentation details our experiences in developing this platform from scratch with a focus on minimal resource use. This includes lessons learnt in deploying and comparing both an Elasticsearch and OpenSearch clusters, as well as designing various levels of automation and resiliency for our monitoring framework. This has culminated in us effectively indexing, parsing and storing >200GB of logging and monitoring data per day.
Demand-driven provisioning of Kubernetes-like resources in OSG
Authors: Sfiligoi Igor , Würthwein Frank , Dost Jeff , Lin Brian , Schultz David
Published: May 2024

The OSG-operated Open Science Pool is an HTCondor-based virtual cluster that aggregates resources from compute clusters provided by several organizations. Most of the resources are not owned by OSG, so demand-based dynamic provisioning is important for maximizing usage without incurring excessive waste. OSG has long relied on GlideinWMS for most of its resource provisioning needs but is limited to resources that provide a Grid-compliant Compute Entrypoint. To work around this limitation, the OSG Software Team has developed a glidein container that resource providers could use to directly contribute to the OSPool. The problem with that approach is that it is not demand-driven, relegating it to backfill scenarios only. To address this limitation, a demand-driven direct provisioner of Kubernetes resources has been developed and successfully used on the NRP. The setup still relies on the OSG-maintained backfill container image but automates the provisioning matchmaking and successive requests. That provisioner has also been extended to support Lancium, a green computing cloud provider with a Kubernetes-like proprietary interface. The provisioner logic has been intentionally kept very simple, making this extension a low-cost project. Both NRP and Lancium resources have been provisioned exclusively using this mechanism for many months.
Evolving the LHCOPN and LHCONE networks to support HL-LHC computing requirements
Authors: Martelli Edoardo
Published: May 2024

The LHCOPN network, which links CERN to all the WLCG Tier 1s, and the LHCONE network, which connects WLCG Tier1s and Tier2s, have successfully provided the necessary bandwidth for the distribution of the data generated by the LHC experiments during first two runs of the LHC accelerator. We give here an overview of the most significant achievements and the current state of the two networks. It also explains how the two networks are evolving to support Run-3 and how they are preparing to meet the high demands foreseen for Run-4, notably by adopting new transmission technologies to increase the available bandwidth, introducing new software tools to improve the efficient utilization of all the links, as well as new monitoring capabilities to increase the understanding of the network traffic.
HEPScore: A new CPU benchmark for the WLCG
Authors: Giordano Domenico , Barbet Jean-Michel , Boccali Tommaso , Borge Gonzalo Menéndez , Hollowell Christopher , Innocente Vincenzo , Lampl Walter , Michelotto Michele , Meinhard Helge , Ondris Ladislav , Sciabà Andrea , Schnepf Matthias J. , Sobie Randall J. , Southwick David , Sullivan Tristan S. , Valassi Andrea , Wenzel Sandro , Willis John L. , Yan Xiaofei
Published: May 2024

HEPScore is a new CPU benchmark created to replace the HEPSPEC06 benchmark that is currently used by the WLCG for procurement, computing resource pledges, usage accounting and performance studies. The development of the new benchmark, based on HEP applications or workloads, has involved many contributions from software developers, data analysts, experts of the experiments, representatives of several WLCG computing centres and WLCG site managers. In this contribution, we review the selection of workloads and the validation of the new HEPScore benchmark.
Implementation of New Security Features in CMSWEB Kubernetes Cluster at CERN
Authors: Ali Aamir , Imran Muhammad , Kuznetsov Valentin , Trigazis Spyridon , Pervaiz Aroosha , Pfeiffer Andreas , Mascheroni Marco
Published: May 2024

The CMSWEB cluster is pivotal to the activities of the Compact Muon Solenoid (CMS) experiment, as it hosts critical services required for the operational needs of the CMS experiment. The security of these services and the corresponding data is crucial to CMS. Any malicious attack can compromise the availability of our services. Therefore, it is important to construct a robust security infrastructure. In this work, we discuss new security features introduced to the CMSWEB Kubernetes (“k8s”) cluster. The new features include the implementation of network policies, deployment of Open Policy Agent (OPA), enforcement of OPA policies, and the integration of Vault. The network policies act as an inside-the-cluster firewall to limit the network communication between the pods to the minimum necessary, and its dynamic nature allows us to work with microservices. The OPA validates the objects against some custom-defined policies during create, update, and delete operations to further enhance security. Without recompiling or changing the configuration of the Kubernetes API server, it can apply customized policies on Kubernetes objects and their audit functionality enabling us to detect pre-existing conflicts and issues. Although Kubernetes incorporates the concepts of secrets, they are only base64 encoded and are not dynamically configured. This is where Vault comes into play: Vault dynamically secures, stores, and tightly controls access to sensitive data. This way, the secret information is encrypted, secured, and centralized, making it more scalable and easier to manage. Thus, the implementation of these three security features corroborate the enhanced security and reliability of the CMSWEB Kubernetes infrastructure.
Progress on cloud native solution of Machine Learning as a Service for HEP
Authors: Giommi Luca , Spiga Daniele , Kuznetsov Valentin , Bonacorsi Daniele
Published: May 2024

Nowadays Machine Learning (ML) techniques are successfully used in many areas of High-Energy Physics (HEP) and will play a significant role also in the upcoming High-Luminosity LHC upgrade foreseen at CERN, when a huge amount of data will be produced by LHC and collected by the experiments, facing challenges at the exascale. To favor the usage of ML in HEP analyses, it would be useful to have a service allowing to perform the entire ML pipeline (in terms of reading the data, processing data, training a ML model, and serving predictions) directly using ROOT files of arbitrary size from local or remote distributed data sources. The Machine Learning as a Service for HEP (MLaaS4HEP) solution we have already proposed aims to provide such kind of service and to be HEP experiment agnostic. To provide users with a real service and to integrate it into the INFN Cloud, we started working on MLaaS4HEP cloudification. This would allow to use cloud resources and to work in a distributed environment. In this work, we provide updates on this topic and discuss a working prototype of the service running on INFN Cloud. It includes an OAuth2 proxy server as authentication/authorization layer, a MLaaS4HEP server, an XRootD proxy server for enabling access to remote ROOT data, and the TensorFlow as a Service (TFaaS) service in charge of the inference phase. With this architecture a HEP user can submit ML pipelines, after being authenticated and authorized, using local or remote ROOT files simply using HTTP calls.
The CloudVeneto initiative: 10 years of operations to support interdisciplinary open science
Authors: Andreetto Paolo , Costa Fulvia , Crescente Alberto , Fantinel Sergio , Fanzago Federica , Garfagnini Alberto , Lazzaro Loris , Lenzo Rita , Mazzon Paolo Emilio , Menguzzato Matteo , Sella Gianpietro , Sgaravatto Massimo , Traldi Sergio , Verlato Marco , Zangrando Lisa
Published: May 2024

CloudVeneto is a private cloud targeted to scientific communities based on OpenStack software. It was designed in 2013 and put in operation one year later, to support INFN projects, mainly HEP ones. Its resources are physically distributed among two sites: the Physics Department of University of Padova-INFN Padova Unit and the INFN Legnaro National Laboratories. During these 10 years CloudVeneto evolved to integrate also resources funded by ten Departments of the University of Padova, and to support several scientific disciplines of different domains. The use cases the communities have to face up often show a common pattern. This was an opportunity for us to develop and improve the services on our infrastructure to provide common solutions to different use cases. It happened for example with the Container as a Service (CaaS) that makes the management of Kubernetes clusters easier from a user point of view. Moreover, CloudVeneto joined the INFN national cloud infrastructure (INFN Cloud), making available some resources to this federated infrastructure. CloudVeneto is also involved in an R&D project to realize a distributed analysis facility for the CMS experiment based on the HTCondor batch system. In this paper we describe some use-cases of different projects pointing out the common patterns and the new implementations and configurations done in the infrastructure.
CRIU - Checkpoint Restore in Userspace for computational simulations and scientific applications
Authors: Andrijauskas Fabio , Sfiligoi Igor , Davila Diego , Arora Aashay , Guiang Jonathan , Bockelman Brian , Thain Greg , Würthwein Frank
Published: May 2024

Creating new materials, discovering new drugs, and simulating systems are essential processes for research and innovation and require substantial computational power. While many applications can be split into many smaller independent tasks, some cannot and may take hours or weeks to run to completion. To better manage those longer-running jobs, it would be desirable to stop them at any arbitrary point in time and later continue their computation on another compute resource; this is usually referred to as checkpointing. While some applications can manage checkpointing programmatically, it would be preferable if the batch scheduling system could do that independently. This paper evaluates the feasibility of using CRIU (Checkpoint Restore in Userspace), an open-source tool for the GNU/Linux environments, emphasizing the OSG’s OSPool HTCondor setup. CRIU allows checkpointing the process state into a disk image and can deal with both open files and established network connections seamlessly. Furthermore, it can checkpoint traditional Linux processes and containerized workloads. The functionality seems adequate for many scenarios supported in the OSPool. However, some limitations prevent it from being usable in all circumstances.
Using ZOOM-events for Scientific Conferences: The ICHEP 2022 Experience
Authors: Rinaldi Lorenzo , Giacomelli Roberto , Giacomelli Paolo , Bellagamba Lorenzo , Zani Stefano
Published: May 2024

The organization of seminars and conferences was strongly influenced by the covid-19 pandemic. In the early period of the pandemic, many events were canceled or held completely online, using video conferencing tools such as ZOOM or MS Teams or Google Meet. Later, thanks to large-scale vaccination and immunization, it was possible to organize again large events in person. Nevertheless it was still necessary to provide online modalities for participants who could not participate in person, having in fact hybrid events with both remote and in person participation. However, in 2022, the global pandemic situation was patchy, with many countries still affected by travel restrictions, making it necessary to provide online modalities for participants who could not participate in-person, having in fact hybrid events with both remote and in-person participation. In this contribution we describe the experience with the ZOOM-Events platform, used for the ICHEP 2022 International Conference on High Energy Physics, held in Bologna in July 2022, with about 1100 participants in-person and 300 connected remotely. We describe in detail how the ZOOM-Events platform was configured for the management of the numerous parallel sessions and the granting of access to participants and how we dealt with the problems that emerged in the organizational phases.
ATLAS Virtual Visits: Bringing the World to our Detector
Authors: Le Boulicaut Elise
Published: May 2024

The Virtual Visit service run by the ATLAS Collaboration at CERN has been active since 2010. The ATLAS Collaboration has used this popular and effective method to bring the excitement of scientific exploration and discovery into classrooms and other public places around the world. The program, which uses a combination of video conferencing, webcasts, and video recording to communicate with remote audiences, has already reached tens of thousands of viewers, in a large number of languages, from tens of countries across all continents. We present a summary of the ATLAS Virtual Visit service that is currently in use and present a new system that is being installed in the ATLAS Visitor Centre. In addition, we show the reach of the program over the last few years.
INFN Cloud Users and Projects Support, Training and Communication
Authors: Sinisi Francesco , Alkhansa Ahmad , Costantini Alessandro , Duma Cristina , Fanzago Federica , Lattanzio Daniele , Pascolini Alessandro , Stalio Stefano , Giorgio Emidio , Pellegrino Carmelo , Salomoni Davide
Published: May 2024

Having a long tradition in state-of-the-art distributed IT technologies, in the last couple of years INFN made available to its users “INFN Cloud”: a cloud infrastructure and related services portfolio dedicated to the scientific communities supported by INFN.

Given the distributed nature of the infrastructure as well as the considerable number of technical solutions provided to the INFN users, it is important to have a reliable user support service aimed to properly interact both with INFN Cloud users and administrators.

As an added value, proper training activities have been defined and differentiated to different types of users and the training courses are integrated with a rich set of user guides and technical documentation.

In this article, an overview of the INFN Cloud, its evolution to DataCloud project, and the support and training activities will be provided and presented.
MicroBooNE Public Data Sets: A Collaborative Tool for LArTPC Software Development
Authors: Cerati Giuseppe
Published: May 2024

Among liquid argon time projection chamber (LArTPC) experiments MicroBooNE is the one that continually took physics data for the longest time (2015-2021), and represents the state of the art for reconstruction and analysis with this detector. Recently published analyses include oscillation physics results, searches for anomalies and other BSM signatures, and cross section measurements. LArTPC detectors are being used in current experiments such as ICARUS and SBND, and being planned for future experiments such as DUNE. MicroBooNE has recently released to the public two of its data sets, with the goal of enabling collaborative software developments with other LArTPC experiments and with AI or computing experts. These data sets simulate neutrino interactions on top of off-beam data, which include cosmic ray background and noise. The data sets are released in two formats: the native art/ROOT format used internally by the collaboration and familiar to other LArTPC experts, and the HDF5 format which contains reduced and simplified content and is suitable for usage by the broader community. This contribution presents the open data sets, discusses their motivation, the technical implementation, and the extensive documentation – all inspired by FAIR principles. Finally, opportunities for collaborations are discussed.
Scalable training on scalable infrastructures for programmable hardware
Authors: Lorusso Marco , Bonacorsi Daniele , Travaglini Riccardo , Salomoni Davide , Veronesi Paolo , Michelotto Diego , Mariotti Mirko , Bianchini Giulio , Costantini Alessandro , Duma Doina Cristina
Published: May 2024

Machine learning (ML) and deep learning (DL) techniques are increasingly influential in High Energy Physics, necessitating effective computing infrastructures and training opportunities for users and developers, particularly concerning programmable hardware like FPGAs. A gap exists in accessible ML/DL on FPGA tutorials catering to diverse hardware specifications. To bridge this gap, collaborative efforts by INFN-Bologna, the University of Bologna, and INFN-CNAF produced a pilot course using virtual machines, inhouse cloud platforms, and AWS instances, utilizing Docker containers for interactive exercises. Additionally, the Bond Machine software ecosystem, capable of generating FPGA-synthesizable computer architectures, is explored as a simplified approach for teaching FPGA programming.
The SMARTHEP European Training Network
Authors: Gooding James Andrew , Bozianu Leon , Cocha Toapaxi Carlos , Jawahar Pratik , Olocco Micol
Published: May 2024

Synergies between MAchine learning, Real-Time analysis and Hybrid architectures for efficient Event Processing and decision-making (SMARTHEP) is a European Training Network, training a new generation of Early Stage Researchers (ESRs) to advance real-time decision-making, driving data-collection and analysis towards synonymity.

SMARTHEP brings together scientists from major LHC collaborations at the frontiers of real-time analysis (RTA) and key specialists from computer science and industry. By solving concrete problems as a community, SMARTHEP will further the adoption of RTA techniques, enabling future High Energy Physics (HEP) discoveries and generating impact in industry.

ESRs will contribute to European growth, leveraging their hands-on experience in machine learning and accelerators towards commercial deliverables in fields that can profit most from RTA, e.g., transport, manufacturing, and finance.

This contribution presents the training and outreach plan for the network, and is intended as an opportunity for further collaboration and feedback from the CHEP community.
Acceleration of a Deep Neural Network for the Compact Muon Solenoid
Authors: Ourida Tarik , Luk Wayne , Tapper Alex , Barbone Marco , Bainbridge Robert
Published: May 2024

There are ongoing efforts to investigate theories that aim to explain the current shortcomings of the Standard Model of particle physics. One such effort is the Long-Lived Particle Jet Tagging Algorithm, based on a DNN (Deep Neural Network), which is used to search for exotic new particles. This paper describes two novel optimisations in the design of this DNN, suitable for implementation on an FPGA-based accelerator. The first involves the adoption of cyclic random access memories and the reuse of multiply-accumulate operations. The second involves storing matrices distributed over many RAM memories with elements grouped by index. An evaluation of the proposed methods and hardware architectures is also included. The proposed optimisations can yield performance enhancements by more than an order of magnitude compared to software implementations. The innovations can also lead to smaller FPGA footprints and accordingly reduce power consumption, allowing for instance duplication of compute units to achieve increases in effective throughput.
Advances in developing deep neural networks for finding primary vertices in proton-proton collisions at the LHC
Authors: Akar Simon , Elashri Mohamed , Garg Rocky Bala , Kauffman Elliott , Peters Michael , Schreiner Henry , Sokoloff Michael , Tepe William , Tompkins Lauren
Published: May 2024

We are studying the use of deep neural networks (DNNs) to identify and locate primary vertices (PVs) in proton-proton collisions at the LHC. Earlier work focused on finding primary vertices in simulated LHCb data using a hybrid approach that started with kernel density estimators (KDEs) derived heuristically from the ensemble of charged track parameters and predicted “target histogram” proxies, from which the actual PV positions are extracted. We have recently demonstrated that using a UNet architecture performs indistinguishably from a “flat” convolutional neural network model. We have developed an “end-to-end” tracks-to-hist DNN that predicts target histograms directly from track parameters using simulated LHCb data that provides better performance (a lower false positive rate for the same high efficiency) than the best KDE-tohists model studied. This DNN also provides better efficiency than the default heuristic algorithm for the same low false positive rate. “Quantization” of this model, using FP16 rather than FP32 arithmetic, degrades its performance minimally. Reducing the number of UNet channels degrades performance more substantially. We have demonstrated that the KDE-to-hists algorithm developed for LHCb data can be adapted to ATLAS and ACTS data using two variations of the UNet architecture. Within ATLAS/ACTS, these algorithms have been validated against the standard vertex finder algorithm. Both variations produce PVfinding efficiencies similar to that of the standard algorithm and vertex-vertex separation resolutions that are significantly better.
Applications of Lipschitz neural networks to the Run 3 LHCb trigger system
Authors: Delaney Blaise , Schulte Nicole , Ciezarek Gregory , Nolte Niklas , Williams Mike , Albrecht Johannes
Published: May 2024

The operating conditions defining the current data taking campaign at the Large Hadron Collider, known as Run 3, present unparalleled challenges for the real-time data acquisition workflow of the LHCb experiment at CERN. To address the anticipated surge in luminosity and consequent event rate, the LHCb experiment is transitioning to a fully software-based trigger system. This evolution necessitated innovations in hardware configurations, software paradigms, and algorithmic design. A significant advancement is the integration of monotonic Lipschitz Neural Networks into the LHCb trigger system. These deep learning models offer certified robustness against detector instabilities, and the ability to encode domain-specific inductive biases. Such properties are crucial for the inclusive heavy-flavour triggers and, most notably, for the topological triggers designed to inclusively select b-hadron candidates by exploiting the unique kinematic and decay topologies of beauty decays. This paper describes the recent progress in integrating Lipschitz Neural Networks into the topological triggers, highlighting the resulting enhanced sensitivity to highly displaced multi-body candidates produced within the LHCb acceptance.
Deep Learning to improve Experimental Sensitivity and Generative Models for Monte Carlo simulations for searching for New Physics in LHC experiments
Authors: Salt José , Balanzá Raúl , Garcia Azael , Gomez Jon Ander , González de la Hoz Santiago , Lozano Julio , Ruiz de Austri Roberto , Villaplana Miguel
Published: May 2024

ML/DL techniques have shown their power in the improvement of several studies and tasks in HEP, especially in physics analysis. Our approach has been to take a number of the ML/DL tools provided by several open-source platforms and apply them to several classification problems, for instance, to the tt¯ resonance extraction in the LHC experiments. Gradient-boosting Trees, Random Forest, Artificial Neural Networks (ANN), etc. have been used and optimized by means of adjusting several hyperparameters to control overfitting. On top of this, data simulation with traditional models is computationally very demanding, making the use of generative models an alternative for generating simulated Monte Carlo events with similar quality at a lower computational cost. This could help to produce more simulated data statistics available for better sensitivity and more accurate assessment of systematic errors in potential Physics Beyond Standard Model discoveries. In this work, we study the use of generative models based on Deep Learning as faster Monte Carlo event generators in the LHC context, reducing the time and energy cost of currently used methods. In particular, we focus on different configurations of Variational Autoencoders, taking as a starting point the well-known β-VAE and proposing the α-VAE as a new and simpler VAE architecture that improves the results in some experiments. Considerations will be made about the reliability of these simulated data when they are produced with very high statistics.
DeepTreeGAN: Fast Generation of High Dimensional Point Clouds
Authors: Scham Moritz A.W. , Krücker Dirk , Käch Benno , Borras Kerstin
Published: May 2024

In High Energy Physics, detailed and time-consuming simulations are used for particle interactions with detectors. To bypass these simulations with a generative model, the generation of large point clouds in a short time is required, while the complex dependencies between the particles must be correctly modelled. Particle showers are inherently tree-based processes, as each particle is produced by the decay or detector interaction of a particle of the previous generation. In this work, we present a novel Graph Neural Network model (DeepTreeGAN) that is able to generate such point clouds in a tree-based manner. We show that this model can reproduce complex distributions, and we evaluate its performance on the public JetNet dataset.