European XFEL Data Analysis Workflows for SFX and beyond Oleksii Turkot, Fabio Dall’Antonia, Richard J. Bean, Juncheng E, Hans Fangohr, Danilo E. Ferreira de Lima, Sravya Kantamneni, Henry J. Kirkwood, Faisal H. M. Koua, Adrian P. Mancuso, Diogo V. M. Melo, Adam Round, Michael Schuh, Egor Sobolev, Thomas A. White*, Raphaël de Wijn, James J. Wrigley, and Luca Gelisio European X-ray Free Electron Laser, Schenefeld, Germany. European XFEL GmbH, Fabio Dall’Antonia, Holzkoppel 4, 22869 Schenefeld, Germany, Phone +49 40 8998-6411, fabio.dall.antonia@xfel.eu www.xfel.eu The SPB/SFX instrument [1] at the European XFEL (EuXFEL) provides a portfolio of experiment techniques for the structural elucidation of biomolecules, relevant to the life science domain - among those single particle imaging (SPI), small-angle X-ray scattering (SAXS) and serial femtosecond crystallography (SFX). All of these can exploit the MHz repetition rate of both the FEL accelerator and synchronised megapixel detectors in order to capture dynamic processes in the nanosecond time regime. Due to the ultra-short, ultra-brilliant laser pulses combined with rapid sample re-supply in e.g. liquid jets, SFX can be applied to micrometer-sized crystals, producing millions of almost damage-free single-shot diffraction images. Conclusions and Outlook References [1] A. Mancuso et al. (2019). J. Synchrotron Radiat. 26, 660. DOI: 10.1107/S1600577519003308 [2] T. A. White et al. (2012). J. Appl. Cryst. 45, 335. DOI: 10.1107/S0021889812002312 [3] O. Turkot et al. (2023). Crystals 13 (11). DOI: 10.3390/cryst13111533 [4] https://github.com/European-XFEL/DAMNIT Acknowledgements This work was partially supported by the consortium DAPHNE4NFDI in association with the German National Research Data Infrastructure (NFDI) e.V. NFDI is financed by the Federal Republic of Germany and the 16 federal states and the consortium is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - project number 460248799. You are most welcome to contact the EuXFEL Data Analysis Group at da@xfel.eu. HALRIC Meeting 2024 Due to the peculiar pulse structure, a vast number of data sources and specifics of custom- built MHz pixel detectors, EuXFEL data are both large and complex. Accessible and reproducible data analysis workflows require software that can manage the complexity of data and provide low-barrier entry points for scientists. EXtra-Xwiz is a SFX-workflow-specific pipeline tool to interface CrystFEL with the preprocessing of EuXFEL data and distributed computing on HPC clusters. Other pipelines at EuXFEL, such as for X-ray photon correlation spectroscopy (XPCS), exist. DAMNIT follows a generic philosophy for data extraction from primary experiment sources as well as embedded processing software, and can be used for any experiment technique at EuXFEL, including SFX, SAXS and SPI. We are working on better support for the NeXus data standard, which will further enhance interoperability between software as well as between photon science facilities CrystFEL [2] is a widely used software suite that specialises on SFX data processing. EXtra-Xwiz [3] is a tool that manages data processing pipelines using CrystFEL, streamlining aspects such as the ingestion of EuXFEL-format data files and the scheduling of distributed computing on the HPC cluster. It can be integrated to the DAMNIT software [4] for metadata extraction and data analysis workflow management. B SFX at EuXFEL – i. p. the SFB/SFX upstream interaction region in vacuum – exploits a huge number of “shots” at the sample which is delivered in a jet. The X-ray probe exhibits a discontinuous 10 Hz train structure with MHz pulse repetition. The crystal hit rate is mostly below 10%. Peak finding and indexing with indexamajig (right, step 2) allows to distinguish between Bragg diffraction images and useless image frames. DAMNIT is driven by high level Python code and presents run-based primary metadata as well as processing results