dataesr/scientific-paragraphs-categorization huggingface dataset:
DefenseFinder, a tool to automatically detect known anti-phage systems We set out to build DefenseFinder, a tool to detect all known anti-phage defense systems in a given genomic sequence. To do so, we used MacSyFinder  26  , a program dedicated to the detection of macromolecular systems. MacSyFinder functions using one model per system. Each model operates in two steps (Figure  1 ): first the detection of all the proteins involved in a macromolecular system through homology search using HMM profiles; second, a set of decision rules is applied to keep only the HMM hits that satisfy the genetic architecture of the system of interest. This two-steps approach is perfectly adapted for the detection of anti-phage systems, which can exist under different genetic architectures. In fact, it has already been used for the detection of CRISPR-Cas systems  22, 26.
For each of the 105 systems, we defined a model e. g. a set of customized rules and associated protein profiles (see Material and Methods). Briefly, we either used existing pfams/COGs or built custom HMM profiles resulting in a database of 755 profiles. The decision rules are typically defined by a list of mandatory, accessory, or forbidden proteins necessary for the detection of a given system (Figure  1c ). For each of the proteins, several homologs can be interchanged. For example, for CBASS type I, two mandatory proteins are necessary (the cyclase and the effector) and a third one is accessory. Each of these proteins have several homologs possible (Effector can be 1TM, 2TM, phospholipase…. ). Once a first model is defined, when possible,we evaluate it, against existing datasets  1, 19, 24, 27  (See Material and Methods).
For example, for the 10 systems described in  1  , where a detailed detection was provided for each system, we downloaded the same database of genomes and used our models to search for these systems. We could then compare our detection and evaluate which systems were for example missing. The model (either profiles or decision rules) could then be adapted. For example, an initial model was missing some occurrences of Shedu system (Supplementary Figure  1 ). We hypothesized our HMM profile might not have detected some occurrences of SduA (a core protein of Shedu). We built a phylogenetic tree out of all the SduA proteins found in this paper  1  and mapped which ones were missed by our detection (Supplementary Figure  1a ). We then included additional proteins from under detected clade to our HMM profile.
Using similar types of approach, models were improved and we could report a sensitivity for such systems, ranging between 98. 7 % (Shedu) and 99,4 % (Shedu) and a high specificity, ranging between 98,5 % (Druantia) and 99,97 % (Thoeris) (Supplementary Figure  1b ). Details about validation process for other systems are found in the methods section. All models are under a CC-BY-NC license and available online (https://github. com/mdmparis/defense-finder-models). The final step was to use such rules and profiles to build as a user-friendly tool to detect anti-phage systems in prokaryotic genomes (Figure  1. d). We provide a command line interface (easily installable through a python package mdmparis-defense-finder) as well as a webservice (https://defensefinder. mdmparis-lab. com/). Both take as an input a protein multifasta file (either one or several genomes at the same time) and generate two types of detection files: a list of detected systems, list of proteins involved in detected systems.
The update on the command line is distinct of the update of DefenseFinder allowing to have update of the rules and profile more frequently. Overall, we created DefenseFinder, a program that enables the detection of all known anti-phage systems in prokaryotic genomes. ). For systems with several types, each system is represented by one type and other types are indicated in parentheses (Full list, Supplementary Table  1 ). b. Workflow for the creation of DefenseFinder. c. Example of a DefenseFinder rule (in the MacsyFinder syntax) for the detection of system CBASS type I. Cyclase and effector proteins are mandatory while the sensing protein is only accessory. This means the system allows for it to be missing in a detected CBASS type I system. Different profiles are recognized for a protein. ex cyclase (SMODS, AGC_C, Cycl100).
The anti-viral arsenal of prokaryotes is highly variable In order to provide a genome-centric view of the anti-viral arsenal of prokaryotes, we applied DefenseFinder to a database of 21 738 fully sequenced prokaryotic genomes (21 364 bacteria, 374 archaea). We identified 104 737 different defense systems, comprised of 262 960 genes (Supplementary Tables  S2, S3 ). On average prokaryotes encode five anti-phage systems. The number of anti-phage mechanisms per genome varies widely from a minimum of zero (1 567 genomes) to a maximum of 51 in the deltaproteobacteria Desulfonema limicola. (Figure  2a , Supplementary Table  S4 ). 75% of prokaryotes encode more than two defense systems. To estimate the diversity of this anti-phage arsenal, we computed the number of distinct families of antiphage systems per genome (e.
We also observed a stronger positive correlation between the number of families of antiphage systems and the genome (Figure  2b  grey, Spearman r=0. 38, p-value<0. 0001). We reasoned that the number of anti-phage systems might also be influenced by the diversity of phages a prokaryote might encounter. This can be estimated by focusing on the number of prophages encoded in a given genome. We thus focused on the interplay between anti-phage systems and prophages. To do so, we detected prophages in the genomes of our database using Virsorter2 28 (see methods). We found 51 582 prophages in 16 315 genomes, with on average two prophages per genome (Supplementary Figure  3 ) which is in line with previous detection  29. The number of prophages correlates both with the number of systems (Spearman r=0.
Systems located within prophages are shorter (Figure  3. b, median prophagic systems=2 516bp, median chromosomic systems =3 820bp, t-test p-value<0. 0001). This is compatible with the size constraints exerted on such elements and suggest that different systems are encoded on the chromosome and prophages. To check this hypothesis, we computed for each system the frequency of the location (either prophagic or chromosomic) (Figure  3. d). For example, out of the 825 Kiwa, 610 are encoded on the chromosome (74%), and 215 in prophages (26%). We found that some systems such as NixI, Rst_3HP, BtsA and RexAB are encoded in majority on prophages, while some systems such as Dnd, viperin or RADAR are almost never found on prophages. The large size of certain systems (Dnd, BREX DISARM) could explain their absence on prophage, but it does not explain why small systems like viperins are not found on prophages in our dataset.
It is interesting to notice, that most systems primarily encoded in prophages (NixI, BtsA,RexAB, Lit, Rst system) were discovered in prophages and prophages parasites. While our results are restricted to prophages and not their satellites, they suggest that some systems might be prophage specific. Plasmids were excluded from this analysis. A system was deemed to be part of a prophage when the first and last protein of the system was inside the prophage boundary. The size of a system was computed as the difference between the end of the last protein and the beginning of first protein. c. Frequency of genomic location for each system. d. Number of systems per prokaryotic phylum. Only phyla with more than 50 genomes are represented. The number of genomes in the dataset is represented under each phylum's name.
Some systems, while less abundant, are widespread across all phyla (such as Abi2, Retrons, CBASS, Hachiman, AVAST…) whereas some other systems are enriched in specific phyla. Typically, many systems such as RexAB, Retron-Tm, Hp-Tm, Ape, are only present in Proteobacteria. This might be explained by a bias of the database towards proteobacterial genomes (n=12 464), which represents 57% of the database ; it could indicate the existence of more phylogenetically restricted systems, as most of these systems were discovered in prophages of Proteobacteria ; if a system evolves fast it might be missed in distant clades. Finally, some systems seem to be particularly enriched in specific phyla such as BREX in Fusobacteria (Frequency of 29% compared to 8% for all prokaryotes) or Wadjet in Actinobacteria (40% compared to 7% for all prokaryotes).
The anti-viral arsenal of bacteria is species specific Following our observation that some anti-phage systems are enriched in specific phyla, we decided to focus on the link between anti-phage defense and phylogeny. To do so, we examined the differences between the anti-viral arsenal of diverse species. We selected all the species with more than 100 genomes in our dataset (n=21, see Methods) and established a quantitative comparison between their anti-viral arsenal (Figure  4 , Supplementary Figure  5  and 6 ). Both total numbers of systems and number of different families varies widely between species. For instance, we found no anti-phage system in the 558 genomes of Bordetella pertussis, but 16 anti-phage systems on average in Helicobacter pylori strains. This is in line with previous reports of H.
Discussion We created a tool to detect all known anti-phage systems. To our knowledge, this represents the first tool to perform the detection for all known anti-phage systems. We hope that the availability of this tool in command line (for big data analysis) and through a web-service (for occasional detection) will allow the community to learn more about specific strains and bring new insights on large datasets. To that end, DefenseFinder is quite adapted as it runs in less than a day on 10 000 genomes on a regular laptop (using 4 CPU). A major challenge for such program is to adequately reflect the state of the literature in the field. Because the literature on this particular topic is ever changing, we took advantage of the architecture proposed by MacSyFinder to facilitate regular updates.
Finally, the systems are only detected through proteins and no nucleic acid components is detected here (such as msDNA for retrons or CRISPR arrays for CRISPR-Cas). This will be added in a future version. We detected and analyzed thousands anti-phage systems in fully sequenced bacterial and archaeal genomes. We used the entire RefSeq complete genome database, which is known to be biased towards specific prokaryotes, notably to over-represent cultivable bacteria. However, it represents the first census of the full anti-viral arsenal of prokaryotes. While we know this census will change in its details as many anti-phage systems probably remain to be discovered, we expect that the general trends that were observed in our study will not change much. Indeed, the most recent systems to be discovered are present in less than 15% of the genomes.
Validation of DefenseFinder models Following an initial design of rule for each system, we ran an initial detection and evaluated the results. When available, this initial detection was compared to other existing datasets. For example, specificity and sensitivity were evaluated for each system from  1  and is reported in Supplementary Figure  1. Sensitivity was defined as the percentage of system detected in 1 and by DefenseFinder. Specificity was defined as the ratio between the number of genomes where a system was detected by DefenseFinder in genomes where in  1  had not detected any and all the genomes where 1 has not detected the system. For CRISPR-Cas systems, detection was compared to CasFinder v2 from  22. For RM systems, we ran DefenseFinder on REBASE  24  (Supplementary Figure  1c ).
The comparison between these two results revealed almost no differences in frequencies of retrons types (Supplementary Figure  1 ). Some subsystems are less detected by DefenseFinder such as type X or type XIII. However, the database used for detection is not the same for the two analysis and the phylogenetic repartition could modify the type distribution. When no other datasets was available, the quality of the detection was estimated by checking different factors such as the number of occurrences found, size of proteins and systems. Each rule was thus refined through trial and error cycles to reach a final stable version. For each of the species, we took all genomes under a nucleic acid format in NCBI (fna) and annotated them using prodigal (PanACoTA annotate options --cutn 10000 --l90 400 -prodigal).
Nowadays, progress in the determination of three-dimensional macromolecular structures from diffraction images is achieved partly at the cost of increasing data volumes. This is due to the deployment of modern high-speed, highresolution detectors, the increased complexity and variety of crystallographic software, the use of extensive databases and high-performance computing. This limits what can be accomplished with personal, offline, computing equipment in terms of both productivity and maintainability. There is also an issue of longterm data maintenance and availability of structure-solution projects as the links between experimental observations and the final results deposited in the PDB. In this article, CCP4 Cloud, a new front-end of the CCP4 software suite, is presented which mitigates these effects by providing an online, cloud-based environment for crystallographic computation.
Maintenance burden As a method, MX has seen steady and active development over the past several decades and has produced a considerable variety of computational approaches and exemplary pieces of software, allowing one to cope with all computational aspects of experimental MX studies. For example, the CCP4 software suite  (Winn et al. , 2011)  version 8. 0 presents a user with more than 500 executable modules and requires about 6. 3 GB of disk space in a minimalistic setup. Full installation requires a set of additional databases and third-party software components, which require additional maintenance and increase the disk-space demand to over 50 GB. In a distributed framework, all software and databases are stored and maintained remotely (i. e. in the cloud) and thus are always kept in a ready and duly updated state.
, 2012). (vii) The FragFit server for the prediction, visualization and selection of missing segments in protein cryo-EM density maps  (Tiemann et al. , 2018). In addition, CCP4's online services  (Krissinel et al. , 2018 ) comprise a collection of automated tools for the following. (i) Molecular replacement (MR): MrBUMP  (Keegan & Winn, 2008) , BALBES  (Long et al. , 2008)  and MoRDA  (Vagin & Lebedev, 2015). (ii) Experimental phasing: Crank2 (Skubak & Pannu, 2013) and SHELX  (Sheldrick et al. , 2012; Uson & Sheldrick, 2018). (iii) Bias-free MR with fragments: ARCIMBOLDO  (Millan et al. , 2015). (iv) MR with ab initio modelling: AMPLE  (Bibby et al. , 2012). (v) MR with database searches: SIMBAD  (Simpkin et al. , 2018). (vi) Validation and analysis: PISA  (Krissinel, 2015)  and Zanuda  (Lebedev & Isupov, 2014).
CCP4 Cloud Client CCP4 Cloud Client (CC) is a special type of NC that runs on the user's device and enables the use of CCP4 graphical applications with CCP4 Cloud: Coot  (Emsley et al. , 2010 ), CCP4mg (McNicholas et al. , 2011) , DUI  (Winter et al. , 2018) , iMosflm  (Battye et al. , 2011)  and ViewHKL  (Evans & Krissinel, unpublished work). In the current version of CCP4, these applications cannot be executed within a browser and thus must run from a local installation of the CCP4 software suite. The CC provides a seamless integration of the local CCP4 installation with the rest of CCP4 Cloud, making all communication and data exchange completely automatic. CCs are important for image-processing tasks. Typically, X-ray diffraction images have a considerable size, which makes uploading them from the user's machine to remote servers impractical.
Therefore, in addition to image-processing GUIs (DUI, iMosflm and optionally XDSGUI;  Sparta et al. , 2016) , the CC can also run the automatic image-processing pipeline xia2  (Winter, 2010)  locally (note that xia2 can also be run in a remote NC using diffraction images uploaded by other means; for example, directly from a synchrotron beamline via a special link in the background). The CC launcher is distributed as part of the CCP4 software suite. Besides starting the CC, it also starts a chosen web browser and connects to CCP4 Cloud automatically. This means that starting a full-featured CCP4 Cloud session is as easy as starting a desktop GUI. Although very practical, CCs are optional elements of CCP4 Cloud. They require a local installation of the CCP4 software suite and so cannot be used when working from a device that cannot run CCP4 software (for example a tablet or smartphone).
This storage is different from the disk space allocated for user data and projects. The main purpose of Cloud Storage is to keep large volumes of data collected at the synchrotron and make them available for the data-processing pipeline xia2 in user projects. Each user can see only the part of Cloud Storage that was made accessible to them by the CCP4 Cloud administrator. In addition, Cloud Storage contains public areas that are usually used for keeping demo projects, tutorials and similar resources. CCP4 Cloud at RAL Harwell is publicly available and can be conveniently accessed via the pre-configured CC that is included as part of the CCP4 software suite. The CCP4 Cloud configuration in Fig. 1 (d) provides individual FEs for each user or groups of users but utilizes a common line of NCs for running jobs.
Jobs CCP4 Cloud Jobs are usually centred around primary components, such as Phaser  (McCoy et al. , 2007) , REFMAC  (Murshudov et al. , 2011) , Coot etc. , and use several more programs behind the scenes to perform file-format interconversions, prepare intermediate data files and analyse results. This organization results in the user being provided with a refined set of crystallographic tasks, which is more digestible than an exhaustive and overwhelming list of software tools. Task interfaces follow a common pattern (Fig. 3a ): all input data objects are placed in the upper part of the interface, with task parameters below them. All input fields are given default values, including input data objects. In most cases, input data are taken from the Structure Revision that was generated by the previous Job, making the data choice automatic and unambiguous.
However, ultimately the structural model will require multiple manipulations with REFMAC and Coot. The process is highly iterative, and after running each model-building or refinement task a new Structure Revision is created that contains the improved atomic coordinates and electron-density maps. 3. 3. 7. Validation, analysis and deposition. At the end of the structure-solution process, atomic coordinates from the last Structure Revision, along with reflection data, sequences and ligand descriptions, should be deposited in the Protein Data Bank (PDB;  Berman et al. , 2000). Deposition files, in mmCIF format, are prepared with a dedicated CCP4 Cloud task. However, before deposition, and iteratively during the modelbuilding and refinement procedure, the structure should be validated using the tools available in Coot and by inspection of the Quality Assessment section in the Job report for the REFMAC task.
Furthermore, the space-group hypothesis may be checked using Zanuda, carbohydrate structures are validated with Privateer  (Agirre et al. , 2015)  and additional analysis and validation may be performed with PISA. CCP4 Cloud Projects are objects themselves that can be exported as single files (ZIP archives), encapsulating all Jobs, data and metadata. The exported Projects are cross-platform compatible with all Microsoft Windows, Linuxes and Mac OS X. They can be re-imported in any instance of CCP4 Cloud; for example, that running on the user's own machine. Note that importing/exporting Projects is different from project sharing, which is described below. The main purpose of this facility is to mitigate limitations of the disk allocation quota given to users in centralized installations of CCP4 Cloud and also to serve as an additional backup.
Integration of third-party resources and software CCP4 Cloud is designed as a highly synergetic platform for the integration of software components, databases and web services from CCP4 and other projects. The system includes generic frameworks for importing data from external sources [for example, the wwPDB  (Berman et al. , 2003)  and the AlphaFold database  (Varadi et al. , 2022) ] and running programs that are not included in the CCP4 software suite, such as XDS  (Kabsch, 2010)  and XDSGUI, OpenFold  (Ahdritz et al. , 2021)  or AlphaFold2  (Jumper et al. , 2021)  for structure prediction, and BUSTER from Global Phasing Ltd  (Bricogne et al. , 2017 ) for model refinement. The frameworks include mechanisms for requesting or validating user licences online (for example, authorization from Global Phasing Ltd is required before the BUSTER task is made available to a user).
Teamwork CCP4 Cloud provides a convenient environment for organizing and storing structure-solution projects conducted by a research group or institution. Once in CCP4 Cloud, the projects are automatically backed up, and their lifetime is only limited by local data policies and IT support. All metadata in CCP4 Cloud are versioned and designed to be backward compatible: completed projects can therefore be revisited and revised or extended at any point in the future. CCP4 Cloud projects can be shared between two or more CCP4 Cloud users. In this way, several researchers can work on a project simultaneously and see each other's actions in real time. This unique feature is particularly convenient for distributed teamwork, exchange between collaborating organizations, training young scientists and reporting software bugs.
Discussion and conclusions Although CCP4 Cloud does not currently expose all components of the CCP4 software suite through its graphical interface, it represents a sufficiently complete system for structure solution and includes all principal stages of MX computing from image processing to model deposition. A few alternatives exist for each stage, leading to functional redundancy but a higher probability of solving structures in difficult cases for the novice or experienced MX user. We therefore argue that CCP4 Cloud is functionally complete for use in the majority of practical cases. CCP4 Cloud is distributed as part of the CCP4 software suite (version 7. 1 and higher) and may be used out of the box for working with the public instance of CCP4 Cloud maintained at RAL Harwell (https://cloud.
ccp4. ac. uk). In addition, any CCP4 7. 1+ setup may be configured as a server featuring a local instance of CCP4 Cloud, allowing users to benefit from in-house computational resources and avoid transmitting sensitive data to public servers. CCP4 Cloud can be installed on a range of hardware platforms and operating systems (Linux, Mac OS X and MS Windows) and allows mixed setups including individual workstations, in-house clusters, centralized HPC facilities and generic clouds. CCP4 Cloud setups can be adjusted to a wide range of available resources; in the simplest configuration, CCP4 Cloud acts as a mere GUI on a single laptop or PC. Designing a modern, data-driven interface was one of the main ambitions of the CCP4 Cloud project. CCP4 Cloud's GUI eliminates the burden of managing many files, as are typically produced in MX projects, by abstracting them with composite data types and managing all data flows within CCP4 Cloud.