dataesr/scientific-paragraphs-categorization huggingface dataset:
MoBIE viewer implementation The MoBIE viewer is implemented in Java. The majority of the source code is contained in two github repositories: https://github. com/mobie/mobie-viewer-fiji and https://github. com/mobie/mobie-io. The mobie-io repository contains the core functionality for the loading and writing of various big image data formats (BDV-HDF5 13 , BDV-N5 2 , OME-Zarr 1 ) from and to file systems or S3 object stores. The mobie-viewer-fiji repository implements the Fiji plugins for big image data loading, writing and visualization (see above section on MoBIE's core features). The code builds upon various other java libraries, most notably ImgLib2 10 , BigDataViewer  13  , ImageJ 3D-viewer  15  , N5 2 and bigdataviewer-playground (https://github. com/bigdataviewer/bigdataviewer-playground).
Project structure, storage and metadata In order to be accessible via the MoBIE plugin, data has to be organized according to the MoBIE Project specification. A project may contain several datasets, which contain the image data, tabular data and view metadata that can be displayed by a single MoBIE instance. Different datasets in a project could for example correspond to separate specimens from an experiment or to versioned analysis results of the same data. Each dataset corresponds to a sub-folder in the root project directory, which also contains a project. json file that lists all available datasets and contains some further project metadata. This metadata includes the image file formats used in the project, which determine the available modes of data access. Project data can be accessed in different ways, depending on the type of data: image data can be accessed either from the local filesystem or from object storage.
Tabular data and metadata can be accessed from the local filesystem, object storage or github. This allows three different access modes for a project: i) all data is read from the local filesystem, ii) all data is read from object storage, or iii) image data is read from object storage and tabular data as well as metadata is read from github. Options ii) and iii) enable data access in the cloud, and thus enable convenient sharing of MoBIE projects with other researchers. Data can be made fully public, using non-access-restricted object storage and github repositories, or in a closed collaboration, using object storage with credential based access right management. In order to access image data in object storage we rely on modern file formats that allow on-demand chunk access, see next paragraph for detail.
Currently, we only support remote data access in object storage with an Amazon S3 compatible API; but extension to other protocols such as Azure Blob Storage would be possible. Note that the S3 API is open source and tools such as min. io (https://min. io/) that allow setting up on-premise S3 object storage exist. Consequently reliance on the S3 API does not result in vendor lock-in. All metadata associated with MoBIE is stored in JSON files, which follow a json schema (https://json-schema. org/), a JSON based language to describe metadata formats. This schema can be used to statically verify the correctness of any MoBIE project. Specifically, we define schemas for the project. json file (see first paragraph), the dataset. json file (see next paragraph) and JSON files for view specification.
The MoBIE schemas are available at https://github. com/mobie/mobie. github. io/tree/master/schema. A dataset contains the data available to a MoBIE plugin datasets can be switched using the "dataset" dropdown in the plugin's main menu. Each dataset has a root level folder that contains the dataset. json file, which specifies the sources and views available. Sources correspond to image data. We currently support two different types of sources: imageSource for intensity images and segmentationSource for label mask images. For both source types the metadata specifies image data formats and data locations. We currently support the following formats: OME-ZARR 1 , BDV-N5 (https://github. com/bigdataviewer/bigdataviewer-core/blob/master/BDV%20N5%20format. md ) and BDV-HDF5  13. Both BDV file formats use an additional xml file that specifies image metadata; OME-ZARR stores this metadata directly in the zarr file that contains the image data.
Format Read Segmentation sources can additionally specify the location of tabular data stored as tab separated values (tsv). For each segmentation the relative path to a folder, which contains arbitrarily many tables can be given. These tables must contain a column called label_id, which maps to the label mask ids in the corresponding image data, and may contain arbitrary additional columns. By convention, image data should be stored in a subfolder called images, which should contain subfolders for each image format. Tables should be stored in a subfolder called tables, with a separate subfolder for each segmentation (or region display, see below) with associated tabular data. Datasets may contain another subfolder called misc for other data, such as JSON files specifying additional views (see below).
See the image to the right for the recommended MoBIE project folder structure. In addition, the dataset metadata contains a list of views, which correspond to pre-set viewer configurations that are selectable in the MoBIE menu. A view consists of the following elements: sourceDisplays define the image or segmentation sources loaded in the view, and their display settings such as brightness, opacity or blending modes. For segmentation sources they also contain the tables that will be loaded and visible as well as selected segments, if any. In addition, they may also contain regionDisplays, which enable annotations for specific areas in the viewer coordinate space, e. g. annotations for each position in a grid transformation. The region displays must have an associated table, which has a mandatory column region_id, and otherwise share the same properties as the segmentation tables.
Next, sourceTransforms define the on-the-fly transformations, such as affine, grid or crop transformations, that are applied to the sources in the view. Finally, the viewerTransform defines the "camera position" of the view. A full description of the MoBIE project specification, including examples for the JSON specifications, is available at https://mobie. github. io/specs/mobie_spec. html, the version of the specification at time of writing is available at https://mobie. github. io/v021/.
Project creation Fiji The MoBIE Fiji plugin includes a series of tools to simplify the creation of MoBIE projects. This project creator is implemented in Java, with the source code included in the main MoBIE repository: https://github. com/mobie/mobie-viewer-fiji. It can convert images that are loaded into Fiji to both the BDV-N5 and OME-Zarr file formats using functions from the mobie-io repository: https://github. com/mobie/mobie-io. This conversion builds on top of the export functions from the bigdataviewer_fiji (https://github. com/bigdataviewer/bigdataviewer_fiji), bigdataviewer-core (https://github. com/bigdataviewer/bigdataviewer-core) and Saalfeld lab n5-zarr (https://github. com/saalfeldlab/n5-zarr) repositories, adding various new features including downsampling methods for label mask images.
We also employ its annotation features to highlight the bounding boxes and positions of the individual sources for the different figure panels. The individual sources were registered on top of each other using the transformation tools available in MoBIE. We can also visualize the high-resolution content of the dataset in parallel using a grid view to enable direct visual comparison of morphological features. We demonstrate MoBIE's utility for high-throughput microscopy (HTM) on data from an immunofluorescence assay for SARS-CoV-2 antibodies  20. Briefly, the assay uses Vero E6 cells that are partially infected with the virus, mixes the cells with human serum and images them on a 96 well plate. Three channels are acquired, staining nuclei, viral RNA and IgG antibodies in the serum. To determine the antibody response all individual cells are segmented based on the nucleus and serum signals.
Access of data deposition archives Increasingly, imaging data from publication in the life sciences is deposited in online image archives to ensure reproducibility of the corresponding study and enable re-use of the data for other research purposes. Many journals and funding agreements also mandate this data deposition. In most cases, this data is available only for bulk download, which makes re-use of big datasets challenging since a potential user would first need to download a large amount of data before assessing if it even fits their needs. On demand access of image data in online deposition archives is thus highly desirable to enable interested researchers to quickly check this data, or even enable them to perform full data exploration and analysis on demand. With its support for many data modalities, extensive features for data exploration and analysis and capability to stream image and tabular data on demand, MoBIE is an ideal candidate application for this task.
In fact, we have already implemented prototypes for on demand access of data from two online deposition archives. The first is Open Organelle  27  , which is a data deposition archive hosted by HHMI Janelia Research Campus for storing very high resolution EM volumes of cells, derived organelle segmentations and correlative LM data. It uses n5 to store the data, which is publicly accessible via S3 storage and relies on a custom image metadata format. We have added experimental support for this image data format to mobie-io, and can thus create MoBIE projects that access data via Open Organelle. A test project demonstrating this functionality is available at https://github. com/mobie/open-organelle-test. It uses the "jrc-hela-2" cell from the openOrganelle cell repository and only contains the EM data and the endosome segmentation.
Adding further segmentations would be simple. Creating and extending Open Organelle compatible projects is also supported by the MoBIE python library. The second is the BioImage Archive 28 , an image data repository hosted by the European Bioinformatics Institute. It accepts image data submissions associated with publications in the life-sciences. It stores this data on S3 compatible object storage, so data deposited in a format supported by MoBIE can be streamed from it. We demonstrate this capability with an example project that corresponds to a subset of the data from Cortese et al. 21. It is available at https://uk1s3. embassy. ebi. ac. uk/bia-mobie-test.
C. P. has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under grant agreement number  824087. C. T. acknowledges support by grant number  2020-225265  from the  Chan Zuckerberg Initiative DAF , an advised fund of  Silicon Valley Community Foundation. J. M. also received funding from the  Chan Zuckerberg Initiative DAF  for work on OME-NGFF by grant number  2019-207272  and on Zarr by grant numbers  2019-207338  and  2021-237467. D. A. is supported by a grant from the  728 European Research Council  ( NeuralCellTypeEvo 788921 ). We would like to thank  Tobias Pietzsch ,  John Bogovic , and  Stephan Saalfeld  for invaluable help with ImgLib2 and BigDataViewer code development; the  EMBL IT department , in particular  Josep Moscardo , for supporting the S3 infrastructure;  Siân Culley  for inventing the name MoBIE;  Gemma Estrada Girona  for drawing the MoBIE icon;.
Overview FeatureCloud was developed as a unified platform to increase the accessibility of FL for two large user groups as follows: (1) end users running FL algorithms to train ML models on distributed data sets and (2) developers implementing federated algorithms for statistics or ML that are not easily accessible in federated environments yet. As illustrated in Figure  1 , the interface between developers and end users is our integrated AI store. Application developers can easily implement their own applications and publish them in the AI store, making them easily accessible to end users. Out of a broad collection of applications in the AI store, end users can assemble tailored workflows, invite collaborators, and perform FL on geographically distributed data. Therefore, FeatureCloud provides a complete infrastructure, including secure state-of-the-art communication, no raw data sharing, and several mechanisms to keep the actual data private.
System Architecture FeatureCloud was developed as a system consisting of several interacting parts distributed between the participants and a central server. The central components include the backend (Python and Django), frontend (Angular), and Docker registry. The local components include the controller (Golang), the Docker engine, and the application instances (Docker images). Figure  2  shows the system components and the communication channels between them. Further details regarding their implementation and technology used can be found in Multimedia Appendix 1. The frontend is a web application running on a web browser. It uses the FeatureCloud backend API (link 1 in Figure  2 ) to offer all the features of the AI store and for collaborative project management. It is also connected to the controller to allow for monitoring and handing over data for workflow runs (link 2 in Figure  2 ).
In addition, the controller is an integral part of the security and privacy system of FeatureCloud. It handles local data processing and is the only part of FeatureCloud that has access to the local computer system. The controller runs in a Docker container to prevent random access to data on the system. Therefore, it only has access to selected data sets that were actively chosen by a system administrator or a user through a FeatureCloud application. The participants of a federated workflow must also agree on a common relay server. The relay server, implemented in Go, is responsible for transmitting all traffic of the federated algorithms via a secure socket connection (link 7 in Figure  2 ). This central communication hub is aware of all the participants and their roles in the federated execution.
It follows the required communication pattern, sending aggregated models to all the participants and local model parameters to the coordinating party only. Although FeatureCloud provides a relay server instance used by default, it is possible to use a private instance to completely shield the traffic from anyone outside the collaboration by adjusting the configuration file for the controller. As FeatureCloud applications are a dynamic system component, partly contributed by external developers, it is necessary to isolate their implementation. This is achieved by using Docker, which ensures that they cannot access system resources other than required, especially the filesystem and network, and allows for limiting resource use, such as central processing unit or memory. They receive their input data inside a Docker volume and communicate with the controller through a defined API (link 8 in Figure  2 ).
FL Scheme and PETs FL generally involves two possibly alternating operations as follows: (1) local optimization and (2) global aggregation. In FeatureCloud, all running instances of a federated application have 1 of 2 roles (participant and coordinator) performing the respective federated operation. FeatureCloud expects precisely 1 coordinator and an arbitrary number of participants, leading to a star-based architecture. We chose this architecture over others because it mirrors the general design of a FL scheme with a central aggregator and clients with local data sets. After the local learning operation has been completed by a participant, it sends the local parameters to the coordinator. The coordinator collects these parameters and aggregates them into a collective (global) model, which is shared with the participants again.
Depending on the type of ML algorithm, these 2 operations can alternate multiple times, for example, until convergence or a predefined number of iterations has been reached (Figure  S1  in Multimedia Appendix 1). For some algorithms (eg, random forest [RF] and linear regression), only 1 iteration is necessary. However, this strict separation between optimization and aggregation is not actively enforced by FeatureCloud. In many cases, aggregation can start after the first parameters have been received, thereby increasing efficiency through parallelization of the computation. During the implementation of a federated application, the distinction between the coordinator and the participant is of conceptual relevance. However, in practice, the coordinator can also obtain local data that can be used for training.
Therefore, FeatureCloud allows the coordinator to simultaneously adopt the role of a participant. Although FL improves privacy, it can still leak information to the coordinator, who can see all individual models before aggregating them. Local updates of the model based on a previously distributed global model may reveal information regarding the primary data  [29]. Secure aggregation techniques can address this problem. In FeatureCloud, we integrated additive secret sharing as a mitigation method to obtain the global sum without revealing the local submodels. Application developers can use this method with minimal or no added complexity to their algorithms. More details can be found in Multimedia Appendix 1.
This reduces technical details and makes applications user friendly for end users, independent of their background. When users add applications to their library, they can assemble them into a workflow and manage the execution with other collaborators on the FeatureCloud website without having to download any additional software. The AI store has a broad selection of popular ML models, as listed in Table  1. The applications are categorized into preprocessing, analysis, and evaluation. Some analysis applications, such as linear regression and RF, are generic and suitable for different data types and application scenarios. These applications can be easily integrated into a federated workflow with preprocessing and evaluation applications, such as a federated standardization of the input data and a final evaluation of the trained classifier with several performance metrics.
Applications can provide their own user interface, allowing for user interaction if necessary and for showing specific reports. Users can monitor application logs and react in case something unexpected occurs (eg, stop and rerun the workflow with other data or a different configuration). When the last application in the workflow successfully completes its computation, the final results are automatically shared with all project participants. Intermediate results and application logs remain available on the local machines to allow for later verification. For example, the results may include a report showing the effectiveness of the trained model and the model itself. The latter can also be used outside of FeatureCloud. For example, if a project fails because a participant drops out, it can be restarted quickly after the problem has been solved.
Methods and Data Sets To evaluate the practical applicability of FeatureCloud, multiple workflows operating on different data sets were created. Except for DL, each workflow consists of a cross-validation (CV) application (10-fold CV), a standardization application, a model training application, and a final evaluation application (Figure  3 ). For DL, we evaluated a 20% test set, as this is more common for big data to reduce the training time. Individual applications are data-type agnostic and are suitable for various applications. Classification analyses were performed on the Indian Liver Patient Dataset  [34]  with 579 samples and 10 features and the Cancer Genome Atlas Breast Invasive Carcinoma  [35]  data set with 569 samples and 20 features. For regression analyses, they were evaluated on the Diabetes  [36]  data set with 442 samples and 10 features and the Boston  [37]  house prices data set with 506 samples and 13 features, both provided by scikit-learn  [30].
Similarly, we evaluated the Flimma application for differential gene expression analysis  [16]  as an example of biomedical data on a subset of 152 breast cancer expressions from the Cancer Genome Atlas repository  [42]  with 20,536 features. Our Flimma application produced highly similar results to those of the centralized analysis (Figure  S3  in Multimedia Appendix 1). These 2 examples further show that FeatureCloud has the capability of implementing and running different approaches and bringing them into a production system.
Runtime and Network Traffic Multiple executions with varying numbers of clients were performed to assess the scalability of the FeatureCloud platform and the federated methods. RF and linear regression classifiers were chosen as the iterative and noniterative methods, respectively, and both were applied to the Indian Liver Patient Dataset. Both were tested with 2, 4, 6, and 8 clients and the same number of samples to ensure comparability across the executions. To investigate the impact of network bandwidth on runtime, all executions were performed on a normal and throttled internet connection with a maximum transmission of 100 kB per second. Figure  5  shows that runtime mildly increases for logistic regression but decreases for RF. This is because the logistic regression models are of equal size for all clients, whereas the size of the RF models depends on the number of trees.
The local computations occur in parallel such that an increasing number of participants does not have a huge impact. However, because the aggregation step cannot be completed before all participants send their models, the runtime of each aggregation step depends on the slowest participant, which poses a potential problem for large federations. FeatureCloud primarily focuses on being used in a tightly regulated medical research environment. Therefore, there is currently no automatic "matchmaking" in place, but all participants must join each project actively. In this context, running an analysis with data sets of >8 participants is still an uncommon scenario. To demonstrate its scalability and robustness for more sophisticated scenarios, we evaluated the FeatureCloud platform using the logistic regression application for 1, 5, 10, 15, 20, 25, and 30 clients on simulated data, with each client containing 1000 samples and 1, 5, and 10 iterations.
Summary of Results In this study, we presented the FeatureCloud platform, a comprehensive platform for the application and development of privacy-preserving FL workflows in biomedicine and beyond. Through its high generalization, it allows the application of various ML workflows to a variety of data types. In addition, it offers prebuilt solutions for common-use cases in the form of applications in the AI store or application templates for developers. The concept of freely composing applications in a workflow is challenging because of the need for a standard data format, which is not always available and can reduce flexibility. The same applies to the initial data, which need to be provided in a form that is processable and understandable by the desired application. As FL adaptation is still in its early stages, it is necessary to understand which functionality and types of data will be used, which ML techniques prove to be most prevalent in federated settings, and which challenges arise when using the platform.