dataesr/scientific-paragraphs-categorization huggingface dataset:
A. ICS CASE STUDY In our main case study, we consider the SCADA liquid storage infrastructure dataset  [27] , which simulates a fuel storage system supplying an automated production line monitored by an ICS system. The high-level overview of the testbed system is shown in Fig. 6. As depicted in Fig 6 , the system is composed of the main tank and secondary tank with a capacity of 9 and 7 liters, respectively. Data is collected by connecting the sensors to a PLC. Four discrete sensors in the main tank (corresponding to features IN0, IN1, IN2, and IN3) and one in the secondary tank (corresponding to features R4 and Slope) are used to measure the level of fuel. Pump1 and Pump2 control the flow of fuel between two tanks, represented by the pair feature PG-VG and PP-VP, respectively. PLC registers 2 through 4 provided output data elaborating the state of the system used for analysis.
B. EXPERIMENT SETUP To implement the proposed FedeX architecture, we set up a small-scale testbed as follows: • 4 Raspberry-Pi-4-Model-B kits acting as edge devices; Raspberry-Pi-4 equipped with quad-core 1. 5 GHz ARM Cortex-A72 processor and 4 GB RAM with 32-bit Raspbian OS. • 1 Dell Precision 3640 Tower Workstation serves as Cloud Server; the workstation with Intel Core i7-10700K 3. 8 GHz (up to 5. 1 GHz), 16 GB RAM, working on Linux operating system. • All edge devices and the Cloud Server are connected by a router through a WIFI interface. At the edge devices (i. e. Raspberry-Pi-4), we implement our FedeX framework in Python 3 with the TensorFlow 2 platform, which is built with the support of the FL framework -FedML  [6]. In the FedeX architecture, the edge devices and cloud server exchange the weights and bias matrix of the VAE model using the standardized MQTT protocol for an IoT environment  [28].
EMQ X Broker (2021) is hosted on the cloud server as an MQTT broker for better long-term performance. We discover EMQ X Broker as the most scalable open-source broker that could accept more advantageous devices linked to the server. As our proposed architecture leverages edge computing, it is also important to assess the edge efficiency during training in the ICS context. For this purpose, on edge devices, we utilize tool bmon to measure bandwidth occupation on a upstream link to Cloud Server, tool resmon to monitor computational resources of the edge devices. Besides, we measure power consumption with the support of an external monitoring gadget UM25C USB Tester which is directly connected to Raspberry Pi 4. The experimental data set has 10 features, namely IN0, IN1, IN2, IN3, R4, Slope, PG, VG, PP, and VP.
e. the SCADA liquid storage infrastructure dataset  [27] ) • Scenario 2: FedVAE-SVDD and its centralized counterpart versus other previously proposed AD solutions in our main case study of ICS and different SCADA datasets. Also note that, since the SVDD model is used to automatically determine an optimal detection threshold for the FedVAE model on each different training dataset. In our experiments, the optimal thresholds found for 4 distributed zones (i. e. , Zone 1, Zone 2, Zone 3, and Zone 4) are 0. 11, 0. 09, 0. 09, and 0. 09 respectively. In case of centralized learning (i. e. , the whole original SCADA data set is used), the threshold is found 0. 26. Since the learning model converges after 3 communication rounds, the results shown in the following subsections are retrieved after the 3 rounds.
b: SCENARIO 2: FEDVAE-SVDD VS. OTHER ANOMALY DETECTION SOLUTIONS In this experiment scenario, we will study the performance of FedeX in comparison with 14 other reference solutions comprehensively, including: machine-learning reference methods is reported by  [8] , namely LR, LDA, KNN, CART, NB, and SVM; SVM and DNN reported by  [9] ; MLP, CNN, and RNN reported by  [10] ; LSTM reported by  [11] ; MADICS  [12]  based on LSTM, 1D-CNN  [13] , and LAD-ADS  [14]  using the rule-based method. Our experiments are run with 2 cases: with the SCADA liquid storage infrastructure dataset  [27] ), and (2) with the well-known SWaT dataset  [22]. Since SWaT has been considered as an imbalanced dataset, the accuracy metric is not preferred in most of the recent researches as we have investigated. The performance of all 14 other AD solutions, our solution and its centralized learning counterpart can be seen in Table  3 , in which FedVAE-SVDD outperforms 13 other reference methods on the SCADA liquid storage infrastructure dataset in all metrics, even with its centralized counterpart (i.
e the centralized VAE-SVDD). Only the performance of LAD-ADs on the SCADA liquid storage infrastructure dataset has not been reported since its code is not available and can hardly be reproduced the same as the original paper. Therefore the performance of LAD-ADs can only be compared over the Swat dataset with the same experiment setting. But overall, the results show that our learning model is suitable for such a case study. On the well-known SWaT dataset, FedVAE-SVDD obtains the highest performance in comparison with all classic and deep learning references in terms of Recall and F1-score. Again, let us note that Recall and F1-score are the 2 important metrics for ICSs. In the case of this imbalanced dataset, a good F1-score figure is necessary since the number of abnormal samples is so much different from the number of normal samples.
Accordingly, both FedVAE-SVDD and its centralized counterpart are prominent with the maximum F1-score of 0. 9857 and 0. 9855, respectively. Besides, our solution performance is also higher than the average of the remaining references regarding to the Precision metric. In particular, it is worth mentioning that LAD-ADS is, perhaps, one of the best methods in recent researches over the SWaT dataset, with F1-score of 0. 914. But FedVAE-SVDD still outperforms it in this scenario.
In both of these scenarios, SHAP suggests that the R4 feature has a significant impact on predicted anomalies, similar to the analyses mentioned in a SCADA dataset research  [29]. The authors confirm that the ultrasound sensor badly affects most of the abnormal scenarios in the dataset and the value of R4 measured by this sensor is most significant. Accordingly, it can be seen that our XAI-based explanatory solution is capable of precisely identifying the primary cause related to the anomaly in reality. In conclusion, based on these positive findings, we would like to make some comments and recommendations. Firstly, our scheme can make a comprehensive explanation for detected anomalies, boosting the reliability of FedeX. Besides, if there are the occurrence of unknown threats, FedeX will still support operators to determine affected physical components and come up with timely responses rather than inspecting the entire system.
V. DISCUSSIONS A. CONTRIBUTIONS Based on the mentioned motivations, FedeX is a framework intended to achieve high detection performance, learn new data patterns fast, have lightweight, and improve the interpretability of the model. To the best of our knowledge, existing works have not yet achieved all of these features in the literature. In order to verify the performance of FedeX, we utilized an SCADA liquid storage infrastructure dataset as the main case study and another well-known dataset-SWaT. The obtained results show that FedeX achieves remarkable detection performance that outperforms many existing anomaly detection methods, whilst still attaining a fast training time of 7. 5 minutes. This facilitates frequent retraining in ICSs. With edge computing, our proposal enables a faster system response against attacks since the detection module is near attack sources.
as a local dataset. Step 2 : The edge device performs the local model (i. e. , VAE model) and the mechanism for determining a threshold at the last communication round (i. e. , FedVAE-SVDD model) training on the local dataset. Step 3 : The edge device uploads the weight matrix to the cloud aggregator. Step 4 : The cloud aggregator obtains a new global model by aggregating the weights sent by the edge device. Step 5 : The cloud aggregator sends the new global model to each edge device. The steps above are repeated until the global model achieves optimal convergence. This ideal global model can be used by decentralized devices to conduct anomaly detection tasks. Step 6 : Periodically, the XAI-SHAP model will be run to interpret and verify the anomaly detection model; and identify the anomaly-causing elements in ICSs.
! Abstract-We present a method for automated nucleus identification and tracking in time-lapse microscopy recordings of entire developing embryos. Our method combines deep learning and global optimization to enable complete lineage reconstruction from sparse point annotations, and uses parallelization to process multi-terabyte light-sheet recordings, which we demonstrate on three common model organisms: mouse, zebrafish, Drosophila. On the most difficult dataset (mouse), our method correctly reconstructs 75. 8% of cell lineages spanning 1 hour, compared to 31. 8% for the previous state of the art, thus enabling biologists to determine where and when cell fate decisions are made in developing embryos, tissues, and organs. M With recent advances in light-sheet imaging techniques, it is possible to acquire whole embryo developmental datasets over long time scales with high spatial and temporal resolution in complex organisms such as mouse, Drosophila, and zebrafish  (Wan et al.
, 2019a). The resulting datasets contain information required to track the movement and division of nuclei over time, yielding lineage trees and quantitative data on cellular dynamics that are crucial to the study of developmental biology at the cellular level  (Spanjaard and Junker, 2017). However, manually tracing lineages with dedicated tools like MaMuT  (Wolff et al. , 2018)  or Mastodon (https://github. com/mastodon-sc/mastodon) is arduous, and for complex, developing organisms it is only feasible to annotate a small percentage of all tracks, making automatic cell tracking necessary for holistic analysis. Cell tracking algorithms have been developed for and tested on diverse datasets with different characteristics. While handengineered features are sufficient for cell detection and tracking in some model organisms  (Bao et al.
, 2006; Amat et al. , 2014) , learned dataset-specific features, given sufficient training data, improve performance for datasets with heterogeneous cell or nucleus phenotypes and varying imaging statistics over time and space. In particular, deep learning has been shown to improve cell detection  (Kok et al. , 2020; Hayashida et al. , 2020) , segmentation  (Weigert et al. , 2020; Cao et al. , 2020; Stringer et al. , 2021; Medeiros et al. , 2021) , and tracking  (Sugawara et al. , 2021; Ulman et al. , 2017; Moen et al. , 2019; Hayashida et al. , 2020; Medeiros et al. , 2021)  on a variety of datasets. Additionally, it has been shown that tracking methods that take into account global spatiotemporal context perform better, especially for datasets with more movement between time frames  (Ulman et al.
, 2017). Tracking by graph optimization over a large spatiotemporal context allows inclusion of biological knowledge about track length and cell cycle, improving track continuity  (Jug et al. , 2016; Haubold et al. , 2016; Kok et al. , 2020)  and even allowing recovery from noisy detection and segmentation  (Schiegg et al. , 2013). Only a few of the aforementioned cell tracking methods are readily applicable to the unique challenges posed by contemporary 3D light-sheet datasets, the focus of this work. Practical methods for this kind of data should take into account temporal and 3D spatial context, easily scale to multi-terabyte datasets, and ideally should not require a manual segmentation of cells for training, due to the time required to generate per-pixel ground truth. Of methods that fulfill these requirements, Tracking with Gaussian Mixture Models (TGMM)  (Amat et al.
, 2014)  has been shown to work well on model organisms with approximately ellipsoid nuclei. More recently, the ELEPHANT tracking platform employed deep learning for cell detection and per-frame linking in light-sheet datasets with diverse cell appearance and movement  (Sugawara et al. , 2021). ELEPHANT requires a manual pseudo-segmentation of nuclei by ellipsoid fitting, which takes less time to generate than a per-pixel manual segmentation, but more than point annotations. Our method combines global optimization and learned features, generating cell lineages through global graph optimization with learned costs. We show that this combination substantially decreases tracking error on three diverse datasets of different model organisms with different temporal resolution, signal to noise ratio, and nuclear appearance.
Features are learned from sparse point annotations produced by current manual lineage tracking tools like MaMuT and Mastodon, and thus do not require a manual segmentation or dense lineage annotations, which allows rapid generation of training data. Crucially, the steps of our method-including the global optimization-can be computed in a distributed fashion, which is necessary to process multiterabyte light-sheet datasets and enable the study of whole embryo morphogenesis. An overview of our method is shown in Fig. 1. Because we are learning features from the data, the method is not tied to a specific input type or format: we use fused and unfused lightsheet recordings with a single fluorescent nuclear channel, and could easily extend to multi-channel input. We use sparse point annotations to train a convolutional neural network to predict at each pixel a cell indicator value that peaks at the center of each nucleus  (Höfener et al.
Thus, we introduce hard constraints to prevent merging and divisions producing more than two progeny. The objective function incorporates prior knowledge that cell movement is much more common than division, death, and entering or leaving the field of view, encouraging long, continuous lineages by penalizing the start and end of tracks. These tree constraints and continuity costs are similar to those in previous work  (Schiegg et al. , 2013; Jug et al. , 2016; Kok et al. , 2020) ; however, we also incorporate the node and edge scores generated by the neural networks into the objective function as learned costs. Thus, we optimize for valid lineages that are both continuous and supported by the learned cell location and movement features. Our Integer Linear Program (ILP) formulation of the optimization problem additionally allows solving piece-wise in parallel on large datasets by introducing additional constraints to ensure consistent solutions between adjacent regions.
We evaluate our method on three sparsely annotated datasets from different commonly used model organisms to study embryogenesis: mouse  (McDole et al. , 2018 ), Drosophila (Amat et al. , 2014) , and zebrafish  (Wan et al. , 2019b ) (see Supplementary Note 1 for details about the datasets and annotations). We compare the performance of our method against TGMM, the previous state-of-the-art method on these datasets  (Amat et al. , 2014; McDole et al. , 2018) , and greedy tracking using a per-frame nearest neighbor linking algorithm similar to the ELEPHANT tracking method  (Sugawara et al. , 2021). We compute multiple metrics, including the fraction of perfectly constructed lineages over a range of time periods, and errors per ground truth edge, broken into the following error types: false negative edges (FN), identity switches(IS)-when two tracks switch off following the same cell-false positive divisions (FP-D), and false negative divisions (FN-D), as illustrated in Fig.
3. False positive edges cannot be computed using sparse ground truth, because we cannot tell if unmatched reconstructions are false positives or tracking unannotated cells, and thus they are not included in our quantitative analysis. We show in Fig. 2  that, with around 20 hours of ground-truth annotation effort, our method correctly reconstructs more cell lineages than both baselines over all time ranges for all datasets. The largest improvement compared to TGMM is on the mouse dataset: our method correctly tracks 75. 8% of mouse cells over a time span of 1 hour (12 time frames), compared to 31. 8% for TGMM. By 175 minutes (35 frames), our method still correctly tracks more than half of all cells, while TGMM tracks less than 8%. On all three datasets, our method greatly reduces false negative edges compared to TGMM, while compared to the greedy baseline, our method produces far fewer false positive divisions.
Supplementary Note 2 contains a detailed description of the evaluation metrics and baselines, and further observations about the performance on various metrics across organisms and evaluation regions. Both the candidate graph generation and lineage optimization steps of our method are fully parallelizable and scale linearly with the size of the recording, which enables dense lineage reconstruction on very large datasets in reasonable time. On 20 GPUs and 100 CPU cores, reconstruction of dense lineages took about 44 hours on the 4. 7TB mouse dataset, generating more than 7 million cell detections and 360,000 tracks over the 44 hour recording. Given the estimates that there are 6 million true cell detections in the dataset and that an annotator can click on a cell center every 1. 5 to 3 seconds, it would take 2500-5000 annotator-hours to manually trace all lineages in this dataset.
The source code of our method is publicly available, together with training and inference scripts and extensive documentation (https://github. com/funkelab/linajea). The ability to densely reconstruct cell lineages in such large, information-rich datasets opens up vast opportunities for exploring cell fate dynamics and tissue morphogenesis. Accurately following cells and their progeny over extended time periods allows identification of individual cell behaviors that are not visible with shorter and less accurate lineages. For example, being able to accurately track more than half of all cells over a time window of 175 minutes in the mouse dataset, compared to only 30 minutes with previous methods, greatly reduces the manual curation needed to test hypotheses such as the existence of neuromesodermal progenitors which can produce neural or mesodermal progeny.
M 2. 1 Network Architecture, Training, and Prediction To attain per-voxel predictions for cell locations and movements, we use a U-N architecture with four resolution levels  (Ronneberger et al. , 2015). To incorporate temporal as well as spatial context, we concurrently feed seven 3D frames centered on the target time point and use four-dimensional convolutions until, due to valid convolutions, the time dimension is reduced to one. While we mostly downsample by (2,2,2), our mouse and Drosophila datasets are anisotropic, for the first pass we only downsample by 1 in z. We use 12 initial feature maps and increase by a factor of 3 at each level. When upsampling, we restrict our upsampling convolutional kernels to constant values, as we have observed this reduces artifacts in the output. The cell indicator network is trained on sparse point annotations and predicts the centers of cell nuclei.
The NMS window size is dataset dependent and should be a bit smaller than the minimal distance between two cell centers. To reduce the number of false positives, especially in background regions, we only consider detections with a minimal cell indicator value, determined empirically for each dataset and model. Additionally, if a foreground mask is available (as in the zebrafish dataset) we filter detections to those that lie in the foreground. In addition to the cell indicator network, we train a movement vector network to predict the movement of cells between frames. For a pixel near to a cell in frame t, the movement vector is a 3D offset vector that points to the relative location of the center of the same cell in frame t -1. Predicting the offset to the same cell in the previous time frame, rather than the next time frame, allows divisions to be represented naturally, since each daughter cell points to the center of the parent cell.
We calculate the loss on two different masked regions. Loss L A is the mean squared error between the ground truth and predicted movement vectors, calculated over the same training mask as the cell indicator network. Loss L B limits the error to voxels with maximal cell indicator values after NMS that also are within the training mask. The total loss is the weighted sum L = αL A + (1α)L B , with α = 1 1+exp 0. 01(-i+200000)) , and i being the number of training iterations. This weighting scheme weights L A higher at the beginning of training, when the cell indicator network is still converging, with a smooth transition to L B at 200000 iterations. These networks are trained simultaneously for 400000 iterations, with batch size 1. Batches are randomly sampled from annotated locations, and random augmentations including elastic deformation, mirroring, transposing axes, and intensity augmentation are applied using the G library (https://github.
Let b be the union of b and the surrounding context area. A process reads all nodes and edges in b, solves the ILP, and writes the result for only the target region b into a central database. If the database already contains results in the context region, these selections will be introduced as further constraints into the ILP, ensuring the solution will be consistent across boundaries. At the block boundaries, we set the appear and disappear costs to zero, because we do not want to penalize solutions that cross block boundaries. The introduction of a context region introduces dependencies between neighboring blocks, and thus they cannot be run trivially in parallel. By ensuring that overlapping blocks are never run simultaneously using the D library (https://github. com/ funkelab/daisy), we ensure a valid, consistent global solution while retaining a high degree of parallel processing.
S N : D A - We test our cell tracking method on time-lapse light-sheet recordings from three common model organisms: Drosophila, mouse, and zebrafish. We call these datasets D , M , and ZF , respectively, and summarize some relevant information about them in Table  1. Each dataset records a fluorescent nuclear marker: for ease of discussion, we will refer to each nucleus as corresponding to a single cell. While two of the datasets, D and M , have a single view of the organism, the ZF contains two orthogonal, registered but unfused views. To enable treating these views as interchangeable inputs to our networks, we resample them to isotropic resolution. We use sparse point annotations to train our method. As we leverage annotations originally performed for biological analysis, the annotated lineages are not randomly distributed, instead focusing on the developing nervous system of each organism.
Although not necessary for training purposes, annotators ensured that lineages were fully traced by following a cell and all subsequent progeny until they were no longer visible. Thus, there are more annotations in later frames of each recording. See Table  1  for the number of cells and divisions annotated in each dataset. For each organism, we divide the available annotations by time, location, and lineage into train, validation, and test sections, and report results on each split of the data individually, as well as averaged across splits as a form of k-fold cross validation. Cells in Drosophila and zebrafish rarely cross the center line of the organism, so we split the lineages into two groups based on side, discarding a small number of zebrafish lineages that did cross the center line.