dataesr/scientific-paragraphs-categorization huggingface dataset:
Aortic Ring Assay The aortic ring assay was performed as described  [60]. Aorta from P10 wild type and Pparβ/δ +/- mice were cut into 1mm rings before being embedded in a 96-well plate coated with rat tail collagen I gel (BD Biosciences, USA). Explants were treated with 50ng/mL VEGF and the media changed every other day. Explants from P3 wild type aorta were treated with 100nM of 10h or DMSO vehicle control. At day 10 of culture, the explants were fixed in 4% PFA and stained with Griffonia Simplicifolia isolectin B4 (IB4) (Vector Lab, Burlingame, CA, USA). Vessel outgrowth was imaged using Eclipse Ti-E Inverted Research Microscope (Nikon, Tokyo, Japan). The IB4 + areas were analyzed using ImageJ software (National Institutes of Health, Bethesda, MD, USA).
Statistical Analysis All data are shown as mean ± SEM. Four independent repeats were carried out for each experiment unless stated otherwise. One-way ANOVA followed by Turkey's post hoc analysis or unpaired two-tailed Student's t-test was used to determine the statistical significance using GraphPad Prism 6 (GraphPad Software, San Diego, CA, USA). * p < 0. 05; ** p < 0. 01; *** p < 0. 001; **** p < 0. 0001.
e. core and clamp loader complex) were reconstituted as described previously  (26). The concentration and purity of proteins was estimated by resolving the protein of interest and BSA of known concentration by SDS-PAGE, followed by comparative densitometric analysis with the use of ImageJ or ImageLab softwares. The protein concentration was additionally verified by NanoDrop measurements at 280 nm. The proteins used in this study were highly pure (95% purity or higher). The molar concentration of proteins is shown for monomers. Immunodetection assays were performed with the use of polyclonal anti-DnaA antibodies (rabbit) and polyclonal anti-Lon antibodies (mouse), goat anti-mouse and anti-rabbit conjugated with Horseradish peroxide (HRP) secondary antibodies (BioRad). PolyP (biotinylated and non-biotinylated), synthetic long chain of ∼700 phosphate residues long (linear as stated by the producer), were purchased from Kerafast; Cardiolipin, Fluphenazine and creatine phosphate were from Sigma-Aldrich.
ACKNOWLEDGEMENTS We thank  Dr Matthias Bochtler  for fruitful discussion. We also thank  Dr Monika Maciag-Dorszynska  for technical advisory with in vivo DNA replication measurements and  Dr Magdalena Rajewska  for constructing poriC plasmid. We are grateful to  Drs Andrea Lipinska  and  Malgorzata Nucleic Acids Research ,  2020 ,  Vol. 48 , No. 10 5465 Graul for their help with performing flow cytometry and advisory on data analysis. We are grateful to  Dr Malgorzata Ropelewska  for insightful comments. We thank  Dr Kirsten Skarstad  for sharing E. coli MG1655 dnaC2 strain and  Dr Michael Gray  for sharing E. coli  MG1655  and  MG1655  ppk strains.
We present an unsupervised deep learning approach for post-disaster building damage detection that can transfer to different typologies of damage or geographical locations. Previous advances in this direction were limited by insufficient qualitative training data. We propose to use a state-of-the-art Anomaly Detecting Generative Adversarial Network (ADGAN) because it only requires pre-event imagery of buildings in their undamaged state. This approach aids the post-disaster response phase because the model can be developed in the pre-event phase and rapidly deployed in the post-event phase. We used the xBD dataset, containing pre-and post-event satellite imagery of several disaster-types, and a custom made Unmanned Aerial Vehicle (UAV) dataset, containing post-earthquake imagery. Results showed that models trained on UAV-imagery were capable of detecting earthquake-induced damage.
The best performing model for European locations obtained a recall, precision and F1-score of 0. 59, 0. 97 and 0. 74, respectively. Models trained on satellite imagery were capable of detecting damage on the condition that the training dataset was void of vegetation and shadows. In this manner, the best performing model for (wild)fire events yielded a recall, precision and F1-score of 0. 78, 0. 99 and 0. 87, respectively. Compared to other supervised and/or multi-epoch approaches, our results are encouraging. Moreover, in addition to image classifications, we show how contextual information can be used to create detailed damage maps without the need of a dedicated multi-task deep learning framework. Finally, we formulate practical guidelines to apply this single-epoch and unsupervised method to real-world applications.
Various researchers focused on utilizing pre-and post-event imagery to its best advantage. Reference  [29]  showed how fusion of multi-temporal features improved damage localization and classification. Similarly, reference  [30]  aimed to detect different building damage degrees by evaluating the use of popular CNNs and multi-temporal satellite imagery. Reference  [11]  proposed an efficient method to update building databases by using pre-disaster satellite imagery and building footprints to train a CNN, which was fine-tuned using post-disaster imagery. Reference  [31]  proposed a U-Net-based segmentation model to segment roads and buildings from pre-and post-disaster satellite imagery, specifically to update road networks. Progress has also been made towards real-time damage detection. Reference  [32]  made use of a lightweight CNN that was placed on board an UAV to detect forest fires in semi-real time.
Reference  [7]  developed a similar near-real time low-cost UAV-based system which was able to stream building damage to end-users on the ground. Their approach was one of the first to validate such a system in large-scale projects. Finally, reference  [14]  showed how adding 3D information to UAV imagery aided the detection of minor damage on building facades from oblique UAV imagery. Most deep learning methods towards post-disaster damage mapping, including the ones mentioned above, are supervised. However, a persistent issue in supervised learning is the lack of labelled training data  [4]. The issue of unbalanced datasets or the lack of qualitative datasets is mentioned by most  [2, 12, [28] [29] [30]. As mentioned earlier, researchers bypass this issue by using training datasets from other projects that resembles the data that are needed for the task-at-hand, or by applying transfer learning to boost performance.
Generative Adversarial Networks Generative Adversarial Networks (GANs) were developed by reference  [33]  and gained popularity due to their applicability in a variety of fields. Applications include augmented reality, data generation and data augmentation  [34] [35] [36]. A comprehensive review of research towards GANs from recent years can be found in reference  [37]. A GAN consists of two Convolutional Neural Nets (CNNs): the Generator and the Discriminator. The Generator receives as input an image dataset with data distribution p data. The Generator aims to produce a new image ( x) that fits within the distribution p data. Therefore, the Generator aims to learn a distribution of p g that approaches p data. The Discriminator receives as input an image (x) from the original dataset as well as the image ( x) generated by the Generator.
ADGANs are only trained using normal, non-damaged input data. The resulting trained model is proficient in reproducing images that do not show damage, and less proficient in reproducing images that depict damage. Therefore, the distance between the input image and the generated image is large when inference is done using an image that contains damage, which subsequently can be used to produce anomaly scores  [24]. The first examples of ADGANs are Efficient GAN-Based Anomaly Detection (EGBAD), which was developed using curated datasets such as MNIST, and AnoGAN, which was geared towards anomaly detection in medical imagery  [27, 42]. Reference  [26]  applied an EGBAD-based method to detect malign masses in mammograms. The main limitation in AnoGAN was its low inference speed. This was resolved in f-AnoGAN  [43].
Details on how these parameters are tuned can be found in section 3. 4. A modification was made to the network. In the original network, after each epoch of training, the Area Under the Curve (AUC) score was calculated using the validation dataset. After training finished, a model for inference was selected based on the epoch in which it obtained the highest AUC score  [24]. This makes the original implementation not a truly unsupervised approach, since a validation dataset is still required (i. e. , examples of damage are still needed). Therefore, we choose to save the best performing model when the lowest Generator loss was found. This ensures that the model is chosen that is best able to generate fake images, which is the main principle of Skip-GANomaly. We verified that this approach yielded performance comparable to the original implementation, without the need of annotated test samples.
Details on how these parameters are tuned can be found in Section 3. 4. A modification was made to the network. In the original network, after each epoch of training, the Area Under the Curve (AUC) score was calculated using the validation dataset. After training finished, a model for inference was selected based on the epoch in which it obtained the highest AUC score  [24]. This makes the original implementation not a truly unsupervised approach, since a validation dataset is still required (i. e. , examples of damage are still needed). Therefore, we choose to save the best performing model when the lowest Generator loss was found. This ensures that the model is chosen that is best able to generate fake images, which is the main principle of Skip-GANomaly. We verified that this approach yielded performance comparable to the original implementation, without the need of annotated test samples.
Ideally, a model with a high descriptive value should result in non-overlapping distributions of the normal and abnormal samples with a clear threshold. Finally, alterations and additions were applied to Skip-GANomaly in an attempt to boost results for the satellite imagery dataset. First, with the idea of weighing the generation of building pixels more than other pixels, we attempted to direct the attention of Skip-GANomaly by adding building masks as input in an approach similar to the one described in  [48]. Furthermore, with the idea of utilizing the building information in the multiple epochs, similar to the approach described in  [16] , we stacked pre-and post-imagery into a 6-channel image and implemented an early, late or full feature fusion approach. These additions only provided marginal improvements.
xBD Dataset We made use of the xBD satellite imagery dataset  [49]. It was created with the aim of aiding the development of post-disaster damage and change detection models. It consists of 162. 787 pre-and post-event RGB satellite images from a variety of disaster events around the globe. These include floods, (wild)fire, hurricane, earthquake, volcano and tsunami. The resolution of the images is 1024 × 1024, the GSD ranges from 1. 25 m to 3. 25 m and annotated building polygons were included. The original annotations contained both quantitative and qualitative labels: 0-no damage, 1-minor damage, 2-major damage and 3-destroyed  [50]. The annotation and quality control process is described in  [50]. The dataset contained neither structural building information nor disaster metrics such as flood levels or peak ground acceleration (PGA).
UAV Dataset The UAV dataset was constructed manually from several datasets that depict the aftermath of several earthquake events. Examples can be found in Figure  3k-o. The UAV images were collected for different purposes and, therefore, the image resolution and the GSD vary and range around 6000 × 4000 pixels and from 0. 02 to 0. 06 m, respectively  [13]. Moreover, the camera angle differed between nadir and oblique view. The UAV dataset contained no pre-event imagery and, therefore, the undamaged patches were obtained from undamaged sections in the images (see Section 3. 3). Finally, the dataset contained neither structural building information nor PGA values.
Data Pre-Processing and Selection Before the experiments were executed, the datasets were first treated to create different data-subsets. This section describes the different data treatments, while the next section describes how they were used in different experiments. The data treatments can be summarized into three categories: (i) varying patch size, (ii) removal of vegetation and shadows, and (iii) selection of data based on location or disaster type. For each dataset, we experimented with different cropping sizes. The rationale behind this step was that the larger the image, the more area is covered. Therefore, especially in satellite imagery where multiple objects are present, larger images often contain a high visual variety. As explained earlier, the Generator attempts to learn the image distribution, which is directly influenced by the visual variety contained in the images.
Open source repositories such as OpenStreetMap provide costless building footprints for an increasing number of regions, and supervised or unsupervised deep learning are proficient in extracting building footprints from satellite imagery  [52] [53] [54]. Therefore, the proposed cropping strategy and subsequent training can be completely unsupervised and automated. and rooftops. Since no pre-event imagery was available, undamaged patches were obtained by extracting image patches from regions where no damage was visible. Binary labels were assigned to each image: 0-undamaged, or 1-damaged. The cropping strategy for the smaller sizes consisted of simply cropping around the center pixel (Figure  5 ). Next, the cropped patches were pre-processed. In order to investigate how sensitive this method is against different pre-processing, images were removed from the dataset based on the presence of vegetation or shadows.
Therefore, we created data subsets for training following the same procedure, using the Shadow Index (SI; Equation (  6 )) and the Green-Red Vegetation Index (GRVI; Equation (  7 ))  [57, 58]. Images containing more than 75 or 10 percent vegetation and/or shadows, respectively, were removed from the original dataset. Using these datasets, we showed how increasingly stricter preprocessing, and thus decreasingly visually complex patches, influences performance. Removing images from a dataset is not ideal since it limits the practicality of the proposed methodology because it reduces the proportion of patches on which it can do inference (see Figure  6 ). The test dataset in the novegshad@10% data subset is 8 percent of the original test dataset. Therefore, we further experimented with masking the pixels that contain vegetation and shadow in an attention-based approach, as explained in Section 3.
1. However, this was not considered as a further line of investigation since results did not improve. The UAV images were cropped in sizes of 512 × 512, 256 × 256 and 64 × 64. Larger patch sizes were chosen than for the xBD dataset to compensate for the difference in image resolution. More detail could be observed in larger sized UAV patches. Compare, for example, the amount of detail that can be observed in the smallest patches of Figures  4  and 5. Unlike for the xBD dataset, building footprints were not available. In general, building footprints for UAV imagery are difficult to obtain from open sources because, compared with satellite imagery, they are not necessarily georeferenced. Moreover, footprints would be difficult to visualize because of the varying perspectives and orientation of buildings in UAV imagery.
Next, the cropped patches were pre-processed. In order to investigate how sensitive this method is against different pre-processing, images were removed from the dataset based on the presence of vegetation or shadows. Vegetation and shadows remain challenging in deep learning-based remote sensing applications. Shadows obscure objects of interest, but also introduce strong variation in illumination  [55]. Depending on the varying lighting conditions, vegetation is prone to produce shadows and, therefore, varying Red, Green, Blue and illumination values  [56]. Therefore, the image distribution learned by the Generator is expected to be broad. This means that any damage found on buildings is more likely to fall within this learned image distribution and, therefore, to be well reconstructed in the fake image.
A well-reconstructed damage leads to lower anomaly scores, which is not the objective. We showed in  [23]  how removing these visually complex patches from the training set improve damage classification because the learned image distribution was expected to be narrower. Therefore, we created data subsets for training following the same procedure, using the Shadow Index (SI; Equation (  6 )) and the Green-Red Vegetation Index (GRVI; Equation (  7 ))  [57, 58]. Images containing more than 75 or 10 percent vegetation and/or shadows, respectively, were removed from the original dataset. Using these datasets, we showed how increasingly stricter preprocessing, and thus decreasingly visually complex patches, influences performance. Removing images from a dataset is not ideal since it limits the practicality of the proposed methodology because Next, the cropped patches were pre-processed.
In order to investigate how sensitive this method is against different pre-processing, images were removed from the dataset based on the presence of vegetation or shadows. Vegetation and shadows remain challenging in deep learning-based remote sensing applications. Shadows obscure objects of interest, but also introduce strong variation in illumination  [55]. Depending on the varying lighting conditions, vegetation is prone to produce shadows and, therefore, varying Red, Green, Blue and illumination values  [56]. Therefore, the image distribution learned by the Generator is expected to be broad. This means that any damage found on buildings is more likely to fall within this learned image distribution and, therefore, to be well reconstructed in the fake image. A well-reconstructed damage leads to lower anomaly scores, which is not the objective.
We showed in  [23]  how removing these visually complex patches from the training set improve damage classification because the learned image distribution was expected to be narrower. Therefore, we created data subsets for training following the same procedure, using the Shadow Index (SI; Equation (  6 )) and the Green-Red Vegetation Index (GRVI; Equation (  7 ))  [57, 58]. Images containing more than 75 or 10 percent vegetation and/or shadows, respectively, were removed from the original dataset. Using these datasets, we showed how increasingly stricter pre-processing, and thus decreasingly visually complex patches, influences performance. Removing images from a dataset is not ideal since it limits the practicality of the proposed methodology because it reduces the proportion of patches on which it can do inference (see Figure  6 ).
The test dataset in the novegshad@10% data subset is 8 percent of the original test dataset. Therefore, we further experimented with masking the pixels that contain vegetation and shadow in an attention-based approach, as explained in Section 3. 1. However, this was not considered as a further line of investigation since results did not improve. Each data subset was divided into a train and test set. Figure  6  shows the sample size of each subset. The train-set only consisted of undamaged images, and the test set contained both undamaged and damaged samples. For the satellite imagery, the undamaged samples in the train set came from the pre-event imagery, whereas the undamaged samples in the test set came from the post-event imagery. For the UAV imagery, the undamaged samples both came from the post-event imagery.
The samples were divided over the train and test set in an 80 and 20 percent split. The original baseline dataset denotes the complete UAV or complete satellite dataset. SI = (256 -Blue) * (256 -Green) (6) GRVI = ρ green -ρ red ρ green + ρ red (7 We note that the UAV dataset size was relatively low. However, the authors of  [44]  found that the ability of an ADGAN to reproduce normal samples was still high when trained on a low amount of training samples. We verified that low number of samples had no influence on the ability of Skip-GANomaly to produce realistic output imagery, and thus we conclude that the Generator was able to learn the image distribution well, which was the main goal. For the reasons explained above, these numbers of UAV samples were deemed acceptable.
Here the assumption was made that buildings were more similar in appearance if located in the same continent or country. Trained models were expected to transfer well to other locations if the buildings looked similar. Additionally, satellite patch selections were made based on the disaster type in order to investigate whether buildings affected by the same disaster type could yield a high performance. Here we consider that end-users might already possess a database of pre-event imagery of the same disaster of different locations around the globe, while they are not in possession of pre-event imagery of the country or continent that appears similar to the location of interest. Table  1  shows a summary of all the resulting satellite and UAV data subsets. Each data subset was divided into a train and test set.
Before training, the model was tuned for the hyper-parameters, w adv , w con , w lat , learning rate and batch size, using grid-search. When the best set of hyper-parameters was found, the model was retrained from scratch using these parameter values for 30 epochs, after which it did not improve further. As explained earlier, using a modification, during training the best performing model was saved based on the lowest generator loss value. After training was completed, the model was evaluated on the test set. Training and evaluation ran on a desktop with a dual Intel Xeon Gold (3. 6GHz) 8-cores CPU and a Titan XP GPU (12GB). Training for 30 epochs took approximately 8 hours using patches of 64 × 64 and a batch size of 64. Inference on a single image of 64 × 64 took approximately 3. 9 ms. For part two of the experiments, the transferability was analyzed by testing each of the previously trained models on the test set of all other data subsets.
Comparison Against State-of-the-Art In order to investigate how close our results can get to those of supervised methods, we compared the results of our experiments against results obtained using supervised deep learning approaches. In earlier work, we showed how unsupervised methods drastically underperformed compared to our method and, therefore, unsupervised methods such as One Class Support Vector Machine are left out of the comparison  [23]. In order to make a fair comparison, we considered approaches that made use of a single epoch and, ideally, datasets that resemble ours in GSD, resolution, location and disaster-types. Therefore, we compared the results obtained using satellite-based models against the xView2 baseline model, and ranked competitors in the xView2 competition  [29]. The xView2 baseline model first trained a U-Net architecture to extract building polygons.
Afterwards, they used a Resnet50 architecture pre-trained on ImageNet to classify different degrees of building classifications. The ranked contenders  [29]  used a multi-temporal approach where both localization and classification was learned simultaneously by feeding the pre-and post-disaster images into two architectures with shared weights. The architectures consisted of ResNet50, which was topped with Feature Pyramid Heads, and were pre-trained on ImageNet. Finally, we compared the results obtained using UAV-based models with results obtained by  [13]. Here, comparable UAV-images were used from post-earthquake scenes to train an adapted DenseNet121 network with and without fine-tuning. The authors carried out several cross-validation tests where each time a different dataset was used for testing, to investigate the influence of training data on performance.