dataesr/scientific-paragraphs-categorization huggingface dataset:
More recently,  Colin et al. (2018)  developed the software CombCount to assess capped brood and capped honey. Although the software is able to detect both classes, it requires a user to distinguish the contents using selection tools. To the best of our knowledge, none of the studies published so far has employed Neural Networks such as CNNs in honey bee comb image analysis. Furthermore, most of the available methods are limited to detecting capped brood, with only two of them being capable of assessing food resources  (Colin et al. , 2018; Höferlin et al. , 2013). As such, there is an excellent opportunity for innovation using new methods to address a major challenge in honey bee research, which is assessing brood and food resources in the hive in a time-and cost-effective manner and with a high degree of accuracy.
IMAGES CAPTURE AND ANALYSIS To assure image capture standardization, we developed a wooden tunnel sealed for external light and with optimized dimensions (Fig. 2 ). This tunnel had a retractable architecture for easy transportation, having a length of 247cm, when fully opened, and 92cm, when retracted. The comb frame and the camera were placed at the opposite sides of the tunnel. The comb frame was positioned in two holders (Fig. 2b ). The holders had an angle of 11° (to make up for to the 9-13° natural inclination of comb cells) for a better image capturing of the interior of the comb cells (see the 3D model file in https://github. com/AvsThiago/DeepBee-source). Close to the comb frame (40cm from the top), there was pair of Light-Emitting Diode (LED) sources, with 7 Watts of power. The LEDs were turned to the walls at 45° to provide homogeneous light conditions and to avoid shadows during image acquisition.
The camera was fixed with a screw on the opposite side of the tunnel (Fig. 2c ). In this study we used a digital camera Nikon D3300, with lens AF-S DX VR Zoom-Nikon ED 55-200mm F4-5. 6G, and the following settings: aperture -10; ISO -100; shutter speed -1/60; autofocus -on; flash -no; compression -JPEG; white balance -on. During image capturing, the tunnel was closed on both sides and the camera was activated by an external trigger. The images captured had a resolution of 24MPixels (6000x4000px). Using these settings, 1,102 comb frames were photographed on both sides making a total of 2,204 images. The image dataset is available online at https://cloud. ipb. pt/d/aa29c989ab1944aaa222/?p=/DS-COMB-PT.
3. 1 Circle Hough Transform  Duda and Hart (1972)  developed the classical Hough Transform method currently available. This method was originally developed for detecting lines in images. Later on, it was discovered that it could also be used for identifying arbitrary shapes, such as circles and ellipses, even when they were partially occluded. The Circle Hough Transform (CHT) method uses a voting process to calculate the probability that a set of pixels form a circle. There are several implementations of the method. Herein, we used the implementation contained in the OpenCV v. 4. 0 library (https://github. com/opencv/opencv/releases/tag/4. 0. 0). The parameters expected by this method are: a grayscale image, size of an internal accumulator that will store intermediate results, minimum distance between two detections centre, threshold to be applied to the internal Canny operator, number of votes that a circle must have in the accumulator to be set as true, minimum circle radius, and maximum circle radius.
Dataset creation for comb segmentation We created the annotations using the Quick Selection Tool from the software Adobe Photoshop ® CS6. To define the classes, we painted white the comb area and black the remaining area. We labelled 61 comb images  (Alves et al. , 2019) , which were selected to represent a high diversity of cell content (e. g. honey, pollen, capped and uncapped brood) and age (the older the comb the darker it gets). Fig. 5  illustrates annotations made on those images. The annotations were split in three sets: training (85%); validation (10%), and testing (5%). Using the strategy proposed in  Ronneberger et al. (2015) , we divided the input images and the labels in tiles, as shown in.
Semantic segmentation architecture and training policy The architecture had a depth of 5 convolutions with 3×3 filters and layers of maximum pooling with 2×2 filters and a stride of 2, as proposed by  Ronneberger et al. (2015). Our modifications to the original model, made to obtain the best results in our semantic segmentation dataset, were tuned using a trial-error approach and the following settings: input image with 128×128 resolution, dropout  (Srivastava et al. , 2014)  ranging from 0. 1 to 0. 3 between the convolution layers, use of Exponential Linear Units (ELU) activation function  (Clevert et al. , 2015) , and use of 16 filters (channels) in the first layer, doubling the amount at each inner level and returning 16 filters in the penultimate layer with the last layer only having two dimensions.
Transfer learning for cells classification Using the transfer learning technique, it is possible to transfer the weights of feature extraction layers (e. g. convolutions) from a trained model over a dataset to another model that will be trained in a new dataset  (Oquab et al. , 2014). Because the new model received kernels already trained to recognise generic features, like lines and curves, it will be easier for the model to generalise a new dataset being unnecessary to learn the filters from scratch. We made a sanity check and trained an architecture in our dataset with and without the transfer learning before applying this approach to the next experiments. In this experiment, we used the same policy in both trainings. We transferred the weights from a pre-trained model on the ImageNet dataset  (Deng et al.
Finding the best region of interest (ROI) size to crop the cells Before we defined the CNN architecture for the classifier, we needed to find at which window size around the cells the image should be cropped. If we had cropped only the interior of the cells, our classifier could have had difficulty in distinguishing between capped brood and honey cells, as these classes typically exhibit a similar texture (Fig. 11 ). To select the best input size, we created and tested 16 datasets, all of them with the same annotations but with different ROI size. We defined each dataset with 10% of the annotations from the main dataset. The sizes (pixels) of the squared crops were  40, 50, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 280, 298, 400  and 500. The training framework for the tests was the Keras version 2.
2 and the InceptionV3 architecture  (Szegedy et al. , 2015). For the feature extraction layers, we used weights pre-trained on the ImageNet dataset using the transfer learning technique. The architecture and trained weights were provided by Keras library. This architecture does not allow inputs lower than 139×139px. Image datasets lower than 139×139px were resized to the minimum input size. We added three layers at the  The classifiers were compiled using the Categorical Cross Entropy loss function. We chose Adam with default parameters β1 = 0. 9 and β2 = 0. 999 as the optimiser. The training started with the LR at 10 -3 and using the technique Reduce Learning Rate on Plateau  (He et al. , 2015a). We defined that the LR would be halved after 3 epochs without improvement in the Loss metric, being the minimum value 10 -6.
Data augmentation It is not always possible to obtain large datasets for training CNNs, either due to difficulties in gathering images with the object or in affording human resources to annotate the datasets. One way to enlarge the working dataset is through the Data Augmentation (DA) technique  (Krizhevsky et al. , 2012)  Based on the models with the best metrics discovered in the experiment described in Section 4. 3, we made a new training with additional data. We generated the new images using DA. Flips, brightness changes, rotations, shift, and zoom were applied in the newly created images. As a result, we generated a dataset of 250,000 images evenly spread across the 7 different classes. These images were used in the training set. Validation and test sets were kept with the original images.
Finding the optimal input image size After training 16 different classification models, we obtained a plot relating ROI size with cell classification accuracy (Fig  17 ). The trending line shows a steady increase in the accuracy as the ROI size increases up to 300px; after that point the quality decreases. Accordingly, input images with sizes between 200px and 300px should be preferred as they tend to produce better results. We chose 224×224px as the default input size for all the following tests. We based this choice on the computational cost. Additionally, although the best result in the test set was the one trained with 220×220px images, it was not possible to use this input size because architectures like MobileNetV2 only had pre-trained weights in the ImageNet dataset for sizes 128, 160, 192, or 224px (https://bit.
Comparing different CNN architectures During the training of different architectures with the 224×224px input size, we faced some difficulties. We noticed that models VGG 16 and 19 were unable to converge over the used dataset (10% of our original dataset), even after trying different LRs, loss functions and optimisers. Therefore, we decided to remove these two models from the tests. Another model that caused problems during the training stage was NasNet (Large). This model suffers from a known bug (see https://github. com/keras-team/keras/issues/8711#issuecomment-354585187) and there are bypasses for it, but we decided to discard it due to the large amount of time required for retraining. Table  3  presents some metrics for the computational performance of 11 models. The MobileNet model produced the best results regarding the number of epochs required for reaching convergence and the average time per epoch.
Data augmentation for cells classification We chose four models to be trained using data augmentation (DA) for the following reasons: InceptionResNetV2 and NasNetMobile for having the best F1score, MobileNet for having the lowest loss in the validation and test sets, and DenseNet201 for having the best accuracy in the validation and test sets. Fig. 20  presents the F1-score calculated for the selected models trained on the dataset without and with DA. Except for DenseNet201, all other models showed a higher F1score after training using DA. MobileNet outperformed more complex models, with an F1-score over 94%.
Comparison with methods from the literature Here we compared our method with those reported in the literature, although this endeavor may not always be fair for three main reasons: (i) we classified a wider array of cell types, (ii) we do not have the same dataset, and (iii) some works employed different classifiers. Cornelissen et al. (2009)  compared their semi-automatic method of counting capped brood cells in comb images with the Liebefeld method. While annotations with the Liebefeld method took 26s per frame, the semi-automatic approach took 19s for image capturing plus 30s for image processing. This semi-automatic method consists of manual segmentation of the capped brood area followed by automatic count of cell number. Cornelissen et al. (2009)  reported a correlation of 99. 37% between the actual and the predicted number of cells, which was substantially higher than the 90.
47% and 99. 41%, respectively. Wang and Brewer (2013) reported a 97. 4% hit rate with the HoneybeeComplete commercial software developed for counting capped brood cells. This value increased to 99. 5% when the search area was delineated by the user in the comb image. Herein, the MobileNet DA architecture produced a value for the capped brood class very close (99. 47%) to that obtained by the HoneybeeComplete software, but without human assistance. The commercial software HiveAnalyzer  (Höferlin et al. 2013) , which is able to classify detected comb cells into seven classes, achieved 94. 3% accuracy on cells that were classified with high confidence (78% of 20,000 analysed cells). In this study, we achieved 94. 31% accuracy (very close to the 94. 3% F1-Score value) using the MobileNet model DA on 100% of the test set (53,914 analysed cells).
Further analyses on datasets creation and cells classification The task of correctly classifying all the cells in a comb image is not trivial because of the wide range of colours, shapes, and textures of cell contents and wax types typically found in a hive. While working with the datasets, we realized that comb classification is further challenging due to the impact of some factors on the results quality. One such factor was related with cell contents. Cells with multiple contents (e. g. pollen and egg) or cells with contents in a transition stage, such as from larva to pupa (capped cell) or from egg to larva, revealed to be problematic (Fig. 23 ). The co-occurrence of different cell contents makes evaluation of the classifier less precise, as there may be cases where it hits one of the classes but the alternative class has been defined as the ground truth of the image.
This problem has been handled in competitions of image classification using Top-n accuracy  (Krizhevsky et al. , 2012). With this methodology, the model earns credit for correctly classifying the image in its Top N guesses. We evaluated our model using Top-2 accuracy, which corrected the cell content if the correct class was between the two more likely predictions. By reprocessing the test set with the Top-2 accuracy method, the quality of our results improved 5% on average (Fig. 24 ). To test this effect, we assessed the distribution of the annotations across comb regions. To that end, using the annotations of the main dataset, we generated a heatmap plot displaying the areas of the comb that were preferentially annotated. As shown in Fig. 26a , annotations were more concentrated in the upper left area of the comb, suggesting that models trained in this dataset would have a better classifying performance in that region.
Next, we predicted all cells that were homogeneously annotated in the test set and generated a new heatmap showing the location of most of the wrong predictions (Fig. 26b ). remove false detections that may occur on the background. Although we obtained a cell detection rate of 98. 7%, we believe that the false positive rate may decrease by training the semantic segmentation model with an input larger than 128×128px. After we trained over thirty CNN models with different training techniques, such as transfer learning and data augmentation, and comparing them using different perspectives, we recommend MobileNet. While InceptionResNetV2 showed the best results in our dataset, the time performance of MobileNet was superior, due to 93% fewer weights. Using MobileNet, we achieved 94. 3% of correctness with the metric F1-score weighted over the seven classes.
Following the methodology of Liew et al. (2010) , we selected 10 images from our dataset to be analysed by our detection algorithm. Subsequently, we evaluated manually the false positive (FP) and false negative (FN) detections. From these annotations, we collected the following metrics: number of cells identified by humans; number of cells detected by the algorithm; true positive (TP) detections; true cells detected correctly; FP detections on inexistent cells; and FN cells undetected by the algorithm. Finally, as in Liew et al. (2010) , we calculated for each image the cells detection rate using equation 3. This metric is based on the total number of cells automatically detected, excluding the falsely identified cells, divided by the manual count.
dataset created for classification should contain cells representing all different classes in different comb images. In this study, the annotations were made by an experienced beekeeper, assuring the high quality required for developing the models. A piece of software was developed to facilitate the beekeeper's work. The software allowed choosing a label corresponding to each class (nectar, honey, pollen, egg, larva, capped brood, and other) and, at the same time, pointing the centre of the cells, adjusting the contrast, brightness and gamma of the images. A total of 71,915 cells were annotated on 1,202 comb images, with an average of 25 cells per image. The number of annotations by class is shown in Fig. 9. We divided the annotations in three sets for the training as follows: 80% of the original dataset for training, 20% of the training set for validation, and 20% of the original dataset for testing.
selected by their superior performance in image classification competitions in the past years. These architectures included DenseNet 121, DenseNet 169, DenseNet201 (Huang et al. , 2016) , InceptionResNetV2 (Szegedy et al. , 2016) , InceptionV3 (Szegedy et al. , 2015) , MobileNet (Howard et al. , 2017) , MobileNetV2 (Sandler et al. , 2018) , NasNet; NasNetMobile (Zoph et al. , 2017) , ResNet50(He et     al. , 2015a), VGG 16, VGG 19 (Simonyan & Zisserman, 2014), and Xception. Each architecture was trained using all training and validation images from the core dataset. The images were extracted with the cell centralised 224x224px. Before starting training each model, we made some modification in the CNN architectures. We added a set of fully-connected layers, as shown in Fig. 12. Prior to processing by the model, the images were normalised by subtracting the ImageNet Mean Image(103.
Spike sorting is a crucial step in electrophysiological studies of neuronal activity. While many spike sorting packages are available, there is little consensus about which are most accurate under different experimental conditions. SpikeForest is an open-source and reproducible software suite that benchmarks the performance of automated spike sorting algorithms across an extensive, curated database of ground-truth electrophysiological recordings, displaying results interactively on a continuously-updating website. With contributions from eleven laboratories, our database currently comprises 650 recordings (1. 3 TB total size) with around 35,000 ground-truth units. These data include paired intracellular/extracellular recordings and state-of-the-art simulated recordings. Ten of the most popular spike sorting codes are wrapped in a Python package and evaluated on a compute cluster using an automated pipeline.
SpikeForest We have addressed the above needs by creating and deploying the SpikeForest software suite. Spi-keForest comprises a large database of electrophysiological recordings with ground truth (collected from the community), a parallel processing pipeline that benchmarks the performance of many automated spike sorters, and an interactive website that allows for in-depth exploration of the results. At present, the database includes hundreds of recordings, of the types specified above (paired and state-of-the-art biophysical simulation), contributed by eleven laboratories and containing more than 30,000 ground-truth units. Our pipeline runs the various sorters on the recordings, then finds, for each ground-truth unit, the sorted unit whose firing train is the best match, and finally computes metrics involving the numbers of correct, missing, and false positive spikes.
A set of accuracy evaluation metrics are then derived per ground-truth unit for each sorter. By averaging results from many units of a similar recording type, we provide high-level accuracy summaries for each sorter in various experimental settings. In order to understand the failure modes of each sorter, SpikeForest further provides various interactive plots. A central aim of this project is to maximize the transparency and reproducibility of the analyses. To this end, all data-the set of recordings, their ground-truth firings, and firing outputs from all sorters-are available for public download via our Python API. SpikeForest itself is open-source, as are the wrappers to all sorters, the Docker  (Merkel, 2014)  containers, and all of the parameter settings used in the current study results.
Results In conjunction with the SpikeInterface project  (Buccino et al. , 2019) , the SpikeForest Python package provides standardized wrappers for the following popular spike sorters: HerdingSpikes2  (Hilgen et al. , 2017) , IronClust  (Jun et al. , in preparation) , JRCLUST  (Jun et al. , 2017a) , KiloSort  (Pachitariu et al. , 2016) , KiloSort2  (Pachitariu et al. , 2019) , Klusta  (Rossant et al. , 2016) , Mountain-Sort4  (Chung et al. , 2017) , SpyKING CIRCUS  (Yger et al. , 2018) , Tridesclous  (Garcia and Pouzat, 2019) , and WaveClus  (Chaure et al. , 2018; Quiroga et al. , 2004). Details of each of these algorithms are provided in Table  1. Since each of these spike sorters operates within a unique computing environment, we utilize Docker  (Merkel, 2014)  and Singularity  (Kurtzer et al.
, 2017)  containers to rigorously encapsulate the versions and prerequisites for each algorithm, ensuring independent verifiability of results, and circumventing software library conflicts. The electrophysiology recordings (together with ground-truth information) registered in SpikeForest are organized into studies, and studies are then grouped into study sets. Table 1 details all study sets presently in the system. Recordings within a study set share a common origin (e. g. , laboratory) and type (e. g. , paired), whereas recordings within the same study are associated with very similar simulation parameters or experimental conditions. Each time the collection of spike sorting algorithms and ground-truth datasets are updated, our pipeline, depicted in Figure  1 , reruns the ten sorters on the recordings.
It is likely that modifications to sorting parameters could reduce the number of crashes, but attempts so far, including contacting the author, have not yet fixed this problem. In the synthetic datasets, KiloSort2 had the largest number of false positive units (distinct from the false positive rate of a single unit), but this is not currently reported by SpikeForest (see Discussion). and study (a simulated drift dataset from the SYNTH_JANELIA study set). The SNR threshold for the main table calculation is shown as a dashed line, and the user-selected unit is highlighted. Marker area is proportional to the number of events, and the color indicates the particular recording within the study. (right) A subset of spike waveforms (overlaid) corresponding to the selected ground truth unit, in four categories: ground truth, sorted, false negative, and false positive.
IronClust appears among the top average accuracies for eight of the study sets, and is especially strong for the simulated and drifting recordings. For most study sets, IronClust has improved accuracy over its predecessor JRCLUST, and is also improved in terms of speed and reliability (no crashes observed). Although a substantial portion of the development of the IronClust software took place while it had access to the SpikeForest ground truth datasets, the same sorting parameters are used across all studies, limiting the potential for overfitting (see Discussion). MountainSort4 is among the top performers for six of the study sets (based on the average accuracy table) and does particularly well for the low-channel-count datasets (monotrodes and tetrodes). It is not surprising that MountainSort4 is the top performer for MANUAL_FRANKLAB because that data source was used in development of the algorithm  (Chung et al.