dataesr/scientific-paragraphs-categorization huggingface dataset:
We obtain the results from PADIweb, ProMED and Empres-i by discretizing the time (resp. spatial) dimension into monthly intervals (resp. country zones) and by applying the ST method with the parameters ι = 12, = {0. 5, 1. 0} and α = 1000km. Recall that in our dataset the full periodicity, with = 1. 0, amounts to be the events occurring every year for the same time period from 2019 to 2021. Regarding the partial periodicity, with = 0. 5, a seasonal pattern is valid in our dataset, when the events occur during two consequent years between 2019 and 2021 for the same time period. In the comparison, we need to take into account the fact that the detection time for the same events can differ up to 30 days in average for PADI-web and ProMED with respect to the Empres-i events (see Figure  5 ). Therefore, it is reasonable to observe some time delay in the results.
Thematic Dimension In this section, we study how detailed EBS systems provide the thematic information encoded in their data. We want to know how similar the frequent multidimensional patterns across EBS systems are. We compare the results based on the Empres-i dataset to see to what extent PADI-web and ProMED can capture similar patterns. To ease our discussion, we also visualize the relations between spatial and thematic entities with a chord diagram in Figure  12 , which is found in the Appendix for space matters. Table  14  shows the most 13 frequent static and temporal multidimensional patterns for PADI-web, ProMED and Empres-i. We obtain these patterns with the parameters ι ∈ {10, 30, ∞} and = 1 (in count). Note that the use of ∞ represents a very large value for eliminating the periodicity aspect from the method ST.
Europe). In the last part, the results are produced from the whole dataset. performances of the news outlets for a fair comparison. First, we compare PADI-web and ProMED in terms of their important news outlets for information dissemination, obtained with the PageRank algorithm. Table  15  shows the first 10 (resp. 9) news outlets having the largest PageRank scores for PADI-web (resp. ProMED) based on the events occurring in Asia, Europe and in the whole world, respectively. We can summarize the results in two points. First, overall, the results show that the most important news outlets for both sources are almost completely different. PADI-web relies mostly on the Indian news outlets for the events in Asia and French and British/Scottish ones for Europe, whereas spatially more diverse news outlets are in ProMED's results, with a slight prevalence for the Russian (and nearby countries such as Ukraine and Kazakhstan) and British/Scottish news outlets.
Conclusion In this article, we have presented a new evaluation framework to identify the strengths and drawbacks of EBS systems in terms of epidemic surveillance. This evaluation is very valuable from the epidemiological standpoint, since it allows end-users to select the most appropriate EBS system(s) for an effective surveillance of a particular situation. We want not only to compare EBS systems, but also to produce results that the end user can easily interpret. For this purpose, we proposed a two-step framework based on our review of the literature. It first transforms the raw input event data into a set of normalized distinct events, then conducts a descriptive retrospective analysis of these events with four objectives: spatial, temporal, thematic and source analysis. We illustrated its relevance by applying it to an Avian Influenza dataset collected by PADI-web, ProMED and Empres-i.
We showed that our framework allows identifying the strengths and drawbacks of the considered EBS systems. For some of our evaluation aspects, our results confirm the findings already published in the literature. For others, the systematic nature of our approach uncovers new findings for the considered EBS systems. Our work could be extended in several ways. First, our method can be applied systematically to other EBS systems and other animal diseases, for the sake of completeness. This would give a better overview of the capabilities of the existing systems. Second, the source dimension can be better evaluated in Section 3. 6, if we can obtain an appropriate gold standard dataset. This would imply to conduct an extensive work like in  [76] , but tailored to Epidemic Intelligence. Third, our evaluation framework focuses only on a descriptive retrospective analysis.
B. 2 Event Normalization This task consists in normalizing the attributes of each event in an event database, assuming that the events are defined as in Section 3. 1. As mentioned before, a normalization task aims at transforming a raw text into one of well-defined taxonomy classes, which results in hierarchical information. We start with the normalization of spatial entities, also called geocoding. This task consists in assigning geographic coordinates to spatial entities. This can be a challenging task due to the ambiguity among place names  [51]. For instance, Avignon is a regional county municipality in Quebec/Canada, but also a city in France. In this work, we perform the geocoding task with the gazetteer GeoNames 11. For a given query of spatial entity, GeoNames outputs a ranked list of most appropriate geographic coordinates associated with the input text.
g. France for Le Monde  12  ), thanks to a publicly available dataset  [58]. The rationale behind this processing is according to pragmatic rules when one reads a news document of a given country  [34]. Often, a city name is explicitly mentioned without citing the associated country name. Nevertheless, if the news document is about another country, it is more likely to be mentioned so that the reader has fewer chances of misunderstanding. We extract spatial entities from the title using SpaCy  [27]  and a lookup table. This lookup table is used to locate the spatial entities which are not detected by SpaCy. It can be constructed with a gazetteer and extracts spatial entities with keyword search. We also extract nationalities using a second lookup table. Then, we finalize the geocoding task from the ranked list provided by GeoNames by selecting the first result whose associated country is the same as the one extracted from the title.
In terms of the temporal dimension D T , the temporal expressions are usually in a well-structured format. The only task we need to do is to correctly estimate this format  [65]. For instance, a date can be written in MM-DD-YYYY or DD-MM-YYYY. We do this estimation by checking the whole data. Finally, we normalize the temporal expressions according to the TIMEX3 annotation standard. On what regards the thematic dimensions D D and D H , we normalize these disease and host names against the NCBI Taxonomy database  [21] , using a manually composed table of species name synonyms. Moreover, we tailor some of these NCBI taxonomies to diseases of interest. This is because the NCBI database is supposed to be generic enough in order to be used for any application domain. Nevertheless, for a particular area of expertise (e.
C. 2 Quantitative Evaluation for Spatial Dimension In this section, we describe how we compute the spatio-temporal representativeness score of an event database E with respect to a gold standard database E R. The definition of the spatio-temporal representativeness requires fixing the spatial and temporal scales, denoted by l Z and l T , respectively. Depending on these scales, we first discretize the spatial and temporal dimensions over a set Z of geographic zones and a set T of time intervals. Then, for a particular disease, we say that an event database E of an EBS system represents well a specific geographic zone z for a given time interval t, if it finds at least one event in E R , which is defined in Equation  12 as 1 l Z ,z l T ,t (E) = 1, |σ {z} {t} (E l Z l T )| > 0 0, otherwise. ( 12 ) The calculation of the spatio-temporal representativeness scores is done with respect to a gold standard database E R.
C. 3. 1 Quantitative Evaluation for Timeliness In this section, we describe how to compute the timeliness score between an event database E and a gold standard database E R. First, we calculate in Equation  16 the temporal distance between an event e ∈ E R and its corresponding one e ∈ E as δ(e, e ) = t(e ) -t(e), t(e ) ≥ t(e) 0, t(e ) < t(e),  (16)  where the function t(e) gives the date of a given event e. Note that in Equation 16 when an EBS system finds an event before the corresponding report date in E R , we do not reward or penalize it, and put a score of 0. Then, we calculate the timeliness score for E with respect to E R as Ψ(E, E R ) = 1 |E| e∈E 1 -exp -δ(f (e,E R ),e) L , (17) where the variable L in Equation  17 indicates the maximum expected reporting delay, which controls the temporal similarity of two events.
C. 3. 2 Quantitative Evaluation for Periodicity In this section, we explain how we quantitatively evaluate the performance of an event database E of an EBS system in terms of its ability to detect the continuous and seasonal periodic patterns with respect to a gold standard database E R. In the following, we first evaluate an event database E for each pattern type separately, then we take an average of the obtained scores. We first start with the evaluation of E in terms of its ability to detect continuous full and partial periodic patterns with respect to a gold standard database E R. Let l Z and l T be spatial and temporal scales for E and E R. Moreover, let I and P be also two sets of parameter values for ι and in ST. On top of that, the parameter value α controls the spatial closeness between the spatial entities in ST.
( 19 ) Second, we propose to perform the evaluation of E in terms of its ability to detect seasonal full and partial periodic patterns with respect to a gold standard database E R. We propose to do it, similar to the spatio-temporal representativeness evaluation defined in Equation  14. To detect such a seasonal effect within the data, we need to apply the method ST to a subset of E, where the events occur only within a particular time interval over several years, and we repeat this process for all the subsets. For instance, a subset of data can represent all the events occurring in every January, whatever the year is, and we repeat similar selections for the other months. To do so, in Equation 20 we first need to define if a spatial pattern x is in the output of ST(σ t (E l Z l T ), ι, , α) for a given time interval t and the input parameters ι, , α, i.
C. 4 Quantitative evaluation for thematic dimension In this section, we explain how we evaluate the modified version E + of an event database E in terms of its ability to detect static and temporal multidimensional patterns with respect to the modified version E R + of a gold standard database. Let I and P be the parameter values for ST. Note that the set I of values allows handling both the static and temporal cases, as a very large inter-arrival time value for ι amounts to omit the partial periodicity condition in ST. Then, we propose to calculate the evaluation score Ω(E + , E R + , I, P, α), which corresponds to the average score of F , as defined in Section C. 1, over the combination of all the considered parameters. It is calculated in Equation  25 as.
C. 5 Quantitative evaluation for source dimension Despite the existence of several prominent works aiming at ranking a large number of news outlets in terms of various objectives (e. g. news quality, popularity  [76] ), to the best of our knowledge, none of them is specialized in Epidemic Intelligence. This means that, unlike the previous studied dimensions, we are not able to compare the obtained results with a gold standard database. For this reason, we propose to evaluate an EBS system for the source dimension from a different perspective. Instead of evaluating the quality of the results, we rather assess the consistency of the results obtained in Sections 3. 6. 1 and 3. 6. 2 one another. The rationale is that for an effective surveillance system, we expect news outlets playing a key role in epidemiological information dissemination (as detected in Section 3.
6. 1) to be those which are timely in event detection (as in Section 3. 6. 2). Otherwise, this inconsistency amounts to decrease the main strength of an EBS system, which is its timeliness. Concretely, given an event database E we consider the ranked list produced for a timely detection objective (Section 3. 6. 2) as a reference list, and compare it with that obtained in Section 3. 6. 1 by applying Equation  11 , as shown in Equation  26. F (E) = F (PageRank(G E , k), CELF(G E , k))) (26) Moreover, we also plot the score differences in Figure  11  to ease the comparison between PADI-web and ProMED. In this figure, it is colored in blue (resp. red) when ProMED (resp. PADI-web) gives better spatio-temporal representativeness score for a country and in yellow in case of non-zero equality.
Acknowledgments. This study was partially funded by  EU  grant  874850 MOOD  and is catalogued as MOOD063. The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of the European Commission. We thank the  ProMED-mail  staff for sharing with us their dataset, as well as for their constructive comments to improve the content of this paper. ProMED is a program of the  International Society for Infectious Diseases (ISID).
To characterize the growth of brain organoids (BOs), cultures that replicate some early physiological or pathological developments of the human brain are usually manually extracted. Due to their novelty, only small datasets of these images are available, but segmenting the organoid shape automatically with deep learning (DL) tools requires a larger number of images. Light U-Net segmentation architectures, which reduce the training time while increasing the sensitivity under small input datasets, have recently emerged. We further reduce the U-Net architecture and compare the proposed architecture (MU-Net) with U-Net and UNet-Mini on bright-field images of BOs using several data augmentation strategies. In each case, we perform leave-one-out cross-validation on 40 original and 40 synthesized images with an optimized adversarial autoencoder (AAE) or on 40 transformed images.
Ref. [10]  reduces only convolutional layers and uses dropout regularization  [10]. In U-Net++, the structure is lightened by employing densely nested strip pathways  [11]. The stripped-down U-Net (SD-UNet) is composed of depthwise separable convolutions with a weighted standardization to accelerate performance and decrease computing time. This model contains fewer parameters to learn, a smaller architecture and a shorter computation time than U-Net  [12]. In contrast to SD-Unet, SmaAT-UNet adds an attention module alongside use of depthwise separable convolutions  [13]. AFF-UNet uses skip connections with an attention channel and a spatial attention module  [14]. DMF-AU includes multiscale constraints on the backbone to reduce information loss and lighten the architecture for small datasets  [15].
This network employs a differential match filtering layer to identify structures and anisotropic attention to reinforce information on the most representative features. Others use cascaded networks to reduce the number of parameters to learn  [16]. In the CNL-UNet structure, a pretrained encoder is proposed to allow the segmentation of small datasets. By using the transfer learning properties of the encoder, the modified skip connections for reducing gaps between mirrored layers and the classifier and localizer module CNL, they efficiently learn the small amount of data  [17]. Another encoder architecture update consists of using transformers after convolutional layers to lighten the architecture, such as in LeViT-UNet  [18]. However, these procedures modify a simple aspect of the U-Net architecture, the encoder and decoder structure.
A recent U-Net variation is µ-Net, which reduces the vanishing gradient phenomenon and accelerates the model convergence with weight matrices in the decoder, which are the transposes of those in the encoder. However, this strategy must be improved for complex boundary segmentation on variable biological structures due to the architecture depth  [19]. Recently, a UNet-Mini architecture was created to segment small datasets. This architecture drastically reduces the number of parameters by updating only three elements  [20] , using element-wise skip connections (addition instead of concatenation) and reducing the number of layers and the kernel size. In this paper, we propose Mu-Net, a reduced U-Net architecture dedicated to the segmentation of images from small bright-field BO datasets. To validate our results, we compare our architectural proposition with the current U-Net and UNet-Mini architectures using shallow methods, such as active contour or k-means, and an ablation study.
Resources To our knowledge, only one of the six open-access organoid image datasets available in the literature concerns BOs with bright-field acquisition, which allows us to study the global development of these cultures  [3]. Other imaging methods that require coloration and slicing, for instance, focus mainly on cell composition in BO developmental structures or organoid regions. The original dataset, consisting of 40 images, is from  [22]. Twenty pathological and physiological cultures are captured with a bright-field microscope over 3 days. The grayscale images are 1088 × 1388 pixels. However, to execute our script in a reasonable time, the input images are cropped and resized to 250 × 250 pixels, maintaining the original proportions. Transform groups are composed by original images transformed by randomly chosen linear data augmentation strategies (flip flop, rotation and whitening).
For more details on the optimized generative methods used to synthesize images, please refer to  [6]  for loss update and normalization and to  [21]  for noise injection. Used datasets and amount of images are summarized in Table  1. We develop all scripts in Python 3. 6 with an Anaconda framework containing Keras 2. 3. 1 and TensorFlow 2. 1 and run the scripts on an Intel Core i7-9850HCPU with 2. 60 GHz and an NVIDIA Quadro RTX 3000s GPU device. 1 ) is from  [22] , (3) and (  4 ) samples are respectfully from  [6, 21]. Only samples from (2) are purposely transformed from the original dataset (1).
UNet-Mini Small dataset segmentation is a challenge in the domain. In 2020, ref. [20]  proposed a new architecture based on U-Net, with a few changes: they updated the skip connections with the addition of an elementwise product instead of concatenations; they reduced the layer number to four to reduce overfitting situations; and they reduced the number of kernels used in each convolution (ks = 3 × 3 and stride s = 1. They used a soft-max activation task for non-binary classification. )  [20]. This last update reduces the execution time and the number of hyper-parameters to learn (from 128 K instead of 17 M with the original implementation).
Training To segment the original images, we train the chosen architectures in a leave-one-out strategy summarized in Figure  2 : all original and augmentation groups are used during the training stage except one original image, which is segmented during the test phase using the generated model. We process 40 leave-one-out training models × 6 loss groups + 40 leave-one-out training models × 4 noise groups + 40 leave-one-out training models using a classic augmentation dataset constituted by flip-flopping, rotating, or whitening of original images, resulting in 440 training launches for each model. We retrieve the training time of the models. To end the training and avoid overfitting, we retrieve the training time before the loss value plateaus. Original, transformed and generated images are manually segmented with ITK-SNAP software to obtain ground truth images  [26].
Quantitative As we cannot decipher the real precision of segmentations using human observation on a single image from a dataset, we calculate median scores for each case summarized for loss optimizations, presented in Table  6  and for noise injections, presented in Table  7. Dice scores are higher for all architectures under AAE loss optimization strategies than under classic data augmentation. The highest Dice score is achieved by the U-Net architecture when AAE Perceptual Wasserstein loss optimization is used during training. Use of the loss optimization in UNet-Mini training also results in a high Dice score, but this score is lower than that of the U-Net architecture. As shown by the median Dice scores in Table  7 , Mu-Net seems to be the most accurate, followed by U-Net and UNet-Mini.
Discussion The main outcomes of this study are as follows: we present a new light segmentation methodology (Mu-Net), similar to architectures from the literature and employing various data augmentation strategies, to segment small bright-field image datasets of BOs, which have particular formations that are essential to extract. The presented framework, based on the U-Net network, runs the quickest for a single launch or for an entire leave-one-out process but has a computational efficiency similar to that of UNet-Mini. Mu-Net exhibits nearly the highest precision scores obtained with U-Net. It reaches these scores using classic data augmentation strategies during training, whereas similar scores obtained with U-Net use optimized AAE strategies. The ablation study confirms that another intermediate architecture, based upon a single layer reduction, could be used to obtain higher scores.
UNet-Mini does not obtain the highest scores in some noise optimization cases, rendering this architecture less robust than Mu-Net or U-Net. Some ML segmentation algorithms seem ill-suited for extracting the shape of BOs without optimization through preprocessing, and these algorithms are only semiautomated. Mu-Net outperforms almost all the ML procedures except the Kmean clustering algorithm according to some scores. To our knowledge, this is the first automated segmentation dedicated to a small dataset of two-dimensional bright-field images of BOs. Segmentation of small datasets (which are not always labeled) remains an issue in the biomedical image domain. Segmentation is crucial for extracting the morphological shapes and characterizing the physiological or pathological development of BOs to form a brain in vitro model that could fill the gaps of in vivo models  [1].
Only U-Net has been used for automatically segmenting ventricular zones in the Scout software for clarified BO images and for extracting the shapes of two dimensional BO images to validate the accuracy of the data augmentation strategy  [4, 6]. However, the training time, the required number of labeled images and the approximate segmentation of neuro-epithelial regions (essential for determining whether a culture is well developed) convinced us to find an appropriate solution for these particular cultures. Among light U-Net architectures, Mu-Net and UNet-Mini reduce the training time. This could be explained by the drastic reduction of learned hyperparameters and the layer reduction in these networks  [16, 20]. Usually, the architecture is specified to produce highly accurate segmentation.
Mu-Net may be limited from reaching similar high results on other datasets with higher resolutions. However, to overcome this potential issue, employing an attention module in our reduced architecture can improve the result  [13, 14]. Mu-Net is almost as accurate as U-Net depending on the data augmentation strategy used during training. In contrast, UNet-Mini never achieves higher Dice scores than 0. 85, despite being implemented similarly to Mu-Net. This is explained by the transfer learning method used during training: while U-Net and our Mu-Net architecture use concatenations for long skip connections, UNet-Mini uses an additional long skip connection based upon an elementwise product  [5, 20]. A skip connection skips layers in the neural network and uses the output of one layer as the input to future layers.