dataesr/scientific-paragraphs-categorization huggingface dataset:
In this last case we note the two tools as IRSOM_cross and CPAT_cross, respectively. The default parameters of the tested tools have been used during the training and prediction as explained by the authors in their respective documentation. For each model computed with CPAT, we determined the threshold separating the coding and the ncRNAs by maximizing the sensitivity and specificity. For PLEK, we set the minimal length of an accepted transcript to 20 in order to keep the small transcripts (which represent a small part of the datasets). In case of our tool, there are different parameters. The SOM dimension which is a grid of size 10 Â 10. The parameter a in the r function is set to 0. 5. The regularization factor is set to 0. 001. The size of the batches is set to 100. The learning constants l 1 is set to 0.
Cross-validation results As mentioned above, we performed a 10-fold cross-validation with our tool IRSOM. For each species dataset D, we applied IRSOM on the subset D sub1 (as described in Section 3. 2). The obtained results are given in Figure  4  and show the performance of IRSOM for different rejection thresholds. As we can see on the Figure  4 , the performance of IRSOM increases when we increase the rejection threshold. But as we can see in Figure  5 , the number of rejected prediction increases when the rejection threshold increases. In order to define a good rejection threshold, we need to find a trade off between the performance and the reject rate. For the cross-species model of IRSOM, we set the rejection threshold at 0. 7. This threshold gives good performance for all species (Accuracy greater than 0.
Prediction results On each species dataset D, we performed predictions on the subset D sub2 , using the different tools IRSOM, CPC2, CPAT, CNCI and PLEK. The predictions were processed using the provided models in case of CPC2, CNCI and PLEK and with the models obtained after a training on the subset D sub1 in case of IRSOM and CPAT. With IRSOM and CPAT, we performed two types of tests, one where the corresponding model is used for the considered species, and one where the cross-species model is used for each of the species. Figure  6  shows the obtained results. For IRSOM, we use the rejection thresholds defined previously on the subset D sub1. The tools retrained on our datasets, i. e. IRSOM and CPAT, are represented by full lines and the that we could not retrain (we used the provided models) are represented by dotted lines.
The obtained results show a good performance of our tool IRSOM compared to the other tools. IRSOM exceeds 0. 95 in accuracy for all the species. Compared to CPAT, the only tool we succeeded to retrain on our data and the second best tool, IRSOM shows slightly better results for all considered species. Furthermore, the two models of CPAT show the same performance on all datasets as for the two models of IRSOM (except on the Human) but with different reject rate (Fig. 7 ). CPC2 shows an accuracy greater than 0. 9 on every datasets except on the Zebrafish (0. 88) and the O. sativa (0. 8). These results  are explained by the lower specifitciy on these datasets (0. 84 for the Zebrafish and 0. 77 for O. sativa). CNCI was used only on the vertebrates (Human, Mouse and Zebrafish) and plants (A. thaliana and O.
sativa) datasets according to their models. CNCI shows lower results than CPAT, CPC2 and IRSOM on all the datasets, except on the Zebrafish where it gives better results than CPC2. The tool that gives the worst results is PLEK. As mentioned above, the model provided by the authors is the result of a training on Human dataset. On the Human and Mouse datasets, PLEK shows suitable results. As prospected, the best results are those on Human (0. 93). On the Zebrafish, we obtained an accuracy of 0. 78 when the authors have shown in  (Li et al. , 2014)  an accuracy of 0. 91. Finally, IRSOM gives the best performances (accuracy mean: 0. 98) compared to the others tools [CPAT (both models): 0. 97, CPC2: 0. 91]. Every tool gives a nearly perfect sensitivity [IRSOM (both models): 0. 99, CPAT (both models): 0.
97 and CPC2: 0. 97] but lower performance in term of specificity except for IRSOM (0. 98 for both models) and CPAT (0. 97 for both models). These results suggest that all the tools are able to correctly identify the ncRNAs, but predict wrongly a part of the coding RNAs. With our rejection option, we can identify and reject the prediction of the ambiguous transcripts and improve our specificity. Furthermore, the mean reject rate of both models is around 15% with a slightly higher reject rate for the species specific model (Fig. 7 ). These results confirm the ability of our method to gives accurate prediction and identify the ambiguous transcripts without rejecting too much data. In addition the rejected transcripts can be visualized and analyzed using the SOM.
Study case Here, we demonstrate how the rejection option improves the prediction results. To do so, we investigate the prediction by visualizing the SOM prototypes and the distribution of the labels in the SOM. One of the most interesting properties of SOM is its capacity to visualize the data by projecting it to a low dimensional space. By using this property, we can extract the profiles of the transcripts that are close to a neuron prototype by taking its representative. In our case, we look at the transcripts that are rejected in the Human dataset using the species specific model of IRSOM. The Figures  8  and 9  show, respectively, the true label distribution and the predicted label (with rejection) distribution of the SOM in the hidden layer of IRSOM. We can see in Figure  8  that the map separates well the coding RNA from the ncRNA.
In Figure  9 , we can see that the neurons assigned to the rejected predictions are at the boundary between the coding and the non-coding regions in the map except, for one neuron in the top right corner of the map. In the case of this neuron, the non-coding neuron output is too low and so the difference between the two output values is lower than the threshold used for the Human dataset (which is 0. 8). As a reminder, a prediction is rejected if the difference between the coding and the non-coding neurons outputs is lower than a threshold b. The Figure  10  shows the profiles of the representatives for the ORF features. Each point in the figure represents the value of the representative for a given feature. We can see that the coding transcripts have a high ORF coverage while the non-coding transcripts have a lower or near zero ORF coverage as expected.
Running time To compare IRSOM's time performance with that of the other tools, we measured the prediction time of each of them on each species dataset (with all the sequences) as well as on the cross-species dataset. The obtained running times on the different datasets, ordered from the smallest to the biggest one, are given in Table  2. As we can see in Table  2 , our tool IRSOM gives comparable running time compared to the existing tools. For instance, on the Human dataset, which is composed of around 38 000 sequences, it took around less than 2 min for the prediction and also less than 4 min to generate the Human model. The prediction time difference between IRSOM and CPAT and CPC2 (which are the two fastest) is due to the computation of the features. In our tool, the features are computed by a cþþ executable and imported in a python script for the prediction (or training).
Conclusion We presented here a new approach and tool for identifying rapidly and efficiently ncRNAs. Our tool, called IRSOM is able to accurately discriminate coding and ncRNAs. Furthermore, with our rejection option, we are able to identify the ambiguous transcripts and analyze them with the SOM. Compared to the state of art, our tool gives the best results on several species of different reigns. It gives also good time computing for small and large datasets. By using the rejection option, we are able to increase the prediction accuracy. Moreover, we highlight the fact that the limit between coding and non-coding transcripts is not well defined. And so, the coding and the non-coding transcripts have to be seen as a range of transcripts instead of two separable types of transcripts. One of our future work to improve the rejection part of our algorithm is to define a method to compute the optimal rejection threshold.
Big data in healthcare and medicine refers to these various large and complex data, which they are difficult to analyse and manage with traditional software or hardware  [3] ,  [4]. Big data analytics covers integration of heterogeneous data, data quality control, analysis, modeling, interpretation and validation  [5]. Application of big data analytics provides comprehensive knowledge discovering from the available huge amount of data. Particularly, big data analytics in medicine and healthcare enables analysis of the large datasets from thousands of patients, identifying clusters and correlation between datasets, as well as developing predictive models using data mining techniques  [2]. Big data analytics in medicine and healthcare integrates analysis of several scientific areas such as bioinformatics, medical imaging, sensor informatics, medical informatics and health informatics.
Big Data Characteristics The term big data is described by the following characteristics: value, volume, velocity, variety, veracity and variability, denoted as 6 "Vs"  [13] ,  [14] , shown in Figure  1. Besides these 6 "Vs", some authors has defined more than these 6 properties to describe big data characteristics  [15]. The volume of health and medical data is expected to raise intensely in the years ahead, usually measured in terabytes, petabytes even yottabytes  [14] ,  [16]. Volume refers to the amount of data, while velocity refers to data in motion as well as and to the speed and frequency of data creation, processing and analysis. Complexity and heterogeneity of multiple datasets, which can be structured, semi-structured and unstructured, refer to the variety. Veracity referrers to the data quality, relevance, uncertainty, reliability and predictive value  [14] , while variability regards about consistency of the data over time.
The value of the big data refers to their coherent analysis, which should be valuable to the patients and clinicians. Considering the big data characteristics, data searching, storage and analysis, a very appropriate and promising software platform for development of applications that can handle big data in medicine and healthcare is the open-source distributed data processing platform Apache Hadoop MapReduce  [1] ,  [17]  that is based on data-intensive computing and NoSQL data modeling techniques  [18].
Despite of these drawbacks of the -omics data, EHRs data are very influenced by the staff entered the patient's data, which can lead to entering missing values, incorrect data as a result of mistakes, misunderstanding or wrong interpretation of the original data  [5]. Integration of data from various databases and standardization for laboratory protocols and values still remain challenging issues  [10]. High dimensionality of the -omics data means, that there have many more dimensions or features than the number of samples, and on the other side the EHRs data which regard to the individuals/patients, makes data mining techniques to be more challenging task. The subsequent stage is the pre-processing of the data, which usually envelop handling noisy data, outliers, missing values, data transformation and normalization.
Discussion and Future Work Big data analytics in medicine and healthcare is very promising process of integrating, exploring and analysing of large amount complex heterogeneous data with different nature: biomedical data, experimental data, electronic health records data and social media data. Integration of such diverse data makes big data analytics to intertwine several fields, such as bioinformatics, medical imaging, sensor informatics, medical informatics, health informatics and computational biomedicine. As a further work, the big data characteristics provide very appropriate basis to use promising software platforms for development of applications that can handle big data in medicine and healthcare. One such platform is the open-source distributed data processing platform Apache Hadoop MapReduce that use massive parallel processing (MPP)  [20] ,  [24].
Recent breakthroughs in the computer vision community have led to the emergence of efficient deep learning techniques for end-to-end segmentation of natural scenes. Underwater imaging stands to gain from these advances, however, deep learning methods require large annotated datasets for model training and these are typically unavailable for underwater imaging applications. This paper proposes the use of photorealistic synthetic imagery for training deep models that can be applied to interpret real-world underwater imagery. To demonstrate this concept, we look at the specific problem of biofouling detection on marine structures. A contemporary deep encoder-decoder network, termed SegNet, is trained using 2500 annotated synthetic images of size 960 × 540 pixels. The images were rendered in a virtual underwater environment under a wide variety of conditions and feature biofouling of various size, shape, and colour.
Introduction Deep learning techniques have attracted significant interest in recent times as they have produced impressive results on benchmark and real-world datasets across a wide range of computer vision tasks. While deep learning methods are gaining popularity, a number of key obstacles still persist. Most notably, deep learning methods typically require very large training datasets to achieve good results and significant amounts of computational memory are necessary during inference and training stages. While computational power is continually improving and is becoming less of a barrier, there is still a scarcity of high-quality labelled training datasets for many applications, and this is especially true for underwater imaging applications. Curating a dataset takes time and domain-specific knowledge of where and how to gather the relevant information, and it often involves a human operator having to manually identify and delineate objects of interest from real-world images.
This is a tedious and time-consuming task considering that datasets of up to thousands-or even tens of thousands-of training images are required to build a robust and effective deep network. Having good quality training data is arguably one of the most important elements of any machine learning system and this is particularly the case for deep learning methods. A number of large-scale datasets have been created to provide dependable training data and to assist algorithm designers when devising new deep learning techniques for classification, detection, and segmentation tasks. This paper is concerned with semantic segmentation, which is the task of assigning each pixel in the input image a class in order to get a pixel-wise dense classification. Common datasets that can be used for training deep networks for semantic segmentation include:.
• SYNTHetic collection of Imagery and Annotations (SYNTHIA)  [5]  is a collection of imagery and annotations which has been generated with the purpose of aiding semantic segmentation and related scene understanding problems in the context of driving scenarios. SYNTHIA consists of a collection of frames rendered from a virtual city under a wide range of lighting and environmental conditions. It includes classes for buildings, roads, cyclists, and cars amongst others. Since the imagery is synthetic in nature, the pixel-level semantic annotations are precisely known. Road scene understanding/autonomous driving is a popular application area and there are several other datasets relating specifically to this topic, such as KITTI  [6] , Cityscapes  [7] , and CamVid  [8]. While these datasets are incredibly valuable for developing methods capable of interpreting road scenes/detecting common everyday objects, they are not particularly useful when it comes to developing custom underwater imaging techniques for applications such as damage detection, biofouling assessments, fish detection, etc.
There is an underwater dataset, known as the Underwater Lighting and Turbidity Image Repository (ULTIR)  [9] , which contains annotated images of various types of damage, such as cracks and corrosion, captured under a host of underwater visibility conditions, however, there is less than 100 images per damage class and this is not sufficient to train deep models. To help bridge this gap, this paper proposes the use of synthetic imagery to train custom deep networks that can then be applied to interpret real-world underwater imagery. The focus of this paper is on interpreting underwater imagery for two main reasons. Firstly, underwater imaging is often severely hampered by factors-such as poor visibility, challenging underwater light-field, floating particle matter, air bubbles, colour attenuation, poor image acquisition practices, etc.
-which limit the ability of cameras to observe and record details in the scene, and consequently, the effectiveness of image analysis algorithms is impaired. With this in mind, methods that perform with credibility when applied to underwater images are particularly valued. Secondly, there is an acute lack of labelled datasets containing underwater imagery which makes it harder for researches to develop deep learning methods for underwater applications. In order to harness the power of deep learning methods for underwater applications, researchers must look at new sources of training data. This study demonstrates that photorealistic synthetic data can help in this regard. Synthetic image datasets have been developed and used to the train deep learning algorithms for detection and classification of underwater mines  [10].
A virtual scene of an underwater inspection site is created, and from this, a large dataset of synthetic imagery is generated with accurate ground-truth information that reveals the exact composition of the scene. This large dataset is then used to train a deep fully convolutional neural network architecture for semantic pixel-wise segmentation. While semantic segmentation/scene parsing has long been a part of the computer vision community, a major breakthrough came in 2014 when fully convolutional neural networks (CNNs) were first used by Long et al. [11]  to perform end-to-end segmentation of natural images. Their approach took advantage of existing CNNs which are designed for classification problems, such as AlexNet  [12] , VGG (16-layer net)  [13] , GoogLeNet  [14] , and ResNet  [15]. These classification models learn high-level feature representations, however, instead of classifying based on these extracted features, the compact feature representations are upsampled using fractionally strided convolutions to produce dense per-pixel labelled outputs.
Fully convolutional neural networks allow segmentation maps to be generated for input images of any size and are faster than previous patch-based deep learning approaches. They also achieved a significant improvement in segmentation accuracy over traditional methods on standard datasets like Pascal VOC while preserving efficiency at inference. Many of the leading techniques that have emerged since Long et al. 's breakthrough work are also based on fully convolutional neural networks. A comprehensive review of deep learning techniques for semantic segmentation is provided by  [16]. The performance of deep learning methods depends on the architecture of the deep network. Using an existing network/architecture, which is known to work well, as the basis for training new deep networks is usually a good strategy.
Moreover, underwater three-dimensional (3D) imaging techniques, such as  [19] , are well-suited for collecting in-situ measurements of the thickness and size properties of biofouling instances. With this in mind, imaging systems may be regarded as a convenient and standalone tool capable of collecting all of the necessary data for biofouling assessments  [20]. Performing semantic segmentation on images of biofouling scenes is a challenge as, like many natural objects, marine growth species do not have consistent and well-defined shapes, and have non-uniform colour and textural properties. The reduced underwater visibility conditions add an additional layer of complexity to the detection problem  [21]. To tackle this problem, this study creates a virtual scene of an underwater inspection site, and from this, a large dataset of synthetic imagery is generated with accurate ground-truth information that reveals the exact composition of the scene.
This large dataset is then used to train a deep encoder-decoder network, which is applied to interpret real-world inspection imagery. This typically results in a good, albeit crude, segmentation. As an additional step to improve the quality of the segmentation, support vector machines are trained based on the initial segmentation map and boundaries of the detected biofouling regions are then iteratively refined using SVM classification so that the final segmentation map more closely aligns with the outline of objects in the scene. This paper is structured as follows. Section 2 presents the sample virtual scene employed in this study and describes the deep network and the training process in more detail. Section 3 presents the results, while Section 4 concludes the paper.
Animate Scene Properties In order for a deep neural network to generalise well and to be successful when presented with new scenes, the network should be trained on a large and diverse dataset (i. e. , a dataset consisting of thousands of images featuring varying lighting and visibility conditions, viewing perspectives, different plants from the same marine species category, etc. ). For this case study, the created dataset consists of 2500 rendered images. It exhibits a high degree of diversity as several parameters of the.
Animate Scene Properties In order for a deep neural network to generalise well and to be successful when presented with new scenes, the network should be trained on a large and diverse dataset (i. e. , a dataset consisting of thousands of images featuring varying lighting and visibility conditions, viewing perspectives, different plants from the same marine species category, etc. ). For this case study, the created dataset consists of 2500 rendered images. It exhibits a high degree of diversity as several parameters of the virtual scene were animated (i. e. , they evolved over time) to reflect many different conditions that may be encountered in a real-world setting. These parameters include:.
Render Images and Material Masks Since a virtual scene was the source of imagery in this study, the process of populating the dataset with new images could be easily automated and new images could be added on-demand. The chief limiting factor on the number of images was the rendering time needed to produce each image. On average, it took approximately 50 s to render an image and the corresponding ground-truth class label image. The size of each image is 960 × 540 pixels. A sample of the rendered images and the ground-truth class labels is shown in Figure  4. may be encountered in a real-world setting. These parameters include:  Properties of the underwater medium: transparency/visibility range and the colour of the water.
Render Images and Material Masks Since a virtual scene was the source of imagery in this study, the process of populating the dataset with new images could be easily automated and new images could be added on-demand. The chief limiting factor on the number of images was the rendering time needed to produce each image. On average, it took approximately 50 s to render an image and the corresponding groundtruth class label image. The size of each image is 960 × 540 pixels. A sample of the rendered images and the ground-truth class labels is shown in Figure  4. For this case study, the 'background' and the 'structure' class (with reference to the class legend in Figure  4 ) were combined into one class. Every pixel which was not deemed to represent biofouling regions was considered to be a member of the background class.