dataesr/scientific-paragraphs-categorization huggingface dataset:
1%), presence of artifacts on the images (37, 4. 3%), overlapping wings (43, 5. 0%), or broken wings with missing landmarks (49, 5. 7%). The proportion of rejected wings varied among datasets, ranging from 0. 0% (Portugal, 0 wings) to 39. 1% (Ouessant, France, 43 wings). This meant an average (± SD) success in automatic landmark annotation of 92. 23% ± 9. 36 across the individual datasets. The final sample sizes identified by the five-subspecies classifier of DeepWings© were 13,960 for wings and 2488 for colonies. Each wing was classified into the subspecies that showed the highest prediction probability, ranging from as low as 0. 300 to 1. 000, with a median of 0. 968. The classification results at the colony level (inferred from the average wing) were similar, as the highest probability ranged from 0.
For example, in the dataset of Terceira, 23. 0% of the samples lie in the upper quarter for the molecular marker (>0. 75) and in the lower quarter for the morphological marker (<25), indicating a poor agreement between the two. Nonetheless, for the whole dataset (N = 1214), a significant association was found between the two markers, as revealed by Spearman's correlation test (r = 0. 31; 0. 25 < 95% confidence interval < 0. 36; p < 0. 0001).
Discussion In this study, 14,816 wings representing 2601 colonies from 15 countries and covering the native ranges of A. m. iberiensis, A. m. mellifera, and A. m. carnica were analyzed using Figure  3. Scatter plots showing the probability of belonging to M lineage (Y-axis) vs. membership proportions in M lineage (X-axis) for individual datasets and for the whole dataset. Classification probabilities were inferred from wing images using the software DeepWings©, whereas membership proportions were inferred from microsatellites (Groix, Ireland and Russia) or SNPs (Portugal, Spain, Azores, Avignon, Ouessant, and Wales) using the software Structure or Admixture, respectively. Each dot represents a colony.
Discussion In this study, 14,816 wings representing 2601 colonies from 15 countries and covering the native ranges of A. m. iberiensis, A. m. mellifera, and A. m. carnica were analyzed using the WGM approach implemented by DeepWings©. This large and diverse dataset of wing images, originating from such a wide geographical range and acquired using varied image acquisition systems, offered a unique opportunity to test the performance of DeepWings© under real conditions. Moreover, the interaction with the numerous wing image contributors (beekeepers and researchers), who have different experiences and needs, allowed us to introduce several improvements in the software. These included (i) estimation of CI, HI, and DSA; (ii) display of the landmark-annotated wing images; (iii) production of a table with the landmark coordinates; (iv) inference of an average wing from a batch of wings, allowing classification of a colony from multiple wings; and (v) display of the three best classifications for the analyzed wings.
DeepWings© successfully classified 94. 2% of the wings, consistent with the rate predicted by the software developers  [44]. The classification of the European colonies largely matched the endemic M and Clineages, with proportions of 71. 5% and 97. 6%, respectively, as the top two probabilities were typically assigned to subspecies sharing lineage ancestry. However, when analyzed at the subspecies level, the matching proportions decreased, as samples collected in the A. m. mellifera range were often classified as A. m. iberiensis (the reverse was less frequent) and samples collected in the A. m. carnica range were often classified as A. m. ligustica. These findings are not surprising given that subspecies belonging to the same evolutionary lineage share a recent ancestor and are, therefore, genetically closely related  [48].
The reference database used by the classification module of DeepWings© was constructed from a small subset of the original wings analyzed by Ruttner in his seminal taxonomic work  [44]. Therefore, it only partially covers the natural variation in wing shape patterns that existed at the time for each subspecies. This limitation is further aggravated by the circumstance that Ruttner's collection was assembled over 50 years ago, and wing venation patterns can change through time, as recently reported for Romanian populations  [41]. The detection of wing venation patterns corresponding to the divergent A. m. ligustica and A. m. carnica in France, Switzerland, the UK, Ireland, Poland, and Russia, therefore mismatching the expected M lineage, can be explained by beekeeper-mediated gene flow and is consistent with molecular surveys reporting variable C-derived introgression in A.
This finding calls for an improvement of the software to increase its discriminating power, which implies expansion of the reference database used for training the current version of DeepWings© with wings from collections other than that of Ruttner. DeepWings© is a dynamic tool that can be easily upgraded to include more wings of each subspecies and/or more subspecies by adding their images to retrain a classification model using machine learning  [44]. In contrast to the findings north of the Pyrenees, only a small proportion of the colonies (34 from Spain and 3 from Portugal, 5. 7%) examined in the A. m. iberiensis native range were recognized as A. m. mellifera with probabilities above 0. 950, indicating that DeepWings© performed relatively well. While misclassification of these colonies cannot be ruled out, the detection of wing shapes matching A.
m. mellifera and A. m. iberiensis. Alternatively, but not mutually exclusive, A. m. ligustica genes can were introduced in Poland by undocumented importations of Italian queens and/or by the documented and steadily increasing importations of the artificial strain Buckfast  [63]. This is a plausible hypothesis because, contrary to Poland, in Croatia, Hungary, Romania, and Slovenia the proportions of wings matching A. m. ligustica were low, as would be expected in a territory where A. m. carnica is endemic and favored by local beekeepers  [5]. Moreover, this finding is consistent with the presence in these countries of mitochondrial and nuclear alleles of A. m. ligustica ancestry  [29, 33, [64] [65] [66] [67] [68]. Another possibility is that wing images belonging to C-lineage subspecies unrepresented in the DeepWings© reference database and not included in the five-subspecies classifier (e.
In contrast, molecular markers, such as microsatellites and especially SNPs, are widespread across the honey bee genome  [20, 78] , and molecular assays can be designed to fully cover the 16 chromosomes  [18, 19, 24]. If the purpose of colony analysis is to determine the degree of genetic integrity and to estimate introgression proportions with high accuracy, then molecular markers are preferred over wing markers. Otherwise, DeepWings© offers a good alternative for colony identification. By processing batches of up to 40 wings, the software averages out intra-colony variation from a large number of sampled workers, therefore enabling a more robust colony classification. Given that numerous images can be easily and rapidly processed at no cost, DeepWings© is a valuable tool for colony screening in honey bee breeding programs for conservation or other purposes that do not require or do not have a budget for molecular identification.
Supplementary Materials: The following supporting information can be downloaded at: https://www. mdpi. com/article/10. 3390/insects13121132/s1, Table  S1 : Sampling locations, number of analyzed wings and colonies, and number of loci per molecular marker. Locations highlighted in bold indicate samples originating from conservation programs. ; Table  S2 : Number (N) and probability (median, IQR) of wings classified into each subspecies per dataset. , Table  S3 : Number (N) and probability (median, IQR) of colonies classified into each subspecies per dataset.
Acknowledgments: Authors would like to thank  Szilvia Kusza  for help with collecting samples in Hungary and to  Sergey Gurov ,  Marat Khasanov ,  Vladimir Kugeiko ,  Kirill Kugeiko ,  Vener Sattarov ,  V. I. Kartavy ,  Sergey Igumnov  and  Vladimir Kirikov  for preparing and sending wing images from their apiaries in Russia.
Classifying or predicting occupational incidents using both structured and unstructured (text) data are an unexplored area of research. Unstructured texts, i. e. , incident narratives are often unutilized or underutilized. Besides the explicit information, there exist a large amount of hidden information present in a dataset, which cannot be explored by the traditional machine learning (ML) algorithms. There is a scarcity of studies that reveal the use of deep neural networks (DNNs) in the domain of incident prediction, and its parameter optimization for achieving better prediction power. To address these issues, initially, key terms are extracted from the unstructured texts using LDA-based topic modeling. Then, these key terms are added with the predictor categories to form the feature vector, which is further processed for noise reduction and fed to the adaptive moment estimation (ADAM)-based DNN (i.
e. , ADNN) for classification, as ADAM is superior to GD, SGD, and RMSProp. To evaluate the effectiveness of our proposed method, a comparative study has been conducted using some state-of-the-arts on five benchmark datasets. Moreover, a case study of an integrated steel plant in India has been demonstrated for the validation of the proposed model. Experimental results reveal that ADNN produces superior performance than others in terms of accuracy. Therefore, the present study offers a robust methodological guide that enables us to handle the issues of unstructured data and hidden information for developing a predictive model.
Therefore, the use of the predictive model is utmost important in incident analysis and prevention. In practice, once an incident is taken place, safety professionals narrate the incident event in their own language and log them into the electronic database. Therefore, the exactness of the incident mostly depends on the experience and quality of writing of the personnel who logs information into the system. It is often found that the incident narratives, which are in the form of unstructured texts, remain underutilized or sometimes unutilized since the proper utilization of these unstructured text data for information retrieval demands an extensive human effort. Reviewing the incident narratives during the investigation is extremely time-consuming. In addition, narratives are sometimes written in such a way that useful information can hardly be effectively retrieved, and thus analyzed.
Their experiments explored that all the three models were capable of learning from human knowledge for classification. In 2007, a notable study by Pons-Porrata et al. [13]  showed the development of a topic discovery system based on a new incremental hierarchical clustering algorithm and Testor Theory to extract and classify the implicit knowledge in news streams. Experimental results showed its usefulness and effectiveness in not only topic detection, but also in classification and summarization tool. Brooks  [14]  used SAS Text Miner to mine free texts of workers' compensation claims and classify into two categories. Their experimental results suggested that text mining can be used as a stand-alone tool for free-text analysis. Fan & Li  [15]  used text mining to retrieve historical cases from a case library.
Song & Suh  [21]  used patent analysis using latent Dirichlet allocation (LDA), for extraction of the latent topics and main keywords contained in documents, and network analysis for monitoring change patterns and relations to identify the trends in technology development that prevent the risks of various industrial systems. Apart from these, LDA has been used in different areas, including for pattern extraction from OSHA databases  [22] , construction reports  [23] , and investigation reports generated from manufacturing plants  [24, 25]. All the reports are prepared in natural languages. Therefore, natural language processing (NLP) is an essential task for accident analysis for the extraction of useful information hidden in texts. Brown  [26]  used NLP for identification of the contributors to rail accidents from accident narratives and implemented random forest (RF) to check the predictive power of the contributors toward the accident occurrences.
It is used in a wide spectrum of application domains. For examples, ANN was successfully used with the backward algorithm (BA-ANN) in the prediction of an outburst of coal and gas by He et al. [36]. It was also used to develop an advanced detection system for the prediction of the rating of worker's health in a hot as well as humid conditions in construction sites  [37]. The approaches like ANN and SVM attain widespread popularity since they hold a strong theoretical underpinning enabling us in dealing with the complexity of the problem, learning from the historical information, and more importantly, exploring adaptability with nonparametric theory. However, the interpretation of SVM and ANN model is rather difficult. Although the aforementioned ML algorithms including ANN  [25] , DT  [38] , SVM  [39]  have been used in frequent in accident analysis; however, these algorithms are only capable enough to effectively utilize the existing attributes of the dataset for prediction of accident occurrences.
Nevertheless, there exist a number of hidden attributes or factors within the dataset in a different form of data, which can be hardly retrieved and used by the conventional ML approaches  [40]. For instance, hidden attributes can be characterized as ones that require linear or nonlinear transformation. The deep learning method, in this case, can be a better choice for the researchers to obtain information from the data, including the attributes present in either explicit or hidden form. The approach utilizes a number of hidden layers to extract the hidden factors underlying within data to better predict the incident outcomes  [40]. Deep neural network (DNN), in this case, works extremely well (in particular NLP-based analysis)  [41, 42]  as it offers advantages of efficient generation of new features from raw data and accurate classification of feature vectors  [43].
Hinton and Salakhutdinov  [60]  suggested that the GD approach can be useful for training a DNN architecture in such a case where the optimization parameters like weights are initialized with the values close to an optimum solution. In fact, the condition is very difficult to be fulfilled and consequently, this algorithm gets trapped into local minima. Moreover, the speed of convergence of this algorithm is found to be very slow while dealing with a large dataset. In case of high dimensional optimization problems, stochastic GD algorithm is frequently used due to its faster rate of convergence, and easy implementation. Another optimization algorithm, namely root mean square propagation (RMSProp), in this case, produces better performance since it uses the average of the second moments of the gradients (the uncentered variance)  [64].
In our study, it has been demonstrated that the proposed approach is better than other optimized DNN classifiers, namely RMSProp and SGD-based DNN. In addition, the classification performance of the proposed approach has been tested on five different benchmark datasets from the literature. Finally, a comparative study has been made between the performance of ADNN classifier and that of the other state-of-the-art classifiers, namely SVM, k-NN, and RF to explore the efficiency of our proposed strategy in incident prediction. The remainder of the paper is structured in the following way: In Sect. 2, the proposed methodology has been discussed in brief. In Sect. 3, a case study is provided with data collection, data description, and data preprocessing steps. Results and discussions of the analyses are presented with the statistical tests in Sect.
Methodology The proposed methodology comprises four phases which is displayed in Fig. 1. In Phase-I, common data pre-processing tasks, namely topic modeling, missing data handling, class imbalance handling, outlier handling, and data transformation are performed. In the next phase, i. e. , Phase-II, three optimizers, namely SGD, RMSProp, and ADAM are used to tune the parameters of DNN algorithm. In Phase-III, the best classifier is selected through comparative study and in the final phase, i. e. , Phase-IV, prediction of incident outcomes using the best classifier is performed. Initially, our models have been over-fitted on the test data. The training accuracies have been found much higher than testing accuracies. To overcome this shortcoming, hyper-parameter tuning of the models has been done using 10-fold cross validation on training dataset following some earlier studies  [66] [67] [68].
Testing dataset has been used only for the model evaluation. In this cross validation process, first, the dataset has been shuffled randomly. Then, the dataset has been split into 10 groups. One of the 10 groups has been taken as test dataset and rest of them have been taken as training dataset. Then, the model or classifier has been fitted with the training set and evaluated it on the test set based on the performance score. This procedure repeats for 10 times on different test sets and the model performance has been obtained by averaging out the 10 different performance scores. Based on this average performance score, the final model is evaluated. Then, the algorithm with the highest accuracy is considered as the best one. The methods of data pre-processing (i. e. , text data handling using topic modeling, missing data handling, class imbalance handling, outlier handling, data transformation from categorical to continuous form), classification (i.
Class imbalance handling To handle the class imbalance issue in data, Synthetic Minority Over-sampling Technique (SMOTE) algorithm is used. It was proposed by Chawla et al. [74]. It is an oversampling technique. It oversamples the minority class by generating synthetic samples in the imbalanced dataset. Each sample of minority class is considered initially and the new samples are generated along the line segment which connects this sample with its minority nearest neighbor. It usually works by oversampling minority class and undersampling the majority class simultaneously. That is why it can produce better classification performance than only undersampling. The steps of this algorithm used in this study are displayed in Fig. 2.
Data transformation After the outlier detection and reduction, the reduced dataset is transformed from its categorical form to the continuous form using a gravity factor (GF)-based normalization technique  [6]. In this data processing stage, uncertainty arises as the values of the decision classes are user-defined. GF is a normalized value, which is calculated from the frequency of each of the categories in each attribute. The following Eq. (  2 ) is used to estimate the GF values of the data: GF ¼ P n i¼1 x i y i n Â P n i¼1 x i ð2Þ , where x i and y i denote the percentage of each category of each predictor and a corresponding normalization factor for risk, respectively. The value of n is taken as equal to three since the response attribute 'Incident outcomes' has three classes (i. e. , injury, nearmiss, and property damage).
SGD algorithm SGD is a popular optimization algorithm, which is used to minimize the objective function with model parameters h. The parameters are updated in the opposite direction of the gradient of 5 h JðhÞ  [65]. The size of the steps is determined by the learning rate, g. Let h tÀ1 be the value of the parameters for the ðt À 1Þ-th iteration. Then, the updated value of the parameters for t-th iteration (where input is x and output is y), defined as: h t ¼ h tÀ1 À g Â 5 h tÀ1 Jðh tÀ1 ; x; yÞ ð7Þ , where t ¼ 1; 2; :::; T. Here, T represents the total number of iterations. At each iteration, the values of the parameters are updated for every sample present in the dataset. Due to the capability of the SGD of updating the parameters one at a time, it works very faster and can be used in an online settings.
Case study The data consisting of a total of 9473 incident records have been retrieved from a steel manufacturing plant over the period of 2010 to 2013. After collection of data, they are preprocessed. The dataset comprises a total of thirteen attributes (11 categorical, and two free unstructured text attributes), of which 'Incident outcome' is deemed as the response attribute. A short description of attributes with their percentage of occurrence in the dataset is provided in Table  1. After the data collection, they have been preprocessed. In pre-processing, some basic tasks have been This attribute implicates a guideline, which should be maintained or followed during work. There are a total of six classes in this attribute, namely SOP is available and followed (SAF), SOP is available but not followed (SANF), SOP is inadequate but is followed (SIF), SOP is inadequate and is not followed (SINF), SOP is not available and is not required (SNNR), and SOP is not available but is required (SNR).
Generation of a categorical attribute using topic modeling There are two text attributes, called 'BD' and 'EL' within the dataset, which comprise the description of incidents. LDA topic modeling has been used to create a categorical attribute from the texts. Four metrics have been used simultaneously to determine the number of topics optimally from the attributes of the unstructured text. From the topic modeling, a total of nine topics are extracted (refer to Fig. 5 ). With each of the topics, an exhaustive list of terms, based on the probability of occurrence, is generated. Using the list, a meaningful event is obtained for each topic. For the purpose of visualization, the top eight terms per topic with its corresponding probability of occurrence are shown in Table  2. For instance, in Topic1, the top eight terms, 'road,' 'shift,' 'near,' 'injury,' 'come,' 'sudden,' 'duty,' and 'fell' are found.
Prediction This section demonstrates the classification performances of the three optimized DNNs, namely ADNN, RMSProp-DNN, and SGD-DNN. Evaluation of the performances is done based on incident data and five other benchmark datasets, namely 'Breast cancer,' 'Iris,' 'PID,' 'Hungarian,' and 'Cleveland' retrieved from UCI Machine Learning Repository 1. Besides these, other three state-of-the-art classifiers, namely k-NN, SVM, and RF are also employed  Therefore, convergence plots are exhibited for these three optimizers only. It is to be noted that in all datasets, information related to attributes and their respective classes are given. Using this information, important attributes are extracted and used in analyses for better classification performance. The extraction of attributes is done by using a stacked auto encoder.
For example, the stacked auto encoder is applied over the data, 'Iris,' which has four attributes, namely, 'sepal length,' 'sepal width,' 'petal length,' and 'petal width,' and three decision or response classes, namely 'setosa,' 'virginica,' and 'versicolour. ' Now, the dataset is divided into two sets: training set and test set. The number of inputs used in an auto encoder is same as the number of attributes in the data. Hyper-parameters are multiplied with attribute values for each input sample to extract its (e. g. , input sample) feature value. This feature extraction is done in hidden layer of auto encoder. For all the samples in input data, a feature map is generated. The feature map constitutes all the feature values generated from the input data. In stacked auto encoder, the generated feature map is fed to the next auto encoder to get its reduced feature map.