dataesr/scientific-paragraphs-categorization huggingface dataset:
2005). Fludioxonil, chemically 4-(2, 2-di uoro-1,3-benzodioxol-4-yl)-1H-pyrrole-3-carbonitril (Fig. 1 ) is non-systematic, colorless and odorless phenylpyrrole fungicide. It is strongly effective against fungal pathogens like stem-base browning, snow mold and seeding blight and can be used to inhibit black spot, gray mold and storage mold  (Agostini et al. 2006   Gaurilcikiene et al. 2008). Literature survey reveals that unsafe interaction of udioxonil can have negative health consequences on applicators and farmers. It's excessive exposure can cause diseases like cancer  (Go et al. 2017) , it is also toxic for aquatic life  (Verdisson et al. 2001). CODEX ALIMENTARIUS standard has de ned 0. 05 mg kg - 1   maximum residual limit (MRL) of udioxonil for cereal grains (Codex Pesticides Residues in Food Online Database (2018).
Instrumentation Liquid chromatographic system (Shimadzu Corporation, Japan) equipped with dual LC-20 AT solvent delivery modules connected with DGU-20A3/20A5 on-line degasser, tted with rheodyne manual injector connected with SPD-20A/20AV UV/VIS detector and Shimadzu CBM-20A communication bus module. Data acquisition was performed on LC solution GPC Chromatographic software (version 1. 25). λ max was measured on Shimadzu-1800 double beam UV/vis spectrophotometer. Rotary evaporator (Heidolph G 3 Germany) was used to concentrate the sample.
2 a description of the domain and the specific challenges of our use case are given. In Sect. 3 describes the literature review. Fundamental knowledge about the used metrics is provided in Sect. 4. In Sect. 5 we explain a simplified illustration of our AutoML tool. Furthermore, we describe the used datasets as well as the hyperparameter for the AutoML tool. An algorithm comparison is given in Sect. 6. In Sect. 7 we visualize and evaluate the optimized results. Section 8 summarizes the content of this paper and shows the added value based on the answered research questions.
Domain description In this section, we explain the production line and the specific challenges in this domain. A detailed explanation about the domain description and the production line structure are given in  (Gerling et al. , 2020). The goal of a manufacturing company is to produce as many flawless products as possible. These products are often built in a production line and get tested at test stations T1 to Tn. There is a particular order between tests stations, where Ta < Tb if Ta precedes Tb in the production process. With the data from test stations Ta, we aim to predict errors detected in test stations Tb with Ta < Tb. The objective for the company is to minimize the rate of errors in the production line-this becomes apparent by low error rates in the data. Usually, we have highly unbalanced datasets to train the ML model, because of the few errors in the production line.
We use these test features to train our ML model. Pre-processing the data is required to remove unusable test features like text descriptions. By preference, only numerical features are left after pre-processing to train the ML model, because not all algorithms can handle for example text for the model training. For our experiments, we applied pre-processing to the quality data to make usable datasets for the ML training. To train a ML model with data, we have to choose a ML algorithm. At this stage we face the next challenge because a production quality engineer wants to understand why an algorithm has made a specific decision. Therefore, our system should be capable to explain the results. The rationale is to increase trust in the ML system as well as interpretability. The second requirement concerns the metrics to evaluate the results.
A wide variety of program languages are supported. The H2O tool can be used as a standalone solution or together in a cluster solution. The developers of this tool collaborate with industrial partners and other research institutions. This tool has already been deployed in different domains and can be used by a wide variety of users with the Flow web-based GUI. Our AutoML tool solution differs here, because we provide a specific production domain solution with adjusted functionality. A key point to mention is the adjusted metric for the production and the strong support for the tasks of the quality engineer. (Krauß et al. , 2020)  shows possibilities and limits of applying AutoML in production. Further, it includes an evaluation of available systems. Moreover, a comparison of AutoML and a manual implementation from data scientists in a predictive quality use case was held.
At the moment, AutoML still requires programming knowledge and was outperformed by the implementation of the data scientists. One specific point was the preparation of the needed data. Without predefined domain knowledge, an AutoML system cannot merge the data correctly. Additionally, the integration of the data or the extraction from a database is problematic. A solution to this could be in form of an expert system. A further point was the deployment of the results or the models for the end-user. In conclusion, it can be said that AutoML systems provide the chance to increase the efficiency in a ML projects. This could be done by automating the necessary procedure within Data Integration, Data Preparation, Modelling and Deployment. The expertise of a data scientist and domain knowledge should be included to obtain satisfying results.
Nevertheless, the latest developments provide indicators for future improvements towards the automation of specific steps within the ML pipeline. Several approaches to automate machine learning appeared in recent years. (Maher et al. , 2019)  describe a meta learning-based framework for automated selection and hyperparameter tuning for ML algorithms (SmartML). SmartML has a meta learning feature that emulates the role of a domain expert in the field of ML. They use a meta learning mechanism to select an algorithm to reduce the parameter-tuning search space. The SmartML provides a model interpretability package to explain their results. The SmartML tool differs significantly from our approach. With our AutoML tool we pre-process the data with the provided background information of a product.
This tool supports various algorithms and uses them for training. The results are saved in a persistent database for the purpose of transfer learning. Vizier uses also RandomSearchCV  (Bergstra & Bengio, 2012)  and GridSearchCV  (Worcester, 2019)  to optimize the hyperparameter for algorithms. In  (Golovin et al. , 2017)  the authors do not present in detail which algorithms could be used to train a ML model and it seems to be a general solution to optimize algorithms. In comparison to our approach, we already provide pre-selected decision tree-based algorithms and offer a use case specific metric. Thus, we provide a more specific solution for the manufacturing domain. AUTO WEKA  (Thornton et al. , 2013)  is an open-source automation framework for algorithm selection and hyperparameter optimization based on Bayesian optimization using sequential model-based algorithm configuration (SMAC) and Tree-structured Parzen Estimator (TPE)  (Kotthoff et al.
For our tool, we only use tree-based algorithms to support the information value of the results. Literature for quality prediction In (Sankhye & Hu, 2020) the objective was to design machine learning based classification methods for quality compliance. Afterwards, a validation of the models via case study of a multi-model appliance production line was shown. In this case study, the proposed model for the classification could achieve a Cohen's Kappa  (Cohen, 1960)  of 0. 91 and an accuracy of 0. 99 for the compliance quality of unit batches. The main objective is the implementation of a predictive model for compliance quality, which could be enabled with the proposed method. Another aspect of this paper was to emphasize the importance of dataset knowledge and feature construction within the training of the classification models.
In this work two algorithms, namely RandomForest and XGBoost, was used but the second algorithm achieved better results. Further the Cohen's Kappa metric was used to tackle the imbalanced dataset problem. In this work, a classical machine learning approach without AutoML is used. The disadvantage here is that only two algorithms are used for classification. What is more important, however, is the selection of the metric. In  (Delgado & Tibau, 2019) , the author compared the Matthews correlation coefficient (MCC)  (Matthews, 1975 ) and Cohen's Kappa and concluded that Cohen's Kappa should be avoided as a performance measure for classification. In  (Zonnenshain & Kenett, 2020)  they discuss the challenges for quality engineering in the future. Moreover, they consider the future directions for quality and reliability engineering.
As results, the AdaC2 algorithm could provide the best results. To compare the algorithms the accuracy, balanced accuracy, precision, recall, F-measure, G-Mean and AUC were chosen as performance metrics. This work is similar to ours in terms of the highly unbalanced data and the approach to assigning costs to the different prediction classes. Our AutoML tool also uses decision tree-based algorithms for the fast and understandable prediction. We distinguish ourselves by the use of high-performance decision tree-based algorithms. Further, we optimize the prediction by using a cost-sensitive metric. In  (Moldovan et al. , 2017)  the authors investigated stateof-the-art approaches for using ML techniques on the SECOM dataset. This dataset contains data from a semiconductor manufacturing process and therefore represents an unbalanced real-world dataset.
Based on this dataset, three different feature selection methods were used. Further, the performance of three sample classification algorithms was compared. To measure the performance of the classifier, the F-measure, recall, False-Positive-Rate, precision and accuracy metrics was used. This paper only focuses on the SECOM dataset and therefore, provides only a process to achieve the best results specifically for this dataset. In our work, we focus on the ML part and the optimization of the algorithm. Further, we use various manufacturing datasets for our experiments. We also consider the unbalanced data problem and tackle this by adjusting the hyperparameter within the optimization. Literature for cost-based metric One of the first and bestknown paper regarding MetaCost and cost-sensitive classifier is presented in  (Domingos, 1999).
With this approach, the classifier will be adjusted for the different costs of errors. This procedure is well suited for imbalanced datasets to assign a cost value to the different classes. With this approach a large cost reduction compared to cost-blind classifiers can be achieved. The cost matrix for a two-class classification is set to C(0,0) C(1,1) 0; C(0,1) 1000; C(1,0) 1000r, where r was set alternately to 2, 5, and 10. In this case C(0,1) and C(1,0) are only relevant because of the ratio r. Number 0 represent the minority class and 1 shows the majority class. In our approach, we set our focus on the True Positive (TP) and False Positive (FP) respectively C(0,0) and C(0,1) and use a factor alpha to adjust the real cost savings (see the equation for Expected Benefit Rate Positives Only (EBRP) in the subsequent section).
In  (Loyola-González et al. 2019 ) a proposal of an algorithm for discovering cost-sensitive patterns in class imbalance problems was given. Further, this pattern is used for classification with a pattern-based classifier. This proposal can obtain cost-sensitive patterns, which leads to lower misclassification costs in comparison to patterns mined by well-known state-of-the-art pattern miners. In the approach of (Loyola-González et al. , 2019), the cost matrices are adjusted to C(0,0) C(1,1) 0, which means that the resulting costs of True Negative (TN) and TP are 0. The costs of FP are set consecutively to C(0,1) 2, 5, 10, 20 and further propose to use the imbalance ratio of training database as a cost. The costs of False Negative (FN) is set to C(1,0) 1. To control the results from the classifier, they use a normalized expected cost (NEC) (normalized misclassification cost  (Drummond & Holte, 2006) ), here given in the original full form: (1) Dp and Dn are the numbers of instances from the minority (Dp) and majority class (Dn).
Dataset and class distribution Table  1  shows the used datasets and the class distribution. Glass1 and Yeast-0-2-5-6  (Alcalá-Fdez et al. , 2011)  are unbalanced open-source binary datasets and we use them to check the quality of a classifier with an unbalanced dataset. UCI SECOM  (Dua & Graff, 2020 ) is a real-world dataset and is more relevant for us than Glass1 or Yeast-0-2-5-6 because it contains data from a semi-conductor manufacturing process. The features from the UCI SECOM dataset are e. g. , signals or variables collected from sensors and process measurement points in the manufacturing process  (Dua & Graff, 2020). For our purpose's dataset Line A, Line B, Line C, Line X, Line Y and Line Z are especially interesting. These datasets represent a classification of station Ta to station Tb in a production chain from our partner SICK AG (SICK AG, 2020).
Line X is particularly interesting here because it shows a data chain over several test stations with 1071 features. Further, it shows how highly unbalanced the datasets in the real production can be. Dataset Glass1 has here the lowest imbalance ratio with 0. 550725 and Line Z the highest with 0. 002103. Due to the strong imbalance in Line Z, which represent just a few errors in comparison to good products, the results are particularly interesting. All the used datasets contain only numerical features. A failure instance is represented as class 1 or 1 in the dataset and a flawless instance is be represented by class 0 or 0. A row corresponds to an instance and columns represent the features in the dataset. The column'IR (Class 1/Class 0)' in Table  1 , shows how strongly the classes in the dataset are unbalanced.
Test setup (machine hardware, software versions) For our experiments we used a machine with Windows 10 64Bit. The test system has an i9-9999KS (16 × 4 GHz) processor and 64 GB RAM. We used the Anaconda Distribution with Numpy Version: 1. 18. 1, Pandas Version: 1. 0. 1, Scikitlearn Version: 0. 22. 1, Catboost Version: 0. 21, Lightbgm Version:2. 3. 0, Py-xgboost Version 0. 90, HyperOpt Version 0. 2. 2 and Python 3. 7. 6. All experiments shown were executed on the CPU.
Algorithm comparison In this section we visualize the results based on the implementation of twelve algorithms in Figs. 3  and 4. The objective of this comparison is to get an overview, how good the classifiers perform with hardly any adjustments. Those results are used as a baseline for the metric comparison. To get comparable results from the different algorithm, we had to adjust the parameters for k-nearest neighbors (KNN) hyperparameter'n_neighbors' 1, Support Vector Machine (SVM) hyperparameter'kernel' linear. Table  2  shows the best and worst achievable classification results regarding the used dataset. The best case is calculated by P *α and the worst case is represented by -N. The value P represents the number of corrupted parts (TP + FN) and the value N the number of good parts (TN + FP) in the test data.
These values are corresponding to the used dataset. To calculate the Metric Quality Index (MQI) we use: The MQI represents a normalized value to compare the results of optimizations. With the MQI, we can compare the results of different metrics in a single visualization or table. The best possible result for this column is 1. 0 and a result above 0. 0 will save costs. We assume that a correctly predicted corrupted product part (TP) is worth 10 times more than a good product that must be tested in a separate test station again costs (FP) (i. e. , the savings enable by a true positive are assumed to be 10 times higher than the cost resulting from a false positive). We set α to 10 but this value must be adjusted for every product. The value 10 is a common recommendation for the cost matrices in other works  (Domingos, 1999; Du et al.
, 2007; Krętowski & Grześ, 2006; Zhang et al. , 2007). The comparison of Figs. 3 and 4 answer the question (Q1) how good the results of established ML algorithms are based on real production data. Table  3  summarizes the results from Figs. 3  and 4. In these two figures, we show which results can be achieved with the different algorithms and get an overview of their performance. As indicated in the introduction, the results have shown that there is no single optimal algorithm. Therefore, it is necessary to compare different algorithms with each other for every dataset. An abnormality in the results can be seen using the MCC and ROC AUC metric. The result from the ROC AUC metric with the Catboost classifier in Line X implies a good result with the value of 0. 814. This would lead us to the assumption that this specific model could be used to predict product errors.
When considering the outcome of the MCC metric, we see that this result is misleading. The value of the MCC metric (-0. 001) seems like a random outcome. We would not use a model that has an MCC value below 0. This effect results from the strong unbalanced classes in the dataset. ROC AUC and PRC AUC are particularly affected by this effect. Therefore, it is necessary to find an adjusted metric for our use case. The Naive Bayes and the Complement Naive Bayes algorithms produce strong negative outliers within the MQI Metric. Further, the SVM algorithm performs poorly over all datasets with most of the metrics. Nevertheless, we show a benefit to use ML in this use case with the results from the MQI metric. Therefore, ML could already be used to predict production errors. Another aspect that can be observed is that the decision tree-based algorithms are often close in terms of results and provide reasonable results.
025. Even more crucial is the comparison in Line Z. With the EBR metric we get a neutral result. On the contrary the MCC metric would increase costs dramatically by a MQI of -2. 888. This example shows the necessity of our EBR and EBRP metrics. Moreover, we show how suitable existing metrics for real production data are (Q2) and present a much better metric for our specific use case. The next point is the needed time to improve the algorithm with hyperparameter tuning. Table  5  shows the average time for a program execution with the associated dataset. Ran-domSearchCV is the fastest competitor in this comparison, but the needed time for an execution with HyperOpt is still acceptable regarding the cost savings that result from the optimization. The maximum amount of time for an execution took average 50 min in Table  5.
This is due to the high number of features in dataset X. Nevertheless, the measured times are close to each other. The visualized results in Table  6  also hold further information with regards to the reusability of hyperparameters. We take a closer look at the results of Line B, Line C, Line X and UCI Secom from the EBRP Metric in Table  4. These datasets created the best results using the Random Forest Classifier. The question rises if the same classifier has used the same hyperparameters to produce the result. Table  6  shows the used hyperparameters for the mentioned datasets: The used hyperparameters shown in Table  6  indicate that it is necessary to optimize the hyperparameters individually   for each ML model, as our experiments do not show a set of parameters that is optimal across different cases.
A comparison of different winner parameter settings for other datasets could be research in the future. In Figs. 5  and 6  we illustrate all executions with all five metrics compared by the MQI, which are distinguished by different seeds for the run execution. The illustrations are divided by the two optimization approaches. With these illustrations we show the variance of the results of all eight run executions with the different optimization approaches. Further, we show in each visualization as baseline the results of the algorithm comparison from Fig. 4  (MQI metric) with an orange boxplot as sixth result. Based on the characteristic of the datasets Line Z and Line X we can derive some further information. Line Z has the most unbalanced dataset and this is reflected in the results. Nevertheless, we achieved an EBR Result which is slightly better than 0.
Line X has the highest number of features and for this dataset, we can also achieve an EBR result above 0. This proves to us that real world data from manufacturing with a high number of features can achieve usable results. In 12 out of 18 experiments based on the cost-oriented metrics (EBR & EBRP) RandomSearchCV shows a better mean MQI value of all datasets compared to HyperOpt, whereas it is the other way round for 6 out of 18 experiments. In Fig. 7  we show that, RandomSearchCV indicate a slight advantage, leading to a preferable use for manufacturing data. In all three illustrations we use a vertical line to show where a new dataset has begun and to highlight our EBR Metric. For these visualizations we had to adjust the y-axes to the maximal reachable result 1. 0 and the threshold 0. 0.
Some metrics performed below the threshold of 0. 0. For the  An interesting point is that the PRC metric performed worse than expected even though this metric should be favourable for imbalanced datasets. Finally, in Table  7  we compare the cost savings from the optimized results in Figs. 5  and 6  with the worst, best and mean results from Fig. 4  with the MQI metric. We took the worst, best and mean results based on the MQI metric of all run executions in Figs. 5  and 6. In Table  7  we find some interesting information about the results. First, the best results outperform the algorithm results from Fig. 4. Secondly, we can get a worse result through the optimization, due to overfitting of the model. However, nearly all results are still in a positive range or at least neutral EBR value, except Line A.
The authors propose an algorithm to identify breast interval cancer (IC) from the French administrative healthcare database (SNDS) and compare them to those of women whose cancer was diagnosed through the organized screening programme or outside the organized screening programme. The care pathways of women with IC are intermediate with respect to those of the other groups. Background: Since 2004, an organised screening programme (OS) for breast cancer has been in place for 50-74 years women who are not at an increased risk. Despite this, 17% of cancers diagnosed within 24 months following an OS mammogram are interval cancers (IC), diagnosed even though the OS had not reported cancer. After identifying IC from the French administrative healthcare database (SNDS), our objective was to describe the care pathways of women with IC in 2016.
Data source and data The study was conducted using data from the French administrative healthcare database (SNDS), medico-administrative data covering the entire French population. 31 It contains information from public and private healthcare facilities (PMSI), non-hospital care consumption, and death certificates in France. The care consumption over the 2014-2017 period of the women included was retrieved, on one hand, to obtain all breast imaging tests (mammograms, ultrasound, MRI and CT scans of the breast) carried out in the 24 months prior to diagnosis, and, on the other, chart their care pathways (breast conservative surgery (BCS) or mastectomy, chemotherapy, radiotherapy) over the 12 months following the first anticancer treatment.