dataesr/scientific-paragraphs-categorization huggingface dataset:
Global CVIIM As outlined above, the value of CVIIM s,n,K obviously depends on the specific dataset. For a general assessment of the bias attributable to a specific step we need a more global measure summarizing the results obtained on several datasets. To this end we define the global CVIIM as the quantity resulting when replacing E[ e incompl,K (S)] and E[ e full,K (S)] in (1) by, roughly speaking, their means over the universe of datasets from the area of interest (see  [21]  for a more formal description of this concept in another context). Consider the following example: at this time the standard approach in microarray data analysis is to perform quantile normalization of RMA on the whole dataset before performing CV. Suppose that the prediction error is, on average, 0. 2 over all datasets from the area of interest, but if full CV were performed with respect to quantile normalization it would equal 0.
201. Then the global CVIIM in this scenario would be 1 -0. 2/0. 201 ∼ 0. 005, a negligibly weak overall bias. To estimate the global CVIIM we suggest the plug-in estimator obtained by replacing e incompl,K (s) and e full,K (s) in Eq. (  2 ) by the averages of their values obtained on several datasets from the considered area of application: CVIIMglobal s (1) ,. ,s (L) ;K := ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ 1 - 1 L L l=1 e incompl,K (s (l) ) 1 L L l=1 e full,K (s (l) ) if 1 L L l=1 e incompl,K (s (l) ) < 1 L L l=1 e full,K (s (l) ) and 1 L L l=1 e full,K (s (l) ) > 0 0 o t h e r w i s e , where s (1) ,. , s (L).
Illustration To give a first illustration of the application of CVIIM as a proof of concept, we apply it to supervised variable selection, which is expected to yield high CVIIM values. We use the datasets ProstatecTranscr, GSE33205, GSE20189 and GSE37356, which are also considered in the PCA study; see Table  1. For each variable a two-sample t-test is conducted to test the equality of the means of the two groups. The variables with the smallest p-values are selected. Because it is expected that the result substantially depends on the number of selected variables, the analysis is repeated for different numbers of variables: 5, 10, 20 and half of the total number p of variables. After selecting 5, 10 and 20 variables we use LDA as a classification method. When selecting half of the variables LDA cannot be applied, because the involved empirical covariance matrices are not well-behaved in general when the number of variables is higher than the number of observations.
In this case, we use Diagonal Linear Discriminant Analysis, i. e. LDA under the simplifying assumption that within the two classes the variables are independent; see Hastie et al. [10]. In all analyses performed in the paper, e incompl,K (s) and e full,K (s) are obtained by averaging the results from B = 300 runs of K-fold CV, where K takes the values 3, 5 and 10 successively. The CVIIM s,n,K -values obtained for all settings are displayed in Fig. 1  While CVIIM s,n,K is especially large for small numbers of selected variables, relatively large values are also observed when half of the variables are selected (with the exception of the dataset with the fewest variables). Although the differences in CVIIM s,n,K for the selection of 5, 10 and 20 variables are not large, the estimates of the global CVIIM given in Table  2  indicate that the bias induced by incomplete CV tends to decrease with an increasing number of selected variables.
Dataset 30 stands out through its noticeably larger CVIIM s,n,K -values in all plots. This dataset comprises only 36 observations but 47,231 variables (see Table  1 ), which may at least partly explain the larger values. Extreme values above 0. 9, however, are surprising. In this illustrative analysis, through our new measure CVIIM we have confirmed the conclusion previously obtained in the literature: performing supervised variable selection before CV leads to a strong bias of the resulting error estimate.
Study design The investigation of normalization is based on the first 12 microarray datasets listed in Table  1. We use the two variants of normalization described in the section (Addon) normalization. Two different classification methods are used successively to derive prediction rules: NSC and Linear Discriminant Analysis performed on Partial Least Squares components (PLS-LDA). For NSC the shrinkage intensity is chosen from the grid {0. 05, 0. 1, 0. 25, 0. 5, 1, 1. 5} and for PLS-LDA the number of components n comp is chosen from the grid {1, 2,. , 10}. Parameter choice is done in the following way. For each considered training dataset, we perform 3-fold internal CV for each candidate parameter value from the grid. The candidate parameter value yielding the smallest 3-fold CV error is selected.
The study on PCA is based on the last 20 microarray datasets listed in Table  1. The constructed principal components are used as predictors in Linear Discriminant Analysis (LDA) and Random Forest (RF), successively. For RF, the crucial parameter mtry, denoting the number of predictors considered as candidates in the splits of the trees, is chosen by 3-fold internal CV from the grid {1, 2, 3, 5, 10}. Since the results can be assumed to strongly depend on the number of principal components used as predictors, we repeat the analyses for four different numbers: 2, 5, 10 and 15. The latter are also given in Table  3. For both normalization approaches we observe very small CVIIM s,n,K -values for all datasets and both classifiers. In the majority of cases the measure estimates suggest no bias resulting from incomplete CV for normalization as defined by our rule of thumb.
Normalization For the individual datasets there is no visible dependency of the measure estimates on K, although in general we expect a negative dependency; see the section Further issues for a discussion of this topic. The fact that we do not observe such a decrease with K for normalization can likely be explained by the small values of the estimates: e incompl,K (s) and e full,K (s) are very similar here. Therefore the non-systematic fluctuations across the different K-values are attributable to small-probably randomfluctuations of e incompl,K (s) and e full,K (s) over K, which could overshadow a potential dependency on K. In contrast to the section Illustration, we do not present iteration-based error bars for the individual CVIIM s,n,Kvalues here. When depicting the results of a study with a larger number of datasets individual error bars make the corresponding plots increasingly unclear.
Principal component analysis Figure  3  and Table  4  show the results of the PCA study. Note that the scale of Fig. 3  is much larger than that of the corresponding plot for normalization (Fig. 2 ). Globally the results suggest a weak but existent underestimation of the true error E[ e full,K (S)] by performing PCA before CV. Exceptions are LDA in those instances where the number of components is greater than five, where zero values of the global CVIIM-estimates are obtained. For LDA the impact of incomplete CV seems to diminish with an increasing number of components in PCA. The global CVIIM-estimates are in general larger for RF than for LDA. While the overall effects of performing PCA before CV seem to be weak, Fig. 3  reveals that there are several settings in which the CVIIM-estimates suggest a strong bias-according to our rules of thumb-for a non-neglible number of datasets.
Therefore, these results strongly suggest the use of full CV over incomplete CV with respect to PCA. A closer look at  Motivated by this observation we performed a simple type of sensitivity analysis. First for each of the two settings we left out the dataset which displayed the largest difference between e full,3 (s) and e full,10 (s) and reestimated the global CVIIM-values. For the LDA with ncomp = 2 the results were 0. 0812 (K = 3), 0. 0681 (K = 5) and 0. 0524 (K = 10), and for RF with ncomp = 10 we obtained 0. 0590 (K = 3), 0. 0351 (K = 5) and 0. 0222 (K = 10). The values are obviously more similar across the three different K-values for both settings compared to the results obtained when using all 20 datasets; see again Table  4. This is especially noticeable in the case of the values for K = 5 and K = 10 in "LDA with ncomp = 2".
Nevertheless there are still significant differences. Therefore, as a second step we repeated the same procedure, this time however leaving out the three datasets with the largest differences between e full,3 (s) and e full,10 (s). The results were: 0. 0676 (K = 3), 0. 0575 (K = 5) and 0. 0499 (K = 10) for LDA with ncomp = 2, and 0. 0067 (K = 3), 0. 0000 (K = 5) and 0. 0000 (K = 10) for RF with ncomp = 10. For the former setting the similarity across K-values has obviously increased, while at the same time the sizes of the values have not decreased strongly. The (almost) zerovalues for the second setting are quite striking given that we observed values as high as 0. 0907 for K = 3 when using all 20 datasets. We also performed the same analysis for all other settings (results not shown): the global CVIIMestimates in these settings tended to be more robust to the removal of datasets than the ones of the settings presented here.
Alternative measures of CV incompleteness An important question with respect to the definition of CVIIM is whether it depends on E[ e full,K (S)]. Such a dependence is not desirable, since CVIIM should not be a measure of the error but of the impact of CV incompleteness. To investigate this in the context of the PCA study, we plot CVIIM s,n,K against e full,K (s) in the upper panel of Fig. 4 , where the different analysis settings for a given dataset are represented using the same colour and number, and the mean of each dataset is displayed as a black point. This plot suggests no relevant dependency of CVIIM s,n,K on the full CV error e full,K (s). For two of the smallest errors we observe extreme CVIIMestimates, resulting from random fluctuations in the error estimates as discussed in Appendix B.
3 (Additional file 2). However, this problem-concerning only two values out of 480 error values in total-seems to be negligible. The lower panel of Fig. 4  displays the zero-truncated difference between e full,K (s) and e incompl,K (s) against e full,K (s). This plot clearly suggests a comparatively strong dependence of the estimates of this measure on the full CV erroras also observed in the results obtained in the simulation study presented in Appendix A (Additional file 2)-and thus provides evidence supporting the use of a ratio-based measure rather than a difference-based measure. Analogous plots give a very similar picture in the case of normalization; see Figure  S6  in Appendix C (Additional file 2). An obvious, but less insightful, way of visualizing the impact of CV incompleteness, is to simply plot e full,K (s) and e incompl,K (s) for the individual datasets.
Outlook: other preparation steps We performed additional analyses for further data preparation steps, although with fewer datasets and fewer analysis settings than in the studies for normalization and PCA. These preparation steps were: optimization of tuning parameters, variable filtering by variance and imputation of missing values. See Appendix D (Additional file 2) for the study designs and detailed results. Here, in general, optimization of tuning parameters was connected with weak, but non-negligible optimistic biases. For variable filtering by variance and imputation of missing values the bias was negligible. Note that, due to the limited number of datasets and analysis settings, the results should not be over-interpreted. Further validation is required before practical guidelines can be formulated with respect to these preparation steps.
Simulation study In addition to the real data studies presented above, we also conducted a simulation study to investigate general statistical properties of CVIIM s,n,K. As the preparation step we used supervised variable selection, which displayed the largest CVIIM s,n,K -values in the real data analyses. The data-driven simulation design uses the ProstatecTranscr dataset and involves 2000 correlated normally distributed predictors. The methods and detailed results are presented in the Appendix A (Additional file 2). Briefly, in the simulations the variance of CVIIM s,n,K as an estimator of CVIIM P,n,K was relatively high and decreased with decreasing CVIIM P,n,K -values. The bias was negligible. When displaying the CVIIM s,n,K -values graphically in the section Illustration we added error bars representing the variability of the (untruncated) CVIIM P,n,K -estimates from individual repetitions of CV.
Further issues In the section Outlook: other preparation steps we used a limited number of datasets in our analyses and noted that the results should thus not be over-interpreted. The results from the normalization and PCA analyses, in contrast, were based on 12 and 20 datasets respectively, and are thus more reliable. As a rule of thumb we recommend using at least 10 datasets for analyses for which the goal is the evaluation of the impact of CV incompleteness for a particular preparation step. However, the number of datasets to consider of course depends on the heterogeneity of the datasets. Quite generally, the variability of the relative performances of different classification methods over different datasets has been found to be large in previous literature  [21, 22]. We can frequently make analogous observations with respect to the distribution of the CVIIM estimates over datasets.
When studying these distributions, we can implicitly also observe variability inherent in individual CVIIM estimates. This variability is probably hard to estimate, given that the estimator involves a fraction of two CV estimates, the variance of which is very difficult to estimate  [23]. In CV the training sets are necessarily smaller than the whole dataset and the CV error estimate is thus an upwardly biased estimator of the error of the prediction rule fit on the whole dataset. This type of bias also affects the relationship between E[ e full,K (S)] and E[ e incompl,K (S)]. Since in E[ e incompl,K (S)] the considered analysis step(s) is/are performed on the whole dataset, the corresponding parameters are estimated more accurately than in E[ e full,K (S)] due to the difference in sample sizes.
This leads to a greater upward bias of e full,K (s) compared to e incompl,K (s) with respect to the prediction error of the prediction rule fit on the whole dataset. This can occasionally result in increased CVIIM s,n,K values. A strong decrease of the CVIIM estimates with increasing value of K is an indication of the presence of this problem. This is because for increasing K the size of the training sets gets closer to the full sample size, thereby diminishing the additional upward inherent bias of e full,K (s). In most of our analyses we observed no substantial dependence on K. We nevertheless recommend estimating CVIIM for several values of K as a form of sensitivity analysis, as done in our analyses. For larger datasets the result of any preliminary step is expected to be more stable, and in fact results approach being deterministic as the sample size tends to infinity.
Therefore with larger sample sizes the result of a preliminary step will be less affected when it is conducted on the whole dataset compared to the correct separation of training and test data. Thus CVIIM depends negatively on the sample size. In Figures S10, S11 and S12 in Appendix E (Additional file 2) for each investigated preparation step we plotted the dataset-specific means of the CVIIM-estimates over all respective settings against the sample sizes of the datasets. Here we clearly observe such a dependency: for large datasets (n ∼ 100) the CVIIMestimates were much smaller in most cases. This was also observed in the simulation study. In practice, data preparation often consists of a combination of several preliminary steps, often with a natural ordering. For example, normalization of microarray data has to be performed before variable selection.
(  1 ) by alternative error measures. The only requirement is that the loss function associated with the respective error type has positive range. Most common loss functions fulfill this requirement, for example the quadratic or absolute loss for linear regression, the integrated Brier score for survival data, the check function in the case of quantile regression or the negative log-likelihood as an alternative to the error rate when the response variable is discrete. Note again that CV provides dataset-internal error estimation. Consequently it estimates the error expected on observations which follow the same distribution as the training data. When a different dataset is used for evaluating the prediction rule-as done in external validationthe error can be expected to be higher  [24]. CV can be used in the process of obtaining an adequate prediction rule when no external data is available, but before ultimately applying a prediction rule in medical practice it must be externally validated  [25, 26].
Conclusions In conclusion, the empirical study using our new measure of CV incompleteness suggests that 1) RMA normalization and RMA normalization in combination with global VSN can be safely performed as preliminary data preparation steps on the whole dataset, since they yielded very small CVIIM-values for all 12 analyzed real datasets; 2) PCA has to be performed anew in each CV iterationi. e. re-trained on each training set-to protect against a potential optimistic bias, since it yielded large CVIIM values in some of the 20 analyzed real datasets. The latter result shows that non-supervised data preparation steps can also lead to over-optimistic error estimation if performed before CV. Given the ubiquitous use of RMA in microarray analysis it is reassuring that the common practice of performing RMA before CV is not harmful.
50 transcription 9 Normalization E-GEOD-39683 20 32321 0. 40 transcription 10 Normalization E-GEOD-40744 20 20706 0. 50 transcription 11 Normalization E-GEOD-46053 20 54675 0. 40 transcription 12 PCA E-GEOD-37582 121 48766 0. 39 transcription 13 PCA ProstatecTranscr 102 12625 0. 49 transcription 14 PCA GSE20189 100 22277 0. 49 transcription 15 PCA E-GEOD-57285 77 27578 0. 45 DNA methyl. 16 PCA E-GEOD-48153 71 23232 0. 48 proteomic 17 PCA E-GEOD-42826 68 47323 0. 24 transcription 18 PCA E-GEOD-31629 62 13737 0. 35 transcription 19 PCA E-GEOD-33615 60 45015 0. 35 transcription 20 PCA E-GEOD-39046 57 392 0. 47 transcription 21 PCA E-GEOD-32393 56 27578 0. 41 DNA methyl. 22 PCA E-GEOD-42830 55 47323 0. 31 transcription 23 PCA E-GEOD-39345 52 22184 0. 38 transcription 24 PCA GSE33205 50 22011 0.
50 transcription 25 PCA E-GEOD-36769 50 54675 0. 28 transcription 26 PCA E-GEOD-43329 48 887 0. 40 transcription 27 PCA E-GEOD-42042 47 27578 0. 49 DNA methyl. 28 PCA E-GEOD-25609 41 1145 0. 49 transcription 29 PCA GSE37356 36 47231 0. 44 transcription 30 PCA E-GEOD-49641 36 33297 0. 50 transcription 31 PCA E-GEOD-37965 30 485563 0. 50 DNA methyl. 32 ArrayExpress accession numbers have the prefix E-GEOD-, NCBI GEO accession numbers have the prefix GSE.
The decrease with larger values of K is induced by the mean of the e full,K (s)-values becoming more similar to the mean of the e incompl,K (s)-values with increasing value of K. For most settings we do not observe a substantial decrease of the global CVIIM-estimates. This suggests that the two cases for which the decrease with K was strong are connected to aberrant results for individual datasets, which was confirmed by more closely inspecting the individual values obtained for each setting and each dataset.
The process of hot bulk forming is simulated with finite element method through ABAQUS calculation code using very fine mesh at contact interface. The main purpose of this simulation is to compare the microscopic level of stress sensitivity according to the geometry of roughness. The paper is structured as follows: in section two, the theory of plastic wave friction is presented. The experimental procedure (hardness test, micrographic study, torsion test) is detailed in section three, the numerical simulation of the bulk forming processes is given in Article published by EDP Sciences section four. The paper ends with concluding remarks and conclusion.
Numerical simulation 4. 1 Procedure The problem similarity allows considering calculation only for one half of the higher punch and a quarter of the billet. Certain processes of bulk forming are carried out at hot to decrease the efforts to be implemented because of temperature conditions and the value of the yield stress σ 0 of deformed material  [12]. The process is known as hot process when the temperature is higher than 0. 5 times the absolute temperature of fusion T f. In this work, a simulation of bulk forming process of a cylindrical workpiece by a conical tool with calculation code using ABAQUS finite elements in thermal state is proposed. The forging of aluminum alloy billets preheated at a temperature of 400 • C, near the melting point is carried out by a conical tool which takes into account the punch roughness.
The use of crawlers as hybrids between mobile ROVs and benthic observatories Mobile Internet Operated Vehicles (IOVs) for seabed surveys, such as the Deep Sea Crawler Wally (http://www. jacobs-university. de/ses/research/oceanlab/crawler; Purser et al. 2013b ) are now available for use from cabled infrastructure nodes (Fig. 1 ). Crawler platforms can carry around a host of multiparametric sensors (including imaging equipment) to locations of interest  (Thomsen et al. 2012 ). The cabled infrastructure allows both a high power supply to the vehicle and a high data flow, allowing HD video capture and high sensor temporal resolution. Crawler may be used to calibrate imaging fish quantifications of cabled observatory fixed camera systems. In the case of coastal platforms, crawlers could be used to carry out day-night and seasonal visual census surveys along transect encompassing different key environment for fish species (e.
Data storage and processing as core of observatories Artificial Intelligence Multiparametric platforms are delivering large, complex, and interrelated biological and environmental data at frequencies never attained before, but our information treatment capacity is presently limited by the lack of automation in processing  (Costa et al, 2011). Artificial Intelligence procedures for statistical treatment of data should be implemented in order to remove the time-costly human factor. Unfortunately, an efficient autonomy in data processing is still far from being implemented. At the core of automated procedures sustaining the Artificial Intelligence there are three important issues: i. video-imaging for species classification and animal counting; ii. multiparametric data pre-processing, and storage into data banks; and finally, iii.
g. , Sir Alister Hardy Foundation for Ocean Science; http://www. sahfos. ac. uk/pil/plankton_image_database_homepage. htm). Specific groups of observers, such as those involved in recreational SCUBA diving, could be enrolled in remote species video-recognition endeavours as trained volunteers (e. g. Azzurro et al. 2013). These operators may propose images depicting animals, according to certain imposed criteria (e. g. best visibility). After experts evaluate their input (see Figure  3 ), these images may be introduced into the library and their morphological indicators may be extracted and used along with those already present for morphological classification. Routines for validation of classification choices by public users could be also automated to a certain extent by using the consensus criterion (i.