dataesr/scientific-paragraphs-categorization huggingface dataset:
More generally, machine-learning systems may fail on data from different imaging devices, hospitals, populations with a different age distribution, and so forth. Dataset biases are in fact frequent in medicine. For instance selection biases-e. g. , due to volunteering self-selection, non-response, or dropout  [8, 9] -may cause cohorts to capture only a small range of possible patients and disease manifestations in the presence of spectrum effects  [10, 11]. Dataset shift or dataset bias can cause systematic errors that cannot be fixed by acquiring larger datasets and require specific methodological care. In this article, we consider predictive biomarkers identified with supervised machine learning. We characterize the problem of dataset shift, show how it can hinder the use of machine learning for health applications  [12, 13] , and provide mitigation strategies.
It can be important to distinguish the source data, used to fit and evaluate a machine-learning model (e. g. , a dataset collected for research), from the target data, on which predictions are meant to be used for clinical applications (e. g. , new visitors to a hospital). Indeed, if the training examples are not representative of the target population-if there is a dataset shift-the empirical risk is a poor estimate of the expected error, and f will not perform well on individuals from the target population.
This process is usually repeated with several splits, a procedure called cross-validation (  [15, Sec. 7, 17] ). When training and test examples are chosen uniformly from the same sample, they are drawn from the same distribution (i. e. , the same population): there is no dataset shift. Some studies also measure the error on an independent dataset (e. g. ,  [18, 19] ). This helps establish external validity, assessing whether the predictor will perform well outside of the dataset used to define it  [20]. Unfortunately, the biases in participant recruitment may be similar in independently collected datasets. For example if patients with severe symptoms are difficult to recruit, this is likely to distort all datasets similarly. Testing on a dataset collected independently is therefore a useful check but no silver bullet to rule out dataset shift issues.
"Deconfounding" does not correct dataset shift for predictive models Dataset shift is sometimes confused with the notion of confounding because both settings arise from an undesired effect in the data. Confounding comes from causal analysis, estimating the effect of a treatment-an intervention, sometimes fictional-on an outcome. A confounder is a third variable-e. g. , age or a comorbidity-that influences both the treatment and the outcome. It can produce a non-causal association between the two (see  [21] , Chap. 7, for a precise definition). However, the machine-learning methods that we consider here capture statistical associations but do not target causal effects. Indeed, for biomarkers, the association itself is interesting, whether causal or not. Elevated body temperature may be the consequence of a condition but also cause a disorder.
It is a clinically useful measure in both settings. Tools for causal analysis are not all useful for prediction, as pointed out by seminal textbooks: "if the goal of the data analysis is purely predictive, no adjustment for confounding is necessary [. ] the concept of confounding does not even apply. " (  [21] , Sec. 18. 1), or Pearl  [22]. In prediction settings, applying procedures meant to adjust for confounding generally degrades prediction performance without solving the dataset shift issue. Figure  1  demonstrates the detrimental effect of "deconfounding" on simulated data: while the target population is shifted due to a different age distribution, removing the effect of age also removes the separation between the 2 outcomes of interest. The same behavior is visible on real epidemiologic data with age shifts, such as predicting the smoking status of participants in the UKBiobank study  [23] , as shown in Fig.
2. Drawing training and testing samples with different age distributions highlights the effect of these age shifts on prediction performance (see Appendix B, "Tobacco smoking prediction in the UKBiobank" for details on the procedure). For a given learner and test population, training on a different population degrades prediction. For example, predictions on the old population are degraded when Figure  1 : Classification with dataset shift-regressing out a correlate of the shift does not help generalization. The task is to classify patients (orange) from healthy controls (blue), using 2D features. Age, indicated by the shade of gray, influences both the features and the probability of disease. Left: Generative process for the simulated data. Age influences both the target Y and the features X, and Y also has an effect on X.
Between the source and target datasets, the distribution of age changes. The 2 arrows point towards increasing age and represent the Healthy and Diseased populations, corresponding to the orange and blue clouds of points in the right panel. The grayscale gradient in the arrows represents the increasing age of the individuals (older individuals correspond to a darker shade). Throughout their life, individuals can jump from the Healthy trajectory to the Diseased trajectory, which is slightly offset in this 2D feature space. As age increases, the prevalence of the disease increases, hence the Healthy trajectory contains more individuals of young ages (its wide end) and fewer at older ages (its narrow end)-and vice versa for the Diseased trajectory. Right: Predictive models. In the target data (bottom row), the age distribution is shifted: individuals tend to be older.
Elderly individuals are indeed often less likely to participate in clinical studies  [24]. First column: No correction is applied. As the situation is close to a covariate shift (see Section "Covariate shift"), a powerful learner (RBF-SVM) generalizes well to the target data. An over-constrained model-Linear-SVM-generalizes poorly. Second column: Wrong approach. To remove associations with age, features are replaced by the residuals after regressing them on age. This destroys the signal and results in poor performance for both models and datasets. Third column: Samples are weighted to give more importance to those more likely in the target distribution. Small circles indicate younger individuals, with less influence on the classifier estimation. This reweighting improves prediction for the linear model on the older population.
Whiskers represent the rest of the distribution, except for outliers, defined as points beyond 1. 5 times the IQR past the low and high quartiles, and represented with diamond fliers. the model is trained on the young population. A flexible model (gradient boosting) outperforms the linear model with or without dataset shift. "Regressing out" the age (as in the second column of Fig. 1 , "+ regress-out" strategy in Fig. 2 ) degrades the predictions in all configurations. For both illustrations on simulated and real data (Fig. 1  and 2), we also demonstrate an approach suitable for predictive models: reweighting training examples giving more importance to those more likely in the test population. This approach improves the predictions of the overconstrained (misspecified) linear model in the presence of dataset shift but degrades the predictions of the powerful learner.
Training examples should not be selected to be homogeneous To obtain valid predictive models that perform well beyond the training sample, it is crucial to collect datasets that represent the whole population and reflect its diversity as much as possible  [6, 25, 26]. Yet clinical research often emphasizes the opposite: very homogeneous datasets and carefully selected participants. While this may help reduce variance and improve statistical testing, it degrades prediction performance and fairness. In other words, the machine-learning system may perform worse for segments of the population that are under-represented in the dataset, resulting in uneven quality of care if it is deployed in clinical settings. Therefore in predictive settings, where the goal is machine-learning models that generalize well, large and diverse datasets are desirable.
Simpler models are not less sensitive to dataset shift Often, flexible models can be more robust to dataset shifts, and thus generalize better, than linear models  [27] , as seen in Figs  1  and 2. Indeed, an over-constrained (ill-specified) model may only fit well a restricted region of the feature space, and its performance can degrade if the distribution of inputs changes, even if the relation to the output stays the same (i. e. , when covariate shift occurs, see Section "Covariate shift" ). Dataset shift does not call for simpler models because it is not a small-sample issue. Collecting more data from the same sources will not correct systematic dataset bias.
Preferential Sample Selection: A Common Source of Shift In 2017, competitors in the million-dollar-prize Data Science Bowl  [28]  used machine learning to predict whether individuals would receive a diagnosis of lung cancer within 1 year, on the basis of a computed tomographic (CT) scan. Assuming that the winning model achieves satisfying accuracy on left-out examples from this dataset, is it ready to be deployed in hospitals? Most likely not. Selection criteria may make this dataset not representative of the potential lung cancer patients general population. Selected participants verified many criteria, including being a smoker and not having recent medical problems such as pneumonia. How would the winning predictor perform on a more diverse population? For example, another disease could present features that the classifier could mistakenly take for signs of lung cancer.
Beyond explicit selection criteria, many factors such as age, ethnicity, or socioeconomic status influence participation in biomedical studies  [24, [29] [30] [31]. Not only can these shifts reduce overall predictive performance, they can also lead to discriminative clinical decisions for poorly represented populations  [32] [33] [34] [35] 80]. The examples above are instances of preferential selection, which happens when members of the population of interest do not have equal probabilities of being included in the source dataset: the selection S is not independent of (X, Y). Preferential sample selection is ubiquitous and cannot always be prevented by careful study design  [36]. It is therefore a major challenge to the identification of reliable and fair biomarkers. Beyond preferential sample selection, there are many other sources of dataset shifts, e.
g. , population changes over time, interventions such as the introduction of new diagnostic codes in Electronic Health Records  [37] , and the use of different acquisition devices. [40]. Y is the lesion volume to be predicted (i. e. , the output). M are the imaging parameters, e. g. , contrast agent dosage. X is the image, and depends both on Y and M (in this toy example X is computed as X := Y + M + , where is additive noise). S indicates that data are selected to enter the source dataset (orange points) or not (blue points). The symbol ⊥ ⊥ means independence between variables. Preferentially selecting samples results in a dataset shift (middle and bottom row). Depending on whether Y⊥ ⊥ S | X, the conditional distribution of Y | X-here lesion volume given the image-estimated on the selected data may be biased or not.
The selection mechanism influences the type of dataset shift The correction for a dataset shift depends on the nature of this shift, characterized by which and how distributions are modified  [27]. Knowledge of the mechanism producing the dataset shift helps formulate hypotheses about distributions that remain unchanged in the target data (  [38, 39] , Chap. 5). Figure  3  illustrates this process with a simulated example of preferential sample selection. We consider the problem of predicting the volume Y of a tumor from features X extracted from contrast CT images. These features can be influenced not only by the tumor size but also by the dosage of a contrast agent M. The top panel of Fig. 3  shows a selection of data independent of the image and tumor volume: there is no dataset shift. In the second panel, selection depends on the CT image itself (e.
g. , images with a low signal-to-noise ratio are discarded). As selection is independent of the tumor volume Y given the image X, the distribution of images changes but the conditional distribution P (Y | X) stays the same: we face a "covariate shift" (see Section "Covariate shift"). The learned association remains valid. Moreover, reweighting examples to give more importance to those less likely to be selected can improve predictions for target data (Section "Importance Weighting"), and it can be done with only unlabeled examples from the target data. In the bottom panel, individuals who received a low dose of contrast agent are less likely to enter the training dataset. Selection is therefore not independent of tumor volume (the output) given the image values (the input features). Therefore we have sample selection bias: As these examples illustrate, the causal structure of the data helps identify the type of dataset shift and what information is needed to correct it.
Importance Weighting: A Generic Tool Against Dataset Shift Importance weighting is a simple approach to dataset shift that applies to many situations and can be easy to implement. Dataset shift occurs when the joint distribution of the features and outputs is different in the source (data used to fit the machine-learning model) and in the target data. Informally, importance weighting consists in reweighting the available data to create a pseudo-sample that follows the same distribution as the target population. To do so, examples are reweighted by their "importance weights"-the ratio of their likelihood in target data over source data. Examples that are rare in the source data but are likely in the target data are more relevant and therefore receive higher weights. A related approach is "importance sampling"resampling the training data according to the importance weights.
Many statistical learning algorithms-including Support Vector Machines (SVM), decision trees, random forests, and neural networks-naturally support weighting the training examples. Therefore, the challenge lies mostly in the estimation of the appropriate sample weights and the learning algorithm itself does not need to be modified. To successfully use importance weighting, no part of the target distribution should be completely unseen. For example, if sex (among other features) is used to predict heart failure and the dataset only includes men, importance weighting cannot transform this dataset and make its sex distribution similar to that of the general population (Fig. 4 ). Conversely, the source distribution may be broader than the target distribution (as seen, e. g. , in Fig. 1 ). Importance weights can also be applied to validation examples, which may produce a more accurate estimation of generalization error on target data.
Importance weighting is a well-known approach, and an important body of literature focuses on its application and the estimation of importance weights. It is illustrated on small datasets for the prediction of breast cancer in Dudík et al. [42]  and heart disease in Kouw and Loog  [43]. However, it cannot always be applied: some knowledge of the target distribution is required, and the source distribution must cover its support. Moreover, importance weighting can increase the variance of the empirical risk estimate, and thus sometimes degrades performance, as seen in Fig. 2. It is therefore a straightforward and popular approach to consider but not a complete solution. It is particularly beneficial when using a simple learning model that cannot capture the full complexity of the data, such as the linear models in Fig.
Other Approaches to Dataset Shift Beyond importance weighting, many other solutions to dataset shift have been proposed. They are typically more difficult to implement because they require adapting or desiging new learning algorithms. However, they may be more effective, or applicable when information about the target distribution is lacking. We summarize a few of these approaches here. A more systematic review can be found in Kouw and Loog  [43]. Weiss et al. [44]  and Pan and Yang  [45]  give systematic reviews of transfer learning (a wider family of learning problems that includes dataset shift). The most obvious solution is to do nothing, ignoring the dataset shift. This approach should be included as a baseline when testing on a sample of target data, which is a prerequisite to clinical use of a biomarker  [12, 27].
With flexible models, this is a strong baseline that can outperform importance weighting, as in the right panel of Fig. 2. Another approach is to learn representationstransformations of the signal-that are invariant to the shift  [46]. Some deep-learning methods strive to extract features that are predictive of the target while having similar distributions in the source and target domains (e. g. ,  [47] ), or while preventing an adversary from distinguishing source and target data ("domain-adversarial" learning, e. g. ,  [48] ). When considering such methods, one must be aware of the fallacy shown in Fig. 1 : making the features invariant to the effect driving the dataset shift can remove valuable signal if this effect is not independent of the outcome of interest. It may also be possible to explicitly model the mapping from source to target domains, e.
Performance heterogeneity and fairness It can be useful not to target a specific population but rather find a predictor robust to certain dataset shifts. Distributionally robust optimization tackles this goal by defining an ambiguity, or uncertainty set-a set of distributions to which the target distribution might belong-then minimizing the worse risk across all distributions in this set (see  [51]  for a review). The uncertainty set is often chosen centered on the empirical (source) distribution for some divergence between distributions. Popular choices for this divergence are the Wasserstein distance, f-divergences (e. g. , the KL divergence)  [52] , and the maximum mean discrepancy  [53]. If information about the target distribution is available, it can be incorportated in the definition of the uncertainty set.
Multi-site datasets Often datasets are collected across several sites or hospitals, or with different measurement devices. This heterogeneity provides an opportunity to train models that generalize to unseen sites or devices. Some studies attempt to remove site effects by regressing all features on the site indicator variable. For the same reasons that regressing out age is detrimental in Fig. 1 , this strategy often gives worse generalization across sites. Data harmonization, such as compensating differences across measurement devices, is crucial but remains difficult and cannot correct these differences perfectly  [57]. Removing too much intersite variance can lead to loss of informative signal. Rather, it is important to model it well, accounting for the 2 sources of variance, across participants and across sites.
Covariate shift Covariate shift occurs when the marginal distribution of X changes between the source and target datasets [i. e. , p t (x) = p s (x)] but P (Y | X) stays the same. This happens, e. g. , in the second scenario in Fig. 3 , where sample selection based on X (but not Y) changes the distribution of the inputs. If the model is correctly specified, an estimator trained with uniform weights will lead to optimal predictions given sufficient training data (prediction consistency  [61] , Lemma 4). However the usual (unweighted) estimator is not consistent for an over-constrained (misspecified) model. Indeed, an over-constrained model may be able to fit the data well only in some regions of the input feature space (Fig. 1 ). In this case reweighting training examples (Section "Importance weighting: a generic tool against dataset shift") to give more importance to those that are more representative of the target data is beneficial  [27, 38].
Prior probability shift Another relatively simple case of dataset shift is "prior probability shift. " With prior probability shift (aka label shift or target shift), the distribution of Y changes but not P (X | Y). This happens for example when disease prevalence changes in the target population but manifests itself in the same way. Even more frequently, prior probability shift arises when 1 rare class is overrepresented in the training data so that the dataset is more balanced, as when extracting a biomarker from a case-control cohort, or when the dataset is resampled as a strategy to handle the "class imbalance" problem  [62]. Prior probability shift can be corrected without extracting a new biomarker, simply by adjusting a model's predicted probabilities using Bayes' rule (as noted, e.
Conclusion Ideally, machine-learning biomarkers would be designed and trained using datasets carefully collected to be representative of the targeted population-as in Liu et al. [63]. To be trusted, biomarkers ultimately need to be evaluated rigorously on 1 or several independent and representative samples. However, such data collection is expensive. It is therefore useful to exploit existing datasets in an opportunistic way as much as possible in the early stages of biomarker development. When doing so, correctly accounting for dataset shift can prevent wasting important resources on machine-learning predictors that have little chance of performing well outside of 1 particular dataset. We gave an overview of importance weighting, a simple tool against dataset shift. Importance weighting needs a clear definition of the targeted population and access to a diverse training dataset.
When this is not possible, distributionally robust optimization may be a promising alternative, although it is a more recent approach and more difficult to implement. Despite much work and progress, dataset shift remains a difficult problem. Characterizing its impact and the effectiveness of existing solutions for biomarker discovery will be important for machinelearning models to become more reliable in healthcare applications. We conclude with the following recommendations: r Be aware of the dataset shift problem and the difficulty of out-of-dataset generalization. Do not treat cross-validation scores on 1 dataset as a guarantee that a model will perform well on clinical data. r Collect diverse, representative data. r Use powerful machine-learning models and large datasets. r Consider using importance weighting to correct biases in the data collection, especially if the learning model may be overconstrained (e.
Data Availability The source files used to create this publication can be found in the suppording data in the GigaScience Database  [64]. They are also available in a Git repository  [65]. The UK Biobank data used in this study are Controlled Access Data. To access these data, one would need to visit the UKBiobank website  [66]  and follow the steps for registering, applying for access, and downloading the data. The data used in this study were the "tabular" dataset that is archived in UK Biobank (i. e. , the data that are neither imaging nor genome sequencing). This research has been conducted using the UK Biobank Resource under Application No. 45551. in the support of p t. When this is verified, by applying Bayes' rule the definition of importance weights in Equation (A2) can be reformulated (see  [70] , Sec.