## 4 Discussion  
The purpose of this study was to verify the robustness of the methods reported earlier for improving the stability of radiomic features. The study used two different datasets with different pixel sizes acquired in a different center using different scanners and protocols, one for lung cancer and the other for head and neck cancer, both available publicly, while additionally testing for reproducibility and stability in the implementation of radiomic features. High levels of correlation were achieved for more than half of the features for both MATLAB and Pyradiomics implementations for the RIDER lung data, and more than one-third of the features were stable for lung and head and neck datasets, suggesting that some features are agnostic to the disease site and generally robust. There are several characteristics of imaging systems that are relevant to their use in healthcare. These include pixel size and spatial resolution, acquisition parameters (such as kVp in x-rays and imaging sequence in MRI), tomographic reconstruction parameters, patient positioning, and when the image is taken on the patient pathway. These are all important parameters at the start of the process and are determined before radiomics analysis is performed. One of the biggest challenges in radiomics is the low reproducibility of the results from various studies [4,11]. Some of the possible causes for the low reproducibility include pre-existing differences in the dataset used, for example, different acquisition parameters [14,15], reconstruction methods [16,17], pixel sizes [18] and slice thickness; low reproducibility of features due to variations in quantization parameters; and low repeatability of the features [14]. Other considerations include the preprocessing of the imaging data [19]. For instance, [PERSON] et al. studied the effects of image resampling and showed that different resampling approaches produced very different error metrics, with Lanczos interpolation performing substantially better than simple linear interpolation [20]. [PERSON] and [PERSON] have detailed the impact that image quality has on radiomic applications and summarized the minimum image quality requirements and recommendations for reducing the impact of image quality on the reproducibility of radiomic studies [11]. Broadly, for radiomics studies, it is important that all parameter choices are documented and reported,  
Figure 3: Venn diagram with a visual representation of features that are reproducible between RIDER Lung 1 and the Head and Neck (HN1) dataset and with an intensity threshold applied. Reproducibility is measured by an _rs_ value greater than or equal to 0.8. Features highlighted in bold had an rs value greater than 0.9.  
and more specifically, it is good practice that imaging parameters are kept as consistent as possible. [PERSON] suggests quantitative methods/metrics to help determine image quality and/or similarity to recognize comparable images that can be used interchangeably or to decide whether an image's quality is adequate for computing radiomics features [21]. Often, radiomic features identified as predictive are based on small datasets, may be biased toward the specific dataset, and have limited predictive power on another dataset. For other sources of variability affecting radiomics models, readers are directed to some of these studies [19; 21; 22]. The aim of many radiomics studies, including those by our group, is to predict an outcome such as response to treatment or disease-free survival using one or multiple features referred to as biomarkers. Predictions are often performed using statistical approaches, including Kaplan-Meier analysis based on a single feature of the data at a time [23] and machine learning approaches with multiple features from a large set of features up to hundreds [1; 24]. Outcome prediction accuracy is heavily reliant on having highly reproducible features.