dataesr/scientific-paragraphs-categorization huggingface dataset:
As a response, research in OR mainly focuses on the input side  (Li, 2018)  to protect data, but other aspects could be considered as well. Hence, more research is still needed on the development of privacy-preserving solutions. • New application domains. New application domains are likely to arise as a result of increased data availability, new data sources, new technologies, new industries, and new societal challenges. Domains like sports analytics, health analytics or analytics linked to robotics are likely to become more important. Embedding all aspects of XAIOR within these new applications is an inspiring challenge. Sometimes the domain as such is a driver for a certain dimension of the XAIOR framework. Indeed, using analytics that assists in a modal shift to low-emission transport contributes to RA goals  (De Moor et al.
Conclusions There is an increasing need to explain analytical solutions, originating from expectations of internal and external stakeholders, yet this is not fully captured in the OR literature. Despite some review papers focusing on explainability, existing research falls short in (i) proposing an OR-oriented definition of explainable AI for the operational research domain and (ii) zooming in on specific methods and application requirements. In this paper, we first define and characterize XAIOR, i. e. , explainable AI for Operational Research. Specifically, XAIOR is defined as an interplay of three dimensions, i. e. , performance analytics, attributable analytics, and responsible analytics. We subsequently discuss the implementation of XAIOR across the data analytics pipeline. In particular, we discuss state-of-the-art methodologies for experimental design & data selection, feature engineering & data preparation, algorithmic design & choice, post-hoc interpretation methods, and evaluation strategies & metrics, and we link these with our XAIOR dimensions.
We find that further research is still needed to integrate all XAIOR dimensions, especially AA and RA. In an overview of applications of XAIOR, we discuss prior work on XAIOR and its subdimensions in 6 crucial OR domains. These include forecasting, risk analysis, inventory control, marketing, supply chain management, and other applications. We find that the maturity of PA, AA, and RA depends on the application domain with an under-representation of AA and RA. Based on these overviews, we identify critical avenues for future research linked to five research themes, i. e. , data innovation, deep learning, integration of the XAIOR framework's dimensions and subdimensions, responding to societal changes, and new innovative applications. We propose specific research questions linked to these five research themes and in relation to the XAIOR framework that might inspire researchers to apply and contribute to XAIOR.
Acknowledgements The authors acknowledge all researchers who, through their work, have advocated and accelerated the adoption of (explainable) analytics in OR. The research of Roman S lowiński was supported by  TAILOR , a project funded by the  EU Horizon   2020 (research and innovation funding ) programme ( EC GA  number  952215 ). Richard Weber  acknowledges financial support from  FONDECYT Chile  ( 1221562 ),  Fondef  ( IT23I0061 ),  ANID PIA/PUENTE  ( AFB220003 ), and  NeEDS , a project funded by the  EU Horizon 2020 programme  ( EC GA  number  822214 ).
ARTIFICIAL INTELLIGENCE: MACHINE LEARNING, NEURAL NETWORK AND DEEP LEARNING 1. Machine learning ML is one of the subfields of AI, which detects the underlying links between inputs and outputs to create an automated algorithm  [10]. To develop ML, large datasets are critical for training the algorithm to find complicated patterns and associations faster than traditional statistical models that usually focus only on a small number of variables, with better or, at least, comparable accuracy  [11, 12]. Differences between data analysis and statistical analysis are shown in Table 1  [13]. The principle in ML modeling consists of three pro-2 0 0 0 2 0 0 1 2 0 0 2 2 0 0 3 2 0 0 4 2 0 0 5 2 0 0 6 2 0 0 7 2 0 0 8 2 0 0 9 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 3 2 0 1 4 2 0 1 5 2 0 1 6 2 0 1 7 2 0 1 8 2 0 1 9 2 0 2 0 2 0 2 1 2 0 2 2 2 0 2 3 3,000 4 www.
Neural network and deep learning Neural networks are a subtype of AI mainly inspired by the work of the human brain, which mimics a set of algorithms comprised of four key components: inputs, weights, bias or threshold, and output. The neural network will use a training dataset, then it recognizes the patterns in these data sets and formulates algorithms that can be used to predict the output of a testing data set. The basic operation is given a weight for each input and set a threshold to decide whether the output is true or false. For instance, training data sets of thousands or millions of hand-written data sets are fed into the neural network to recognize hand-written digits. The neural network will recognize the pattern and develop several algorithms that can be used if a small series of hand-written digits are fed.
Decision tree and random forest The mechanisms by which a ML algorithm works could generally be divided into three different classes, namely supervised, unsupervised, and reinforcement learning  [10]. In supervised learning, the input contains the data and the desired result, and the algorithm will detect the relationship, so it can predict the outcome if raw data is provided. A real example of supervised learning is when smartphones are trained to know fingerprints, so they can later identify the correct fingerprint and reject others. However, in unsupervised learning, only the input data is provided. Thus, the algorithm works on its own to discover information such as a pattern/structure in a set of uncategorized data (clustering) or relationships between variables in larger databases (association).
Radiomics can be used for many imaging modalities, including radiography, ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) studies. Radiomics is based on improvements in quantitative image software analysis, applying automated, or semi-automated highthroughput extraction of substantial amounts of quantitative features of medical images. After stratification processes, the obtained digital dataset can be linked to additional genomic or proteomics data, and histopathologic markers obtained by other novel radiographic techniques. Compared to traditional imaging, radiomics provides quantitative information on meaningful biological characteristics and the application of DL, which sheds light on the complete automation imaging diagnosis  [25].
7% from 10 training samples and 160 test samples (80 normal+80 abnormal)  [81]. The SVM approach has also been used in automating the analysis of sperm morphology classification and in selecting spermatozoa for use in IVF techniques  [82]. Human spermatozoa could also be classified using DL methods. For example, CNN could classify spermatozoa into WHO categories based on head shape using freely available sperm head datasets to achieve a 94% true positivity rate  [83].
Natural language processing NLP is part of AI. It is the knowledge that focuses on building a machine or computer's ability to understand human language rather than equations or programs  [73, 84]. NLP was developed in the 1950s at the confluence of linguistics and AI  [85]. Its application in medicine is crucial in analyzing large-scale data, such as physicians' narrative writings in the EMR  [86]. This technology enables machines to extract classifications, such as International Classification of Diseases (ICD) diagnosis codes, from unstructured data, hence enabling physicians to write more naturally rather than having to input specific formatted text or numbers for data analysis  [84]. As with other AI, NLP is a self-learning tool that will increase the utility of its predictions along with the growing number of datasets.
1 (PI-RADS v2. 1) assessment scores as determined by expert radiologists. The PI-RADS v2. 1 is used to standardize the multi-parametric MRI reporting of the prostate. They found that the assessment by ML models outperformed that of the established PI-RADS v2. 1. Others, such as Sanford et al. [115] , reported similar results in detection rates of malignant lesions between AI and qualitative evaluation by an expert radiologist. They created a CNN model of DL and compared it with previous radiology advice with moderate agreement. As ML continues to develop, and as more information is fed into the databases, AI continues to improve. Thus, it is hoped that we may eventually transition to AIassisted precision medicine in the future.
, 2019)  in a continuous time framework, DeepHit  (Lee et al. , 2018)  and PLANN (Biganzoli et al. , 1998)  in a discrete time framework. We applied the methods to a simulation study, enabling us to estimate a coverage rate of the estimated confidence intervals. We also applied them to real-world datasets, and predicted the survival probability for patients with breast cancer and patients with lung cancer. In the simulation study, CoxCC and CoxTime obtained the mean C-indices numerically closest to those from the Oracle model (mean C-index of 0. 723 for CoxCC, 0. 726 for CoxTime, versus 0. 743 for the Oracle model). Regarding the confidence intervals of survival probabilities, Boot with CoxCC obtained a coverage rate of 96. 5%, the closest to the nominal value of 95%. MCDrop was slightly anticonservative and obtained a coverage rate of 89.
Introduction In recent years, machine learning models have been increasingly used in various domains, particularly in medical research. Indeed, in a framework of high dimensional data, these models can discover and identify patterns and relationships between biomarkers for complex datasets and effectively predict future outcomes. Among them are Multi-Layer Perceptrons (MLPs), a type of model developed by Rosenblatt  [1]  that can learn non-linear and complex relationships in patients' data. While neural networks are well-suited for complex datasets, they cannot be directly applied to survival or time-to-event analysis. makes the model interpretable and trustworthy. For instance, the inclusion of this measure can help identify complex diagnostic cases for further evaluation  [8]. Such measure has been developed in other contexts: for classification problems under dataset shift  [9] , for image classification using Bayesian deep neural networks  [10, 11] , or for regression or classification tasks using other machine learning frameworks such as random forests  [12].
Yet few machine learning models have addressed the assessment of predictive uncertainty for time-to-event output. In predictive models, multiple sources of uncertainty exist  [13]. One source comes from the noise in the data, which stems from technical issues involved with data collection and measurement. Another source comes from adversarial examples or dataset shifts that force deep learning models to extrapolate predictions vastly different from the observed data. Together, these two sources are called aleatoric uncertainty and caused by irreducible structural relationships within the data. A second type of uncertainty is epistemic uncertainty, which depends on the model's parameters, hyperparameters, or even the chosen model's underlying structure. It is linked to the estimator. MLPs are estimators that only return point predictions and do not provide a direct measure of uncertainty.
Training procedure of the MLP To evaluate the performances of the different MLPs, we perform a hyperparameter search in the context of a 5-fold cross-validation (CV). More precisely, the dataset is split into a training and a test set for each model. For the simulation study, 2,000 samples were drawn for the train and the test sets. Regarding applications, the two datasets are divided into 80% for the training set and 20% for the test set. A 5fold CV is performed on the train set to search for hyperparameters. Moreover, we use the Tree-Parzen algorithm  [29]  with the package Optuna in Python, which allows us to select hyperparameters iteratively in an informed manner. It is based on a sequential optimization that is more efficient than a simple random search or an exhaustive grid search. We define a search space for each hyperparameter with specific distribution and boundary values.
LASSO obtained a low average coverage rate, a low average C-index, and a high average BS, as shown in Table  B. 8. These results are consistent when we consider that the LASSO model with linear effects is not well suited to a complex dataset that includes multiple interactions and non-linear effects in the data generation process. Then, we investigated the behavior of the methods in terms of bias. In Fig. 1 , we estimated the bias for the MCDrop method. Note that the results are near zero, with the boxplots close to the red dashed line. MCDrop, BMask, and DeepEns obtained results which were close to each other. CoxCC and CoxTime had a negative bias, while DeepHit had the lowest bias. The bias was more variable with the methods applied to PLANN. Finally, we compared the computation times of the methods and MLP models trained in Python.
Cancer cohorts The first example data set we used is the METABRIC (Molecular Taxonomy of Breast Cancer International Consortium) cohort, which can be extracted from the MetaGxBreast R package  [18]. It comprises clinical features and large-scale gene expression data of breast cancer patients obtained at surgery. We used a nonspecific filter independent from the outcome based on standard deviation to increase the statistical power  [33]. We removed genes with an overall standard deviation below the quantile of 95% and retained the 863 genes with the highest standard deviations. The quantile was chosen to obtain approximately 1,000 genes. We selected the probe with the highest variance if multiple probes corresponded to the same gene. 6 clinical covariates were used: age at diagnosis, tumor grade, tumor size, the number of invaded lymph nodes, hormonal therapy, and chemotherapy use.
We removed individuals with missing values for the survival time. Since one or more covariate values were missing for 106 patients, we imputed them using predictive mean matching for numerical features and a multinomial logit model for categorical covariates  [34]. Then, we standardized the numerical covariates using Z-score normalization and applied one-hot  encoding to categorical covariates. The final data set represents 1,960 patients and 869 features, with a median survival time of 88 months and a censoring rate of 54. 6. We applied our methods to a second data set: the Lung Cancer Explorer (LCE). It is an online tool created by UT Southwestern Medical Center's Quantitative Biomedical Research Center, which is composed of more than 6,700 patients over 56 data sets, including the Cancer Genomics Atlas (TCGA) and various Gene Expression Omnibus (GEO) data sets.
All the data sets provide gene expression and clinical data from lung cancer patients. The expression data stems from 23 genomewide platforms and is predominantly composed of microarrays. It is reprocessed, normalized, and converted from probe to gene levels. In our work, we selected 2 clinical covariates: cancer stage and the patient's age. Only the genes with less than 5% of missing values were kept for the gene expression data. Then, the 1,000 genes with the highest standard deviations among the patients were conserved. The missing values were replaced with the multiple imputation method using chained equations. We used Bayesian polytomous regression for the stage categories and predictive mean matching for continuous covariates. We standardized the numerical covariates using Z-score normalization and applied one-hot encoding to the categorical covariates.
Results Here, we present the results obtained on the two cancer cohorts. First, the performances of the different methods are compared. Then, examples of confidence intervals computed at the patient level are presented. On the METABRIC cohort, we compared the results obtained using either all the input covariates or only the clinical covariates. In terms of predictive accuracy (Tables  3  and 4 ), the methods demonstrated higher mean C-indices and lower BS for neural networks fitted only to the clinical covariates, as compared to those fitted to clinical plus molecular data. This highlights that the clinical covariates already contain a large part of the predictive information for survival. Tables  5  and 6  report the mean C-indices and mean BS at 2 years using the LCE. In this cohort, adding the molecular data improved discrimination and accuracy, regardless of the chosen neural network architecture or the combination method.
Thus, in this lung cancer data set, there was additional prognostic information in the molecular data, that was captured by the neural network beyond the clinical covariates. We further analyzed expected survival probabilities from the perspective of advising a clinical practitioner. This was done for two patients from the test set of the METABRIC data set: Patient 1 had a survival time of 3 years and 6 months with no censoring; Patient 2 had a survival time of 5 years and 3 months, and his follow-up time was censored. Fig. 3  represents the density of survival probabilities obtained with the MCDrop method applied on CoxTime. With this method, the mean survival probability at 5 years for Patient 1 (orange line) was 0. 674 with a confidence interval of [0. 525, 0. 849]. Patient 2 had a 0. 910 mean survival probability at 5 years, with a confidence interval of [0.
MCDrop was the second best method after the Bootstrap one, although it was slightly anti-conservative in the simulations, reaching a coverage rate slightly under 95%. Finally, MCDrop performed well in the simulation study and the applications and was less computerintensive than Boot. Therefore, MCDrop may represent a reasonable compromise in terms of coverage with regards of computational time, especially for deep neural networks, and we recommend to combine it with the CoxTime model for survival analysis. In the METABRIC breast cancer cohort, the neural networks had difficulty capturing additional prognostic information from the molecular data, suggesting the clinical variables alone are sufficient. In contrast, in the LCE cohort, the models led to substantially stronger discrimination values when adding molecular data to the clinical variables.
e. , without changing fuzzer internals). We achieve this by making the test objectives deined by these metrics (such as conditions to activate or mutants to kill) explicit as new branches in the target program. Fuzzing such a modiied target is then equivalent to fuzzing the original target, but the fuzzer will also retain inputs covering the additional metrics objectives for mutation. In addition, all the fuzzer mechanisms to penetrate hard-to-cover branches will help covering the additional metrics objectives. We use this approach to evaluate the impact of supporting two ine-grained coverage metrics (multiple condition coverage and weak mutation) over the performance of two state-of-the-art fuzzers (AFL++ and QSYM) with the standard LAVA-M and MAGMA benchmarks. This evaluation suggests that our mechanism for runtime fuzzer guidance, where the fuzzed code is instrumented with additional branches, is efective and could be leveraged to encode guidance from human users or static analysers.
Finally, the performed experiments also suggest that the proposed mechanism to support additional fuzzing objectives, without modifying the fuzzer but by adding new dedicated branches in the PUT, is efective. In addition to encode objectives from ine-grained coverage metrics, this mechanism could be used in other contexts where additional guidance is to be provided to fuzzers at runtime, like with human directives or bug preconditions computed by static analysers. Listing 1. A buggy program checking if an appliance is running above its allowed temperature limit and taking corrective actions if so. void check ( int current_temp , char * data [] ){ if ( current_temp >=50) // Bug : should be current_temp >50 { // Deal with appliance running above the allowed temperature limit. // The bug triggers a detectable crash only when current_temp ==50 // and when rare specific values are present in data } } Contributions.
To sum up, the two main contributions of this paper are: (1) An efective mechanism to guide at runtime the behaviour of existing fuzzers based on branch coverage. This mechanism involves instrumenting the fuzzed code with additional branches, making the fuzzer keep as seeds the inputs that unlock the conditions crafted to guard the entrance of these branches. Implementing this approach notably requires a careful design (which we detail in the paper) of the guarding conditions and branch bodies, to avoid them either breaking the semantics of the instrumented program, or making the fuzzer report spurious crashes, or being tampered by the compilation and harnessing of the program. The proposed instrumentation mechanism can be used to encode objectives from ine-grained coverage metrics where needed in the fuzzed code, but it has also the potential to serve as a lingua franca for any additional guidance to be provided to fuzzers at runtime, like human directives or bug preconditions from static analysers.
For any program state deemed interesting by the guidance but hard to reach by the fuzzer, our approach enables taking advantage for free from all the means already embedded in fuzzers to enter diicult branches; (2) An experimental evaluation of the impact of supporting two representative ine-grained coverage metrics (multiple condition coverage and weak mutation) over the performance of a state-of-the-art grey-box and hybrid fuzzer (AFL++ and QSYM), while fuzzing more than seven hundreds of thousands of lines of code from the standard LAVA-M and MAGMA benchmarks, for more than two years and a half of CPU time. This evaluation reveals that the impact of such metrics is hard to predict before fuzzing and most of the time either neutral or negative. These results suggest thus that ine-grained metrics should not be used as a general means to guide fuzzers.
MOTIVATING EXAMPLE We illustrate now with a simple example how our approach can make state-of-the-art fuzzers support ine-grained coverage metrics out-of-the-box, by transforming the code of the program under test. We also exemplify how this approach could end up making coverage-based fuzzers more eicient at inding bugs, by making them more sensitive in retaining and mutating inputs that trigger diferent PUT behaviours. Our example PUT is the C program presented in Listing 1. It is basically a C function checking if an appliance is running above its allowed temperature limit and taking corrective actions if so. We suppose that there is a bug in the conditional checking that the temperature limit is exceeded (it should be current_temp>50 instead of current_temp>=50), but that this bug only triggers a program crash (enabling a fuzzer to detect it) when some rare values are provided in the data argument (requiring the fuzzer to generate many inputs making current_temp equal to 50 to actually trigger it).
The transformed program can then be fuzzed using an of-the-shelf coverage-based fuzzer relying on branch coverage. When this coverage-based fuzzer will produce an input entering the then branch of one of these new conditionals (and thus killing the corresponding mutant), it will save it for mutation  2. It should be remarked that the second added conditional explicitly forces the fuzzer to maintain and mutate an input where current_temp is equal to 50 as soon as it generates one. This will increase the chance for the fuzzer to trigger a crash revealing the bug, making bug detection faster in average.