dataesr/scientific-paragraphs-categorization huggingface dataset:
• The "nesting" of decision support interactions. In typical decision support interactions, decision support providers and experts with whom they interact often take advantage of, use or refer to various kinds of outcomes of antecedent or parallel decision support interactions: reports produced when trying to solve similar problems in other contexts, tools such as software or databases constructed in other contexts, methodological reports elaborated on the basis of series of similar missions, scientific publications, etc. In so doing, decision support providers take the role of decision makers supported by other decision support providers, who are themselves, for the same reason, decision makers benefiting from antecedent or parallel decision support interactions. These various decision support interactions can take different forms and, typically, the higher up we climb the hierarchy, the less interactive the "interaction" will be.
We finally show the relevance of our deep learning algorithm for each strategy in different realistic screening scenarii by showing its efficiency for each gene that both has a predictable mutational status and is relevant because it has an associated targeted therapy on the market or in development. A previous use of deep learning in a screening strategy were described in Nielsen et al. (13) , but to our knowledge, this is the first screening strategy described for the prediction of mutational status in cancer. Molecular labels. Molecular labels were determined from the masked somatic mutations maf file of somatic mutation using the MuTect2 (  14 ) algorithm corresponding to the dataset. An IMPACT value of "HIGH" or "MODERATE" categorized by VEP software  (15)  was considered positive while other values were considered negative.
Material and Methods Pipeline. All analyses were performed using Python. Image preprocessing. Aperio SVS files of diagnostic slides (labeled by DX in their name) from the 4 datasets were first selected. We then extracted the foreground, then tiled the images in non-overlapping patches of 600x600 pixels. These patches are associated with the label 1 if the gene is muted, 0 if the gene is non muted. These patches and labels were used as inputs of the EfficientNetB7 (  16 ) neural network. Output prediction was a float between 0 and 1 for each patch. For the slide-level scoring, we extracted the 99th percentile of the patch prediction score distribution. Neural network training, model selection. We used an Ef-ficientNetB7 neural network architecture with a single sigmoid output layer after the global average pooling layer.
A significant proportion of mutations are detectable with deep learning methods on whole-slide imaging. We set up a deep learning pipeline predicting solid tumor mutations from WSI based on an EfficientNetB7 (16) pretrained on ImageNet dataset. In this study, 4 datasets from GDC Portal were analyzed, a lung cancer dataset, a colorectal cancer dataset, a breast cancer dataset and an ovarian cancer dataset, respectively TCGA-LUAD, TCGA-COAD, TCGA-BRCA and TCGA-OV. We systematically trained the deep learning pipeline on every mutation having a prevalence greater than 10% in the dataset to focus on mutations that can show statistical significance. These correspond to 922 tested genes. On TCGA-LUAD, TCGA-COAD and TCGA-BRCA, 43% of the gene had a statistically significant predictable mutational status (Figure  1 ).
The TCGA-OV dataset had too few cases to reach statistical significance for most of the genes, therefore we removed it for later analyses. However, we noticed that the TP53 gene had a predictable mutational status with an area under the roc curve (AUC) of 0. 65. Mutations on targetable genes are detectable. Among the most detectable mutations, we found SCN1A in TCGA-LUAD with an AUC of 0. 90, CDH1 in TCGA-BRCA with an AUC of 0. 83 and GRIK2 in TCGA-LUAD with an AUC of 0. 82 (Figure  1 ). These results show a high morphological effect of many types of molecular alterations. We then look at genes that were relevant as clinical information because they are associated with either a clinically approved drug or an in-development drug. For convenience, we later call the former not clinically relevant genes and the latter clinically relevant genes respectively non-clinical genes and clinical genes.
In clinical genes, we found that mutations in TP53 were detectable in TCGA-COAD, TCGA-LUAD, TCGA-BRCA and TCGA-OVARY with an AUC of 0. 60, 0. 64, 0. 74 and 0. 65 respectively (Table  1 ). TP53 was the only gene found detectable in the 4 datasets which suggest that TP53 is ubiquitously detectable and could have a pan-tumoral morphological signature; TP53 was significantly easier to find in the TCGA-BRCA dataset. In addition, KRAS mutations were found detectable in both TCGA-COAD and TCGA-LUAD with an AUC of 0. 63 and 0. 65 respectively. Also, mutations in the APC gene were detectable in TCGA-COAD with an AUC of 0. 61, EGFR mutations were detectable in TCGA-LUAD with an AUC of 0. 61. These results are similar to the pan-cancer study in  (10) , thus confirming their results.
We have chosen 5% and 10% cutoffs such that it is high enough to have an influence at the population level and low enough to be realistic for a short-track. For each screening strategy our deep learning pipeline performance is compared to a random screening test. Intuitively, the random screening test would provide 5% and 1% avoided tests for 95% and 99% sensitivity respectively. In the "Save-all" strategy (Figure  3 ), for the non-clinical genes, proportion of avoided tests reached 73% for SCN1A in TCGA-LUAD dataset (Figure  3 ). For the clinical genes, the proportion of avoided tests using screening range from 7. 8 to 26. 3 for a 95% sensitivity and range from 2. 6 to 14. 1 for a 99% sensitivity. This means that a quarter of the diagnostic tests can be avoided in the best case, which is with the EGFR gene in TCGA-LUAD.
In the "Prioritize" strategy (Figure  3 ), for the non-clinical genes, the increase in PPV reached +1166% for SCN1A in TCGA-LUAD with a 5% cutoff and +633% for the same gene with a 10% cutoff. For the clinical genes, the increase in PPV ranges from +24% to +181% with a 5% cutoff and from +12% to +117% with a 10% cutoff. Best performances were once more obtained by TP53 in TCGA-BRCA dataset, and EGFR in TCGA-LUAD have an increase in PPV of +125% (2. 25 times higher risk of being mutated when our test is positive) and +87% for a respective cutoff of 5% and 10%. Interestingly, having a high AUC score is not necessarily associated with having better performance in the different screening scenarii, the difference can mostly be explained by the prevalence of the mutation in the patient population.
Clinically relevant genes were defined as genes having an approved or in-development drug associated with their mutational status. TP53 mutation is an indication for the WEE1 inhibitor in a clinical trial of phase 3 in pancreas cancer, for ibrutinib  (18)  and idelalisib  (19)  in chronic lymphocytic leukemia. The absence of KRAS mutations is an indication for cetuximab  (20)  in colorectal cancer. Finally, EGFR mutation is an indication for erlotinib  (21)  or gefitinib  (22)  in lung cancer. As for each cancer type only one dataset was used, we cannot provide an estimation of the robustness of mutation detectability to change in the data distribution that would be caused by images coming from a new center. In other words, robustness to inter-center variation is completely unknown for the specific task of mutational status prediction.
This is a known barrier for clinical application of deep learning based methods. In this paper, we approximated the population prevalence by the dataset prevalence for the mutation. While prevalence does not affect sensitivity, it has influence over the PPV and the threshold of a given sensitivity. Therefore, results shown for "Fixed-Capacity" strategy would be similar for small prevalence adjustment but results for "Prioritize" and "Save-all" strategy are highly dependent on the true mutation prevalence. To describe benefits of the shown screening strategy more accurately, a medico-economic analysis could be done integrating the cost and availability of a diagnostic test in a first part, then the clinical benefits of the considered targeted therapy in a second part. Finally, with the ever-changing possibilities in medical cancerology, non-clinically relevant genes but with highly detectable mutational status might become relevant in the future.
Acknowledgements J. Bobok  was supported by the  European Regional Development Fund , project No. CZ 02. 1. 01/0. 0/0. 0/16019/0000778. J. Činč  was supported by the  FWF Schrödinger Fellowship  stand-alone project J  4276-N35. P. Oprocha was supported by  National Science Centre, Poland (NCN) , grant no. 2019/35/B/ST1/02239. We would like to thank the anonymous referee whose constructive comments helped to improve the article. ( J. Bobok )  Department of Mathematics of FCE, Czech Technical University in Prague, Thákurova 7 ,  166 29 Prague 6 , Czech Republic Email address: jozef. bobok@cvut. cz( J. Činč )  Faculty of Mathematics, University of Vienna, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria. -and -Centre of Excellence IT4Innovations -Institute for Research and Applications of Fuzzy Modeling, University of Ostrava ,  30.
State of the art A number of different approaches have been proposed, leveraging SAT solvers  [22] , automata  [5] , and Bayesian inference  [19] , and extended to more expressive logics such as Property Specification Language (PSL)  [25]  and Computational Tree Logic (CTL)  [10]. Applications include program specification  [20] , anomaly and fault detection  [4] , robotics  [6] , and many more: we refer to  [5] , Section 7, for a list of practical applications. An equivalent point of view on LTL learning is as a specification mining question. The ARSENAL  [15]  and FRET  [16]  projects construct LTL specifications from natural language, we refer to  [21]  for an overview. Existing methods do not scale beyond formulas of small size, making them hard to deploy for industrial cases. A second serious limitation is that they often exhaust computational resources without returning any result.
B Sample generation method To evaluate the performance of the tools FLIE, SYSLITE L , and SCARLET effectively, we rely on our novel sample generation algorithm to generate benchmarks from LTL formulas. The outline of the generation algorithm is presented in Algorithm 3. The crux of the algorithm is to convert the LTL formula ϕ into its equivalent DFA A ϕ and then extract random traces from the DFA to obtain a sample of desired length and size. To convert ϕ into its equivalent DFA A ϕ (Line 2), we rely on a python tool LTL f 2DFA 9. Essentially, this tool converts ϕ into its equivalent formula in First-order Logic and then obtains a minimal DFA from the formula using a tool named MONA  [18]. For extracting random traces from the DFA (Line 4 and 8), we use a procedure suggested by  [3]. The procedure involves generating words by choosing letters that have a higher probability of leading to an accepting state.
2 and 1:2. 8 but up to 1:5 in some cases. Fifty-two analyses were performed on grains devoid of core-rim relationships but showing prominent concentric oscillatory zoning (striped zoning in the most elongated grains). The concordant analyses clustered between 468 ± 6 (#68) and 489 ± 6 (#90) Ma (n = 24) with 2 grains showing younger dates of 428 ± 12 (#75) and 433 ± 9 (#94) Ma and 12 grains older dates from 545 ± 6 (#44) to 611 ± 9 (#86), up to 1212 ± 6 (#76) Ma. Out of the 16 discordant 206 Pb/ 238 U dates obtained, 8 overlapped with the concordant data (490-480 and at c. 540 Ma) while 8 defined a continuous trend from 455 ± 7 (#87) to 414 ± 6 (#36) Ma. The remaining analyses were performed on grains comprising a relict core surrounded by oscillatory-zoned overgrowths. Eleven rims yielded concordant overlapping 206 Pb/ 238 U dates between   To sum up, both gneiss samples were typified by the occurrence of oscillatory-zoned grains and rims with 206 Pb/ 238 U dates in the range 475-500 Ma, i.
, 2016)  and should not be ascribed to limited Pb loss. In line with the volcanic origin of 18CEZ01, it is likely that the analysed dataset comprises syneruptive zircon grains and rims plus antecrysts which formed in deep-seated magma chambers (e. g. ,  Matzel et al. , 2006)  and were scavenged during the eruption. For such cases (mixture of pre-eruptive and syn-eruptive grains, undistinguishable on textural and chemical grounds),  Vermeesch (2021)  argued that the "Maximum Likelihood Age" (MLA) model designed by  Galbraith (2005)  has a high potential to unravel the actual crystallization age of the igneous rock. Indeed, the MLA model presumes that the antecrysts define a continuous range of preeruptive dates, which is arguably more realistic than the "two populations" (i. e. , two age clusters) subdivision (see Sect.
Acknowledgements. SC warmly thanks  Vojtech Janou sek  for providing invaluable insights on granitoid petrogenesis and classification schemes. François Faure  and  Pierre Barbey  are thanked for stimulating discussions. The assistance of  Laurent Tissandier  with the SEM was greatly appreciated. We are grateful to  Michel Faure ,  Jérémie Melleton  and  Michel Ballèvre  for their reviews (and the template of Fig. 1 for the latter) and to  Romain Augier  and  Laurent Jolivet  for editorial handling.
X-ray crystallography The X-ray diffraction data were collected at 100 K using an Agilent Xcalibur Sapphire1 diffractometer, equipped with a graphite-monochromatised Mo Kα radiation (λ= 0. 71073 Å). A multiscan  [35]  absorption correction was applied. The structure was solved and refined by full-matrix least squares based on F 2 using SHELXS-97 and SHELXL-97  [36] , respectively. The hydrogen atoms were treated as riding atoms and refined with N-H, O-H and C-H distances fixed at 0. 89, 0. 83 and 0. 95 Å, respectively, with Uiso (H) values of 1. 2Ueq (N, O and C). The molecular graphics were prepared using Diamond 3 program  [37]. Crystal data and data collection procedures are summarized in Table  1.
Hirshfeld surface calculations The contact proportions on the Hirshfeld surface and their enrichment were computed with the MoProViewer software  [38]. The contact enrichment ratio Exy between chemical species X and Y is obtained by comparing the actual contacts Cxy in the crystal with those computed as if all types of contacts were equiprobable. The nature of contacts and their enrichment in the crystal packing are shown in Table  5. To obtain an integral Hirshfeld surface around each moiety (Cu(II) cation, chloride anion and organic ligand), a set of entities not in contact with each other were selected in the crystal packing.
0°) in the crystal packing (Figure  5 ). The self-contacts are absent or strongly under-represented, except for the C•••C type. The Hirshfeld surface to have an equal share of hydrophobic (Hc and C) and hydrophilic (more charged) atoms. The crystal is constituted by an alternance of hydrophobic and hydrophilic layers parallel to the (b, a+c) plane (Figure  6 ). This can be seen in the contact statistics as both hydrophobic and hydrophilic contacts are enriched at E1. 35. On the other hand, the cross contacts are disfavored at The response of a crystal to an applied stress or force depends on the voids in the crystal packing. When voids are small, the atoms are strongly packed with each other, and the crystal can bear stress of a significant amount. The void was computed by using CrystalExplorer17 software (Figure  8 ).
The primary aim of this study was to assess the agreement between Ki67 RNA and Ki67 staining by IHC (Ki67 IHC ). The other objectives were to analyze the association between the event free survival (EFS) and the expression level of Ki67 RNA in ODX signature; and association between RS and Ki67 RNA. 1 ). Complete data sets from 98 breast cancer patients who underwent RS testing were provided from 4 public treatment centers (public hospitals and university hospitals). The included patients were exclusively female and showed a wide age distribution (from 31-81 years) with a mean age of 57 years. The predominant tumor characteristics were no special type (NST) (91%), N0 or Nmic (71%), grade 1 (62%), and tumor size pT1c (1-2 cm) (58%). All patients had ER positive/ HER2 negative tumors. Table  1  shows the patient and disease characteristics of the full population.
However, our cohort remains comparable to this larger population for most other demographic parameters. Differences between Ki67 RNA and Ki67 IHC. Comparisons of Ki67 status showed a discrepancy between the two different types of evaluation. Indeed, half of Ki67 RNA high (> 6. 35) were found Ki67 IHC low (Table  2 ). The observed correlation between the two techniques, is therefore weak even if significant (R = 0. 288, p = 0. 0047) (Supplementary data 2). Technically, IHC shows greater variability  20, 21  and is less reproducible than the quan-Table  1. Patient and disease characteristics in the population of ER positive/HER2 negative patients. In this population, 98 women were included from June 2012 to April 2014. All patients were tested by Oncotype DX Breast. The Nodal status was stratified in 3 groups: N0 (node negative), Nmic (micrometastatic) or N1 (1-3 positive nodes).
The progesterone receptor status (PR) was considered positive according to standardized European guidelines using a cut-off of ≥ 10% stained tumor cell nuclei. All patients were ER positive and HER2 negative. Figure  1. Threshold determination, specificity and sensitivity of Ki67 RNA. Threshold of Ki67 RNA was determined using package pROC version 1. 16. 1  44. An optimal threshold of Ki67 RNA at 6. 35 units was found with a specificity of 48% and sensitivity of 84%. tification of mRNAs. Another study comparing microarray data with IHC confirm in a series of 520 samples  26. These results are in contradiction with those observed by Finsterbusch et al. 27. These authors found a strong correlation between Ki67 in IHC and mRNA level of Ki67 obtained by Mammatyper in 101 patients. In this retrospective study, their method of assessing Ki67 protein expression was different from ours.
Therefore, it is an overall estimate of Ki67 expression. By IHC, the protein expression is scored only on tumor cells. The two methods do not provide the same types of information on Ki67 status. These two types of evaluation of Ki67 are probably complementary because the Ki67 IHC gives informations about the tumor specific proliferation index and the Ki67 RNA would reflect the proliferation of the tumor and its microenvironment. and analyses from a prospective epidemiological database  33  ). Based on level Ia evidence, the Oncotype DX test has been incorporated in leading internationally-accepted clinical guidelines on the treatment of early breast cancer (St. Gallen 34 , ESMO  35  , and ASCO  36  ). This therapeutic adaptation will therefore smooth the differences between the categories of the ODX test and at the same time demonstrates the value of the information provided by its score.
Methods Ethics statement. All procedures performed in studies involving human participants were in accordance with the ethical standards of the national research committee and with the 1964 Helsinki Declaration and its later amendments. In France, this search is considered like a non-interventional study according to European legislation. All patients were individually informed that their data should be used for scientific research. Informed and written consents were obtained from all subjects, or if subject are deceased, from a parent and/or legal guardian. All experimental protocols were approved by the « comité de protection des personnes » (CCP) of Besançon, France. Patients and tumors characteristics. Our study was an observational multicenter retrospective study collecting data on the real-life use of Oncotype DX Breast Recurrence Score test by physicians in clinical practice settings in France.
Patients eligibility criteria for this analysis correspond to the population for which the Oncotype DX Breast Recurrence Score test is validated, i. e. adult patients with a recent first diagnosis of a single early invasive breast tumor with ER+/HER2-status, plus available documentation of lymph node involvement as either N0 (node negative), Nmic (micrometastatic) or N1 (1-3 positive nodes). The following data were documented: patient age and sex, conventional clinical and pathological disease characteristics including histologic type, tumor size and grade, nodal status, receptor status including ER, PR, and HER2, and the Ki67 proliferation marker by IHC. Between 2012 and 2014, the RS results were interpreted in three categories (low risk; intermediate risk; high risk with two cut-offs: 18 and 30).
16. 1  44. The quantitative values of IHC and RNA were turned in clinically used qualitative variables. The statistical comparison between clinically used qualitative IHC and RNA was realized by Pearson coefficient. The statistical comparison between quantitative IHC and RNA was realized by the Pearson correlation coefficient. Statistical analysis were performed using R version 3. 6. 2  45. The survival follow-up was analyzed with the Cox test on Kaplan-Meier estimates using the package survival version 3. 1-8  46.
Acknowledgements We thank all patients who agreed to participate in this observational study. Authors also thank  Genomic Health, Inc. (now Exact sciences) and  Mahi Y. for the Ki67 RNA data. The authors are grateful to all participating physicians for their cooperation and for private and public anatomopathology departments ( CH de Trevenans ,  Besançon Pathologie  and  Dr Chargeboeuf M. )  for their participation in IHC analyses.
In recent years, machine learning (ML) algorithms have been deployed in safety-critical and high-stake decision-making, where the fairness of algorithms is of paramount importance. Fairness in ML centers on detecting bias towards certain demographic populations induced by an ML classifier and proposes algorithmic solutions to mitigate the bias with respect to different fairness definitions. To this end, several fairness verifiers have been proposed that compute the bias in the prediction of an ML classifier-essentially beyond a finite dataset-given the probability distribution of input features. In the context of verifying linear classifiers, existing fairness verifiers are limited by accuracy due to imprecise modeling of correlations among features and scalability due to restrictive formulations of the classifiers as SSAT/SMT formulas or by sampling.
Introduction The significant improvement of machine learning (ML) over the decades has led to a host of applications of ML in high-stake decision-making such as college admission (Martinez Neda, Zeng, and Gago-Masague 2021), hiring of employees  (Ajunwa et al. 2016) , and recidivism prediction  (Tollenaar and Van der Heijden 2013; Dressel and Farid 2018). ML algorithms often have an accuracy-centric learning objective, which may cause them to be biased towards certain part of the dataset belonging to a certain economically or socially sensitive groups  (Landy, Barnes, and Murphy 1978; Zliobaite 2015; Berk 2019). The following example illustrates a plausible case of unfairness induced by ML algorithms. Example 1. 1. Following  (Ghosh, Basu, and Meel 2021, Example 1. ) , let us consider an ML problem where the classifier decides the eligibility of an individual for health insurance given their income and fitness (Figure  1 ).