dataesr/scientific-paragraphs-categorization huggingface dataset:
Patient selection Patients 18 years or older, with diagnoses of any type of non-Hodgkin lymphoma, Hodgkin lymphoma, CLL or related diseases in accordance with the WHO classification  [5] , are eligible to participate. The registry collects prospective data on incident cases -a case being defined as having received a diagnosis subsequent to or within 6 months prior to the participating site securing HREC approval to participate in LaRDR, in order to minimise selection bias and the burden of retrospective data collection and to maximise data completeness. An exception is made for CLL, which is frequently slowly progressive, and a significant proportion of patients may never require CLL-directed therapy. Therefore, retrospective data on CLL patients diagnosed up to 10 years previously can be included, provided complete data are available.
Establishing a minimum dataset Data items included in the minimum dataset are listed in Table  1. Datasets and case report forms for lymphoma and CLL were designed by the steering committee and project team, and refined iteratively. A CLL-specific case report form was developed due to the particular staging, disease trajectory and therapeutic paradigm applicable to this disease subtype. Data dictionaries are available for reference. The minimum data set includes information on demographics, comorbidities, diagnosis, planned therapy (if any) and supportive care, which are collected at baseline, with relevant updates plus disease response and survival entered at 6 and 12 months, and annually thereafter. Quality of life and biobanking data options were included to accommodate future projects.
Data management, quality control, and analysis LaRDR uses a REDCap database hosted and managed by Helix at Monash University. REDCap (Research Electronic Data Capture) is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources  [8, 9]. The database has a user-friendly interface and requires only basic training for site staff. To minimise data entry error, and aid analysis, most fields were designed to be dropdown, check boxes or radio buttons with minimum free text requirements, since data collection is typically performed by non-medically trained staff who rely on hospital electronic and paper patient medical records and may not be familiar with specific disease-or treatmentrelated details.
Clarification on specific items can be sought from lymphoma and CLL experts on the project team, and/or site investigators. LaRDR project staff conduct quality control activities, review data queries, and provide feedback and reports to site staff, investigators and the steering committee. A data validation committee reviews inconsistencies to refine definitions, data fields and user instructions, and conducts audits to review data completeness and accuracy. Sites may access their own data at any time and can manage local reports to facilitate local audits and data completion. Students, medical specialists in training and others undertake research using registry data. All research projects must be approved by the steering committee who provide oversight to all approved projects, ensuring no overlap between projects and to help ensure timely completion.
Working groups 1. Pathology review working group: There are over 80 recognised subtypes of lymphoma with distinct biol- and management has now diverged significantly from non-Hodgkin lymphoma. Not all patients require treatment, but for those who do, the optimal use of newer therapies, including combinations and sequencing of agents, is yet to be defined. Furthermore emerging evidence supporting a key prognostic role for genetic and measurable residual disease testing in this condition requires an evidence base to support its optimal clinical application. A dedicated working group comprising 11 CLL experts designed the CLL-specific dataset and data fields, which was integrated into the existing LaRDR database and tested before being made accessible to other registry users. 3. Data validation committee: The management of lymphoma is rapidly evolving, with new treatment protocols and diagnostic tests continually emerging.
Communications and reporting Hospital data reports are provided annually to individual sites, with site-specific, aggregate de-identified patient data presented and compared with overall national data. A breakdown of major diagnostic groups and their characteristics, treatment and survival data, and information on data completeness, is included. This allows benchmarking with other health services nationally and participating hospitals can identify site-specific issues for clinical audit and further investigation. Sites with low patient recruitment receive generic reports until sufficient data have accrued (see example: Additional file 1). Summary LaRDR annual reports are published on the LaRDR website (lardr. org). Annual open meetings, usually conducted in conjunction with the national haematology scientific congresses, or virtually in 2020-21, provide opportunities for clinicians, site staff, industry partners and students to learn more about the registry.
Pilot phase and activities The registry commenced with a pilot in 2016 with 6 large metropolitan hospitals in Australia with lymphoma expertise and resources and who had expressed interest in participating. These sites and their teams were crucial in planning, testing and providing feedback on all aspects of the registry, including governance and operations, and refining the minimum dataset, data entry processes and the database. Data completeness reports were generated and fields with low completion rates reported back to data managers and compared with detailed information from site staff on data that were onerous to find in medical records or where instructions were unclear. Results were discussed by the steering committee and a number of important changes made to the database and processes based on this feedback.
Current status Across six Australian states and two territories, and one New Zealand site, more than 5300 patients are currently enrolled from 28 sites, with 33 hospitals open to recruitment and a further 2 sites awaiting governance approval; others have also expressed interest in joining. Whilst still dominated by large tertiary metropolitan centres, site profiles are diversifying, with the addition of five regional hospitals and one private hospital since the pilot phase. Recruitment to date is shown in Fig. 2  and the frequency of cases according to major disease group in Table  2. National Australian coverage is currently estimated at 20% of lymphoma cases diagnosed annually, and continuing to expand. All sites in Australia and New Zealand are now welcome to join the registry. Some key diagnostic and demographic data are presented in Table  2.
Median follow-up time for prospectively enrolled patients on the registry is 18 months, with 35% of patients having more than two-years follow-up. As the data mature, follow-up times will increase which is important given the long disease course and excellent prognosis of some lymphoma subtypes and CLL. The first data linkage with the Victorian Cancer registry is currently underway, and we plan to begin annual linkages with the National Death Index to validate mortality data and improve estimates of overall survival, a key endpoint in many analyses. The registry has already generated interest among the international lymphoma clinical and patient communities, government, and industry partners. To date, 20 research projects have been completed or are underway using registry data and infrastructure, with three international collaborations now formalised, delivery of 26 national and international conference presentations, and provision of 20 data reports to industry, investigators and government, along with publication of a manuscript (see: lardr.
Even expert pathologists can disagree on diagnostic assignments in lymphoma, and this is even more challenging in a national registry setting since it is not practical to require review of primary diagnostic material for all cases from every site -and there would be implications for changing a diagnosis after subsequent review where the patient has already received therapy. Managing the diversity of diagnoses (and accounting for changing diagnoses where low-grade diseases transform to a more aggressive form), and periodic updates to the WHO classification, with implications for existing and new entries in the database, adds further complexity. Furthermore, the registry was established to collect data prospectively, intending to enhance data completeness and reduce bias. However, CLL and some types of lymphoma are typically very indolent in their disease course, with little change in status over years or even decades.
Patients with stable CLL managed with 'watch and wait' approach are not captured in clinical trials (as by definition they do not require treatment), but nevertheless have disease complications such as immune   Highlighting advantages for: • Patients: Knowing their data will help create a national picture of the condition, and that their hospital is participating in benchmarking for best practice failure, and autoimmune disease. These patients can also be markedly under-represented in registry data. The long-term follow-up is also important to capture complications such as second malignancy that tend to occur more commonly over time  [14]. Furthermore, with the dramatic improvements over the last decade with immunochemotherapy and then novel therapies, prolonged survival is much more commonly seen than previously and these long-term complications related to this improved survival will be important to document as the 'new natural history' of CLL.
Future directions LaRDR is now an established CQR. It is well placed to continue its expansion with increased national coverage in both Australia and New Zealand, and to support future research, including by publishing results of analyses and providing epidemiological data (such as numbers and geographic location of patients with data on diagnoses and disease stage) which will inform planning of clinical trials. The registry can also serve as a platform for conducting clinical trials  [15]  and observational studies, and enable efficient, long-term follow up after these studies have been completed. In addition to information contained within the registry itself, LaRDR data can be used for epidemiological modelling and linkage activities to inform policy development and health service planning, especially for new and high-cost therapies and to ensure improved access to and delivery of care for all patients.
Include stakeholders in planning to consider feasibility: registry scope and the content of the dataset, noting that data collectors will likely not be experts in this field • Provide training for site staff and access to ongoing support and resources, and data definitions • Data validation committee and periodic audits of data completeness and utilisation • Per-patient payments for data• MaximisingUse of institutional secure e-research platform (SeRP) permits authorised users to access and analyse data within a data 'safe haven' , under control and supervision of registry data custodian.
data access while maintaining data security Include stakeholders in planning to consider feasibility: registry scope and the content of the dataset, noting that data collectors will likely not be experts in this field • Provide training for site staff and access to ongoing support and resources, and data definitions • Data validation committee and periodic audits of data completeness and utilisation • Per-patient payments for data• MaximisingUse of institutional secure e-research platform (SeRP) permits authorised users to access and analyse data within a data 'safe haven' , under control and supervision of registry data custodian • Patient representation on the Steering Committee • Newsletters, website information and presentations for patient/community groups • Sites and clinicians: Access to own and other data for comparison and • Direct access to own data benchmarking, access to a peer network, opportunities for projects for young • Hospital data reports investigators, access to data on rare conditions where trials are difficult • Participation in registry committees • Research project opportunities, supported by LaRDR team • Industry, policy-makers and others: Access to real world data including on • Presentations at scientific meetings and to government treatment sequencing, uptake of new, high-cost or high-risk therapies, reasons • Annual open investigator meetings for discontinuation, data to support regulatory applications and health service • Peer-reviewed publications planning • Commissioned reports Ethics and governance • 'Opt out' consent model: works well for engaging patients and obtaining • Engagement with site HREC and governance committees clinical data but unfamiliar to some (including some HREC & governance com-• Seek written consent in future to obtain biological samples mittees) • Transition to national mutual acceptance ethics arrangements • Historically fragmented HREC system: time-consuming to apply individually to local HREC and governance committees Sustainability • Pilot phase and managed roll-out: Essential to recognise and address opera-• Respond to feedback, and keep stakeholders informed of progress and plans tional problems early, but few results generated during this stage • Per-patient payments: even modest support is valuable and permits managers • Site resources: generally very limited, alleviated somewhat by per-patient to allocate staff time for data entry payments provided initially • Communicate potential of registry data and how industry, clinicians, researchers, • Secure ongoing funding: always challenging until data mature and can government agencies can access generate analyses and peer-reviewed publications Data and access • Data entry burden: Find a balance between collecting all possible data, • Research activities determining what is feasible to collect, and what will actually be used -deter-mining an initial minimum dataset for the registry, which is subject to ongoing review and potential for additions (e.
We also introduced a set of axioms and definitions to clarify XAI from a mathematical perspective. Finally, we present the Greybox XAI, a framework that composes a DNN and a transparent model thanks to the use of a symbolic Knowledge Base (KB). We extract a KB from the dataset and use it to train a transparent model (i. e. , a logistic regression). An encoder-decoder architecture is trained on RGB images to produce an output similar to the KB used by the transparent model. Once the two models are trained independently, they are used compositionally to form an explainable predictive model. We show how this new architecture is accurate and explainable in several datasets.
This refers to the ability to produce counterfactuals. In addition, "good" explanations are selective, meaning that focusing solely on the main causes of a decision-making process is sufficient. Furthermore, if a causal explanation for the generalization itself is not provided, utilizing statistical generalizations to explain why occurrences occur is insufficient. Finally, explanations are social, meaning that they are a transfer of knowledge between an explainer and an explainee. Deep Learning models suffer from two kind of bias. The first one is a learning bias, due to the presence of a bias in the training data set. This can happen when associations of concepts are over-or underrepresented in the training set. For example there were cases of a dataset with women under-represented in offices compared to men, leading a captioning algorithm to assume that a person in an office was necessarily a man while it could also be a woman.
The contribution of our paper is threefold: • A theory of explainability of deep learning models to qualify what is a "good" explanation. • An explainable by design compositional framework called Greybox XAI. • We show that this new framework provides state-of-the-art results on an image classification task regarding the explainability/accuracy trade-off in various datasets, as its accuracy is close to the existing models while being more explainable. This paper is organized as follows: first we present the literature around XAI and part-based classifiers in Section 2. We present the different notions and terminology used in XAI and propose our definitions in Section 3. We describe our framework in Section 4 and we illustrate its use by experiments on several datasets in Section 5.
Example 2. Intrinsicality In order to produce an explanation for a black box model f , the post-hoc method LIME  [11]  generates a new dataset consisting of perturbed samples and the corresponding predictions of f. On this new dataset, LIME trains an interpretable model h, which is weighted by the proximity of the perturbed samples to the instance of interest. The prediction of the model h should be a good approximation of the predictions of the model f locally, but it does not have to be a good global approximation of the model f. The produced explanation can be expressed as follows: e = Φ(f (x)) = arg min h L(f, h, π x ) + Ω(h) (3) with L(f, h, π x ) the local fidelity, i. e. how close the predictions from h are close to the predictions from f. The proximity measure π x defines how large is the neighborhood around the explained instance.
Greybox Architecture and Data Requirements Let us denote X and Y two random variables, with X ∼ P X and Y ∼ P Y. Without loss of generality we consider the observed samples {x i } N i=1 ∈ X N as vectors and the corresponding labels {y i } N i=1 ∈ Y N as scalars. From the set of observations X N and the set of corresponding labels Y N we derive a training set denoted D = {(x i , y i )} N i=1 with N the number of pair elements (x i , y i ) of dataset D. The elements of dataset D are assumed to be independent and identically distributed (i. i. d. ) according to an unknown joint distribution P X,Y. Let us denote f a DNN. We assume that a DNN is a function that takes two inputs. The first input is the input data x i and the second input is the set of trainable weights θ = {θ k } K k=1 with K the number of weights of the DNN.
Fig. 4  shows the Greybox XAI framework used for an image classification task. Here, the dataset D triplet = {(x i , z i , y i )} N i=1 consists of triples from a set X of RGB images, a set Z of semantic segmentation masks and a set Y of labels. From Z we extract a second subset Z att , which we will call the set of attributes. This set contains a list of all attributes, i. e. all different segmentation masks. These are all the part-of objects that can be detected by the Latent Space Predictor. Here the Latent Space Predictor that constitutes the model g is an Encoder-Decoder model. Its role is to predict from an image x i a segmentation map z i. From this segmentation map is extracted the vector z att,i which constitutes a list of all attributes present on the segmentation map z i. We call this operation a vectorization.
Subset Extraction for Knowledge Base Construction The goal of the dataset subset extraction task is to obtain a Knowledge Base from a Dataset D triplet = {(x i , z i , y i )} N i=1 with a set X of RGB images, a set Z of semantic segmentation masks and a set of (final, whole object) labels Y. We call a Knowledge Base a common repository of semantic annotations to facilitate a fast and efficient search in the given set of resources  [78]. In Description Logics  [79] , a terminology box TBox, or schema extracted from the dataset, together with the class instanciations (in our case, that compose the datasets seen by the network), i. e. , the assertional ABox, form the KB =< T Box, ABox >. This particular KB will serve as database, in form of a knowledge graph (KG) that interlinks individuals (i.
e. instances) of different classes through roles or relationships with possible semantic restrictions. In order to create and populate a Knowledge Base it is necessary that some terminology axioms T Box and assertion axioms ABox can be extracted from some of the training labels, from which a set of Resource Description Framework (RDF) triples of subject, predicate and object (s, p, o) can be extracted. This is the case, for example, when the labels are in a text or caption form  1. The Automatic Knowledge Base Construction (AKBC) from a dataset problem consists of finding a process t : Y KB → {(s, p, o)} able to triplify each data point following the RDF language  2. Concrete examples of AKBC include learning a KB through semantic parsing, using Markov Logic  [82]  or relational probabilistic models  [83].
The triplification process consists of extracting two entities or concepts, subject (s) and object (o), and a predicate (p) that connects them. This predicate expresses a relationship, i. e. , a data property or an object property  [84]  among the subject and the object. The AKBC problem in the Greybox XAI framework consists of finding a process that automatically constructs a KB composed of hierarchical (isA), partOf, and attribute (hasAttribute) relations. Analogically, the ABox in KB must be uniquely composed by description logics assertional axioms solely extracted from applying the triplification process on datapoint. We flatten each semantic segmentation image z i ∈ Z to obtain a single dimension vector and we return z att,i , a sorted list of every unique element contained in z i in order to obtain a set Z att of each element present on each semantic segmentation image.
By using OWL  3  entities and relationships we obtain the following type of KB, linking any element {z att,i,j } N,K i,j ∈ Z N,K att with the corresponding x i ∈ X N and y i ∈ Y N , with N the number of triple elements of dataset D triplet and K the number of different attributes in Z: This Knowledge Base stands for the Explainable Latent Space which will be used to produce the rationale of an explanation. Our framework is composed of 2 sequential and connected models that do not perform with the same representation of information: while it is possible for the DNN to work with images, it is necessary for the transparent model to work with a much more reduced and compact representation of the data. It is not possible for a model such as a logistic regression to take images as input because it would lose its transparency due to the exploding number of variables and parameters.
Training of the Transparent Classifier We use a logistic regression to statistically fit the attributes Z att and classes Y present in the database. The goal is to see if we have fairly discriminating attributes. The purpose of this test is to check if it is possible to train a transparent classifier based on attributes. If there were an excessive difference between the performance of a regression predicting classes based on attributes and the performance of a DNN predicting classes based on images, it would mean that the presence/absence of attributes did not allow for an efficient classification of images and therefore that these labels were of poor quality. The choice of using a logistic regression is motivated by the fact that this model predicts probabilities easily interpretable  [86].