dataesr/scientific-paragraphs-categorization huggingface dataset:
Statistical analysis Statistical procedures were carried out in RStudio (R version 3. 3. 2, R Core Team (2016). Shapiro-Wilk test was performed to test the data normality. A repeated-measures ANOVA was performed with body posture (sitting with support, sitting without support and standing) as factor. Post-hoc analyses were done with Tukey HSD tests when necessary. For all these statistical treatments, the signi cance level was set at p < 0. 05.
Since many machine learning algorithms require a distance metric to capture dis/similarities between data points, metric learning has received much attention during the past decade. Surprisingly, very few methods have focused on learning a metric in an imbalanced scenario where the number of positive examples is much smaller than the negatives, and even fewer derived theoretical guarantees in this setting. Here, we address this difficult task and design a new Mahalanobis metric learning algorithm (IML) which deals with class imbalance. We further prove a generalization bound involving the proportion of positive examples using the uniform stability framework. The empirical study performed on a wide range of datasets shows the efficiency of IML.
ITML uses a LogDet regularization and minimizes (resp. maximizes) the distance between examples of the same (resp. different) class. Another recent Mahalanobis distance learning algorithm is GMML (Geometric Mean Metric Learning)  [35]  where the metric is computed using a closed form solution of an unconstrained optimization problem involving similar and dissimilar pairs. In light of these learning procedures, it is worth noticing that the loss functions optimized in LMNN, ITML and GMML (and in most pairwise metric learning methods) tend to favor the majority class as there is no distinction between the constraints involving examples of the majority class and the constraints on the minority class. This strategy is thus not well suited when dealing with imbalanced datasets. An illustration of this phenomenon on the spectfheart dataset from the UCI repository is shown in Figure  1.
Feng et al. [13]  propose to regularize a standard metric learning problem by using the KL-divergence between the classes. Wang et al. [29]  proposed IMLS that learns a classic metric and then performs a sampling on the training data to account the imbalance. However, as we will see in our experimental study, better performances can be achieved by resorting to a metric dedicated specifically to deal with the imbalance of the application at hand. Note that deep metric learning methods have also received attention by the community to address the problem of imbalanced data  [22, 30]. However these methods often require large training datasets, like in visual tasks. However, this requirement is not always fulfilled by the application at hand. Moreover, it is worth noticing that none of the previous approaches come with theoretical guarantees, a gap we will fill in this paper.
In order to implicitly control the rates of false positives and false negatives, we propose a new algorithm, called IML for Imbalanced Metric Learning, which accounts carefully the nature of the pairwise constraints (by decompos-ing them with respect to the labels involved in the pairs) and weights their impact in the loss function so as to maximize the F1-measure. Beyond this algorithmic contribution, we further provide a theoretical analysis of IML using the uniform stability framework  [3]. We derive a generalization bound which has the advantage to involve the proportion of minority examples. This bound provides some insight into the way to tune the weighting parameters to counterbalance the negative impact of imbalanced datasets. The paper is organized as follows. Section 2 describes our algorithm IML which takes the form of a simple regularized convex problem.
Let ∆M * = M * i -M * and ρ = n + n the proportion of minority examples. Then for any t ∈ [0, 1] we have M * -I 2 F -M * +t∆M * -I 2 F + M * i -I 2 F -M * i -t∆M * -I 2 F ≤ a(2ρ -1) + 2(1 -ρ) λn 2qt M * i -M * F. Proof. See Appendix B. Note that the parameter b of Problem (3) does not appear in this Lemma. While in the experiments, in order to scale to large datasets, the pairs will be generated by using the kneighborhood of the examples, we derived the proof in Appendix B by using all the pairs, that allowed us to get rid of the parameter b. This enables us to provide a more general result that does not depend on additional parameters (here the k of the k-nearest-neighbor rule). Along with the q-Lipschitz property of Lemma 1, Lemma 2 allows us to prove that IML is stable.
7 in  [2]. This means that our β formulation is a generalization of the standard stability constants in Metric Learning. Regarding the term D, the decomposition into four terms allows us to get a tighter bound by a factor 4 with D = 4B 2 M F + (1 + m) while D = 16B 2 M F + 4(1 + m) in [2]. Another interesting interpretation of our bound is that when ρ tends to 0, i. e. , the dataset is more and more imbalanced, a classic metric learning method (with a = b = 0. 5) will converge slower. Indeed, in such a situation, we get a stability constant β which would tend to 3q 2 λn > 2q 2 λn while by parameterizing by a, we have 2q 2 (-a+2)   λn. In this case, a value of a close to 1 allows us to reduce the negative effect of the imbalance.
Experiments Datasets. In this section, we provide an empirical study of IML on 22 datasets. They come mainly from the UCI  3  and Keel 4  repositories. The 'splice' dataset comes from LIBSVM  5. All datasets are normalized such that each feature has a mean of 0 and a variance of 1. For the sake of simplicity, we have chosen binary datasets, described in Table  1  where the minority class is given by the columns "Label". Note that IML can easily be generalized to multiclass problems by learning one metric per class in a standard "one-versus-all" strategy, and then applying a majority vote. Optimization Details. Like most Mahalanobis metric learning algorithms, IML requires that the learned matrix M is PSD. There exist different methods to enforce the PSD constraint  [19]. A classic solution consists in performing a Projected Gradient Descent where one alternates a gradient descent step and a (costly) projection onto the cone of PSD matrices.
The advantage is that the problem remains convex w. r. t. M  [34] , ensuring that one will attain the optimal solution of the problem by correctly setting the projection step in the gradient descent. Another solution  [32]  is based on the fact that if M is PSD, it can be rewritten as M = L L. Therefore, instead of learning M, one can enforce M to be PSD in a cheaper way by directly learning the projection matrix L ∈ R r×d (where r is the rank of M). This can be done thanks to a gradient descent by computing the gradient of the problem w. r. t. L (instead of M). The implementation  6  we propose is based on this latter approach  [32]  where we make use of the L-BFGS-B algorithm  [38]  from the SciPy Python library to optimize our problem: it takes as input our initial point (the identity matrix), the optimization problem of Equation (  3 ), and its gradient, then it performs a gradient descent that returns the projection matrix L minimizing Problem (3).
To prevent us from tuning r and finding the best rdimensional projection space, we set r = d in the experiments. As discussed at the end of Section 2, the pairs of examples considered by IML in its four terms are chosen using the nearest neighbors rule. Indeed, we noted experimentally that the algorithms using this strategy (LMNN  [31] , IMLS  [29]  and IML) perform better than the ones using a random selection strategy (ITML  [8]  and GMML  [35] ). Experimental setup. All along our experiments, we use a 3NN classifier after projection of the training and test data using the metric learned. The metrics considered in the comparative study are the Euclidean distance and the ones learned by LMNN  [31] , ITML  [8] , GMML  [35] , IMLS  [29]  and IML. For each dataset, we generate randomly 20 stratified splits of 70% training examples and 30% test data (same class proportions in training and test) and report the mean results over the 20 splits.
The parameters are tuned by 5-fold cross-validation on the training set through a grid search using the following parameter ranges: for LMNN and IMLS, µ ∈ {0, 0. 05,. , 1} (k is fixed to 3); for ITML, γ ∈ {2 -10:10 }; for GMML t ∈ {0, 0. 05,. , 1}; and for IML we fix a = b = n - n and we tune m ∈ {1, 10, 100, 1000, 10000} and λ ∈ {0, 0. 01, 0. 1, 1, 10} (k is also fixed to 3). Note that to study the impact of the hyperparameters a and b and the impact of how the pairs of examples were selected, we performed an experiment where we artificially decreased the proportion of minority examples in the datasets. The results of this experiment are given in Appendix G. The results of the first experiment are reported in Table  2. On average, the F1-measure of 72. 3% obtained by IML is the best in comparison to 70.
8% for LMNN and IMLS, 70. 1% for ITML, 69. 3% for GMML and 67. 3% for the Euclidean distance. Overall, IML has also the best average rank of 1. 52. We note that IML generally gives better performances on the datasets considered no matter how much they are balanced or not. This means that our reweighting scheme of the pairs can not only improve the performances in an imbalanced setting but is also competitive in more classic scenarios. Second experiment. To address imbalanced data issues, classic machine learning algorithms typically resort to over/undersampling techniques  [1]  or create synthetic samples in the neighborhood of the minority class-e. g. , using SMOTE-like strategies  [6]. We now aim at studying the behavior of those methods when used as a pre-process of the metric learning procedures.
We consider the results of Table  2  as baselines. We compare them to the performances obtained after performing prior to metric learning an over-sampling using SMOTE and a Random Under Sampling (RUS) strategy of the negative data. We use the implementations of these methods from the Python library imbalanced-learn  [21]. The results obtained are reported in Table  3  for SMOTE and in Table  4  for RUS. Note that the results from Tables 2, 3 and 4 where computed using the same training/test splits and the same validation folds and are thus comparable. In each of the three settings considered, IML obtains the best results showing that it is more appropriate for improving the F1-measure. We also note that SMOTE allows one to increase significantly the performances of all methods, while there is no gain with RUS in comparison with an approach without sampling.
Conclusion and perspectives In this paper, we propose to revisit the classic formulation of metric learning algorithms that learn a Mahalanobis metric in the light of imbalanced data issues. Unlike the state of the art methods that do not make any distinction between the labels of similar pairs, we propose to decompose the usual loss with respect to the different possible labels involved in the pairs. This decomposition allows us to give specific weights to each type of pairs in order to improve the performance on the minority class. We derive a generalization bound specific to the imbalanced setting showing a convergence term depending on the class imbalance and illustrating the hardness of learning from imbalanced data. Our experimental evaluation shows that we are able to outperform state of the art metric learning algorithms in terms of F1-measure over balanced and imbalanced datasets.
Study Context The pediatric emergency department (PED) of Lille University Hospital has a capacity of approximately 30,000 patients per year. A total of 10 doctors and 8 nurses (plus residents and trainees) work in the department to take care of the patients. The department is currently equipped with ResUrgences (Berger-Levrault), a patient management software that tracks patients from their arrival in the PED through discharge. ResUrgences is independent of the hospital's electronic health records but is interconnected with the laboratory information system and the picture-archiving and communication system from which it receives notifications when results are available. The PED's clinicians enter the patient's data (eg, name, age, reason for admission, triage decision) and their own observations in ResUrgences.
Work System Analysis and the Tool's Functional Specifications and Design The work system analysis had two main objectives. First, it aimed to identify the needs of clinicians and the constraints that shape their work. This required an in-depth understanding of the organization of the PED and how clinicians manage the flow of patients and prioritize them in overcrowded conditions. Second, it aimed to learn to what extent it was possible to use the data entered in ResUrgences to feed the PPT and PPA. This required knowing whether the data entered in the software accurately represented the actual state of patient care (ie, verifying that the data were entered quickly enough to track each patient's progress through the various steps in care). The data were first collected by structured observations performed by a human factors specialist using an observation grid built with iCoda (Studiocode).
The data collected were time stamped so that the time interval between occurrence and data entry could be measured to know whether events were documented in ResUrgences in a timely manner. In addition, clinicians were interviewed whenever the workload eased. They were asked to state and explain their reasons for patient prioritization, define the information on which their decisions were based, and state how and where they had collected this information. Furthermore, the same clinicians were formally interviewed at the end of their work shift with a focus on the data used to determine which patient should be taken care of first and how patients are prioritized. These interviews were audiorecorded and transcribed. Data collected from the work system analysis were modeled through Unified Modeling Language diagrams  [40]  in order to highlight, for each step of the care process, interactions between clinicians, their usage of ResUrgences (eg, data consulted, data entered), and the data used to advance the process.
Early mock-ups based on these specifications were developed by a human factors specialist using Axure (Axure). These mock-ups represented static screenshots of the whole PPT GUI and used interface components and mock but realistic patient data to look as much like a real interface as possible and to present a realistic occupancy of the PED. The mock-ups were presented to a focus group comprising 3 physicians, 2 registered nurses, 2 human factors specialists, and 2 software engineers. The final set of functional specifications as revised by the focus group was used to improve the mock PPT.
Data collected during the 4 phases were analyzed as follows. In phase 1, to evaluate the intuitiveness of the icons, we calculated the proportion of participants that correctly interpreted each icon. In the event of misinterpretation, we sought to understand the reasons for poor intuitiveness of the design by qualitatively analyzing participants' verbal statements. In phase 2, effectiveness in communicating priority was analyzed. For each screenshot, we compared the participants' choice of the top-priority patient with the patient indicated as such by the PPA. We sought to understand problems by analyzing participants' verbal statements. If clinicians' prioritization and their justification for this decision were consistent with the organization proposed by the GUI, we presumed that the organization matched their work habits.
Throughout the patient flow process, physicians enter data into ResUrgences and complete the patient's records (patient status, prescriptions, lab test results, notes, etc). To streamline patient flow through the PED, registered nurses and physicians apply various rules to prioritize patients to be attended. The main data used by clinicians to apply those rules are depicted in Textbox 1. Figure  4  provides an example of the sorting rules applied by registered nurses (Multimedia Appendix 1 for physicians). Our observations of the timeliness of ResUrgences data entry showed that the data were representative of PED activity, even during busy periods. The median time between the receipt of data by the physicians and their data entry into ResUrgences was 136 seconds (IQR 67-345 seconds). During busy periods, even data that were first collected on paper were entered into ResUrgences no more than 2 minutes later (for details, see Schiro et al  [48] ).
Principal Results This case study aimed to present the human-centered design of a PPT and of the PPA it relies on to show how to apply human-centered design methods to the design of a prioritization system. Representative end users were involved early in the design process. We performed a work system analysis and, based on the specifications ensuing from it, we designed a PPT along with a PPA. Finally, we performed user testing on mock-ups simulating how the PPT and PPA work. Results of the work system analysis underpinned the entire design process, from specification of the users' needs to the development of the PPA. The work system analysis enabled us to understand how the PED was organized and how clinicians managed the patient flow. This analysis also showed a short data entry time in ResUrgences, which would indicate that these data represent the patient's management in real time and therefore can be used to automatically populate the PPA and PPT.
The doctor can quickly release several patients by signing the discharge letters one after the other. Therefore, a fourth block, "Discharge," was added to the GUI for patients who are waiting for their discharge letter to leave the PED. The addition of this block allowed us to eliminate the icons representing that a patient is waiting for discharge letter. A number of sorting rules in the PPA were also modified or created. For instance, rules were changed to enable a distinction between the test and imaging results that were available in ResUrgences and those that had been interpreted by the doctor, because these represent different steps of patient care. Overall, this human-centered design methodology was useful to design a PPT that complies with clinicians' workflow and that automatically retrieves data from the patient management software.
A second limit relates to the designed tool. Its design was based on data that could be retrieved from ResUrgences and used by clinicians to prioritize patients. However, some marginally used elements are not entered into ResUrgences. For example, emotional elements such as crying infants can sometimes prompt clinicians to see a patient more quickly when this is not to the detriment of other patients. Because they are not entered into ResUrgences, these emotional factors cannot be taken into account when suggesting the next patients to see. Despite this technical limitation, the tool is useful because it provides all the other information clinicians need to get an idea of the next patients to see and suggestions for prioritization on the same interface. Clinicians are then free to consider other contextual elements when making their prioritization decision.
Therefore a simplified creatinine index (SCI) based on readily available parameters [i. e. patients' demographics, single-pool K t /V and pre-HD SCr] was developed  [27, 28]. Several recent retrospective and prospective cohort studies demonstrated a high predictive value of SCI on HD patient outcomes  [29, 30]. To further elucidate the predictive value of SCI as a marker of sarcopenia on patient outcomes, we examined the association between SCI and all-cause mortality in the database of the international MONitoring Dialysis Outcome (MONDO) Initiative  [31]. Along with this we tested the reliability of SCI as a surrogate marker of MM in HD patients by comparing it to the instrumental one determined by multiple frequency bioimpedance spectroscopy (MF-BIS).
M A T E R I A L S A N D M E T H O D S For study enrolment, we considered all European adult HD patients documented in the MONDO Initiative. The MONDO Initiative is an international retrospective cohort study that comprises chronic HD patients from 41 countries. Prior to data transfer to the MONDO Initiative, the respective consortia remove all identifiable parameters as per Health Insurance Portability and Accountability Act, General Data Protection Regulation 2016/ 679 and locally applicable rules and regulations. Additionally, multiple levels of internal data validation control are applied before the database can be used for research. All the research activities conducted by the MONDO Initiative comply with national and international ethical, compliance and legal standards. This study was exempted by the Western Institutional Review Board (ES-16-005).
A cubic B-spline function with 95% CI was fitted to investigate SCI trajectories in both sexes. The first derivative of these splines was approximated using finite differences. Simple linear regression was used to assess the relation between equation-derived and measured LTI. Bland-Altman analysis  [33]  was performed to analyse the agreement between equation-derived and measured LTI at different levels of measured LTI. Analyses were performed with SAS version 9. 4 (SAS Institute, Cary, NC, USA) and R 3. 4. 4 (R Foundation, Vienna, Austria)  [34].
Goal of URCLIM The goal of the URCLIM project (www. urclim. eu) is to prove a concept: the realization of Integrated Urban Climate Services (IUCS) for urban planners and related stakeholders using local urban and climate data from multiple open-data sources. A general methodology to produce UCS and to evaluate the associated uncertainties is being developed within the URCLIM project. In order to achieve the project goal, the project has 4 scientific objectives: 1) to develop a methodology for the creation of high-resolution maps of urban parameters for climate studies, 2) to analyze the propagation of uncertainty from regional climate models to urban-scale climate models and local impact models, 3) to evaluate multi-criteria impacts and various types of adaptation strategies, 4) to define pertinent Urban Climate Services in cooperation with stakeholders, and using a visualization interface.
Early results In order to share knowledge among participants and outside, an URCLIM collaborative web platform has been set up (http://geom etadatalabs. eu/Infolab_URCLIM). A QGIS plugin allows to upload information. It can interrogate Copernicus catalogues. The quality of the Open Street Map (OSM) database has also been analyzed, as well as the means how to enrich it. The most relevant geospatial variables used by the Town Energy Balance Model  (Masson 2000; Lemonsu et al. , 2012; Schoetter et al. , 2017)  have been identified, and methodologies to compute the climate indicators and extract and transform OSM data  (Mooney and Minghini, 2017)  to a set of GIS layers, have been developed. The GIS layers are then processed to compute urban indicators that will feed the TEB model. The code source of the algorithms is distributed in GPL and available on a public repository: https://github.
Societal impacts and the collaborations with the stakeholders All URCLIM partners have involved local city and territories stakeholders and these include mostly environmental and climate plan services, but also others are engaged for risks issues (e. g. on floods or health). IGN, the French national mapping agency, also implies its sister organizations in Europe as well as pan European organization like Eurogeographics, to disseminate URCLIM work on the issues linked to urban mapping and data for IUCS. Stakeholders so far have participated to special sessions during the general assemblies of the project. Hence the URCLIM project provides major advances on Climate Services for stakeholders, especially urban planners and city administrations. The services are aimed to provide pertinent climate information on a multitude of impacts and risks.