dataesr/scientific-paragraphs-categorization huggingface dataset:
A. , per- financial support from Novartis, personal fees from ThermoFisher, Menarini, personal fees from UCB, personal fees and other from fees from Novartis, personal fees from Teva, personal fees from sonal fees from Mobile Chamber Experts (a GA2LEN Partner), per- personal fees from Aimmune, grants and personal fees from ALK, AstraZeneca, personal fees and other from GSK, personal fees from Sanofi Aventis, personal fees from Zentiva, personal fees from sonal fees from Indoor Biotechnologies, grants and personal fees non-financial support from MADX, personal fees from EFSA, out-MSD, outside the submitted work. Dr. Naclerio reports other from LEK-AM, personal fees from Celon Pharma, personal fees from from GlaxoSmithKline, personal fees from Astellas Pharma Global, side the submitted work; and is a Co-I or scientific lead in three in-Sanofi, other from Lyra, other from Regeneron, outside the submit- Adamed, personal fees from Polfarma, personal fees from Chiesi, personal fees from EUFOREA, personal fees from ROXALL Medizin, vestigator initiated oral immunotherapy trials supported by the ted work.
5 kg/m 2 ) and the adult male obese model Fats (body mass index of 36. 2 kg/m 2 ). Billie and Fats virtual population models were selected to cover a wide body mass index span across the database population. The RF coils used belong to the MRIxVIP1. 5T library and consist of high-pass resonators with 16 rungs and a constant shielding length of 150 cm. The nominal resonant frequency of the coils is 64 MHz. Four cylindrical RF coils with the following physical dimensions were included (bore size = 70 cm, coils inner diameter = 75 cm and coil length = 40, 50, 60, and 70 cm). The analysis was conducted at 17 landmarks ranges from Z0 (patient eyeline centered at MR scanner iso-center) to the patient feets (Z1600 for Fats and Z1300 for Billie). The scanning positions Z200 and Z300 are expected to represent the worst-case in terms of RF coupling since the CIED system is close to the iso-center.
The tangential electric field, E tan associated to the selected polarizations is determined by conducting a full-wave simulation of the RF exposure system using a commercial computational electromagnetic software package, Sim4Life (SPEAG, Zurich, Switzerland). The RF exposure system simulations were based on the CAD file provided by the RF exposure system manufacturer. The contributions of both RF resonator feeds are simulated independently and are combined to get the resulting E-field for the specific polarization. The tangential E-field for the two sets of polarization used for the model calibration are plotted in Figure  3E  for Lead-1 and Figure  3F  for Lead-2, respectively. For each polarization, the predicted and the measured RF deposited power around each electrode are determined; they are plotted for Lead-1 and Lead-2 in Figures  4  and 5 , respectively.
Related Work In this section, we review the existing evaluation strategies for EBS systems. The performance assessment of these systems are traditionally performed according to the CDC's guideline, which aims at understanding the internal and external performances of EBS systems  [6, 63]. Nevertheless, most of these evaluation metrics are more in line with an end-user perspective, which require human resource for manual assessment. On the other hand, there exist many studies which conduct a retrospective analysis using surveillance dataset, i. e. the output of an IBS/EBS system, without performing any comparative study. These works deal with additional evaluation points that are not considered in the traditional evaluation methods  [5, 6]. In particular, we are interested in those works performing a descriptive analysis, rather than predictive analysis, which is in line with our work.
2). Then, we evaluate an EBS system based on the spatial (Section 3. 3), temporal (Section 3. 4), thematic (Section 3. 5) and source (Section 3. 6) dimensions. Except the last dimension, these evaluations are always performed with respect to a gold standard database, which is supposed to contain all events confirmed and notified by national and supranational authorities. Moreover, we take into consideration the fact that there exist some gaps between EBS systems in disease detection and collection, an issue so-called reporting bias  [28, 29]. To do so, our evaluation relies on either ranking results (see Section C. 1 in the Appendix) or the concept of representativeness (e. g. Section 3. 3).
Based on these previous event definitions, we define an event throughout this work as the detection of the virus for a specific host at a specific date and in a specific location. Moreover, we also consider the fact that an event is reported by a news outlet. We illustrate in Table  1  how we define an event from the following text: "A highly pathogenic strain of bird flu (H7N9) has been detected in two captive birds of prey at a private property in Skelmersdale on March 31. ". Next, we introduce the definitions and notations related to events. An event database E is a finite set of events collected by an EBS system. Let D = {D 1 ,. D n } be a set of dimensions to define the events in E. Every event is expressed as a tuple e = (d 1 ,. , d n ), where we call d i event attribute for every i = 1,.
, n. Concretely, in this work, the set D contains five dimensions, and it is defined as {D Z , D T , D D , D H , D S }. The dimension D Z is the location, where disease events have been occurred, and they are expressed as polygons (e. g. country or city polygons). The dimension D T is the notification date of events and it is a totally ordered domain. Moreover, the dimension D D is the disease which infects a number of hosts. The dimension D H is the host who have caught the viruses of a specific disease. Finally, the dimension D S is the news outlet publishing a given epidemiological event. Each dimension D i is associated with a domain of (discrete) values, denoted by dom(D i ). Given an event database E over D, for every i = 1,. , n, we denote by Dom E (D i ) (or simply Dom(D i ) if E is clear from the context), the active domain of D i in E, which corresponds to the set of all values  Italy, India, Pakistan Table  3 : Illustrative example of E l Z l T , where l Z = country and l T = week.
Moreover, we sometimes need to fix the spatial and temporal scales (i. e. hierarchical levels) of an event database E. This operation amounts to discretize the dimensions D Z and D T over a set Z of geographic zones and a set T of time intervals, depending on the spatial and temporal scales. We denote this fixed scaled event database by E H Z ∼l Z H T ∼l T (E l Z l T for short), where l Z (resp. l Y ) represents a spatial (resp. temporal) scale in H Z (resp. H T ). When this fixed scaled database is ordered by time, then it is defined as E l Z l T = {(t 1 , X 1 ), (t 2 , X 2 ),. , (t |T | , X |T | )}, where |T | represents the size of database, X j ⊆ Z is a set of spatial entities and t j represents a time interval for which X j ∈ Z occurs in E l Z l T. Note that if an event does not have precise information with respect to l Z and l T (e.
g. an event occurring in France, while l Z = city), we do not include it in E l Z l T. We illustrate a fixed scale event database E l Z l T with an example in Table  3. This example relies on a toy fictional event database, in which we fix the spatial and temporal scales to country and week, respectively. Each row in Table  3  includes the countries reporting at least one epidemiological event for a given weekly time interval. For instance, we observe the first disease cases in France, Italy, China and India during the first week. Then, in the second week the viruses spread over neighbor countries, which are Spain, India and Nepal. Finally, it is also possible to restrain all literal values of a dimension D i , i. e. Dom E (D i ), with a fixed spatial or temporal scale. For a given spatial (resp.
Event matching In this section, our goal is to identify common events between two event databases in an automatic manner, which is not a trivial task. We propose here an approximation scheme by modeling this task as an assignment problem, also known as maximum weighted bipartite matching problem, as already done in the literature (e. g. [57] ). In the end, we obtain a set of "putatively " associated events between two event databases. Let E 1 (resp. E 2 ) be two event databases associated with IBS or EBS systems, containing N E1 and N E2 events, respectively. Also, we assume N E1 ≤ N E2 without loss of generality. Moreover, let S be the N E1 × N E2 similarity matrix of E 1 and E 2. The term S ij , with 1 ≤ i ≤ N E1 and 1 ≤ j ≤ N E2 , represents the similarity score between events e i and e j and it is calculated with Equation 8 (Section A in the Appendix).
Spatial Dimension Our evaluation strategy for the spatial dimension relies on the concept representativeness. Barboza et al. [6]  define this concept as the ability of describing accurately the distribution of events in terms of place, time and host. Particularly, geographic representativeness constitutes an important aspect in Epidemic Intelligence. For this reason, we propose to compare the spatial dimension of the events collected by an EBS system through geographic representativeness by taking the temporal aspect into account. We call it spatio-temporal representativeness, and it allows measuring how well the event database E of an EBS system represents geographic zones (e. g. country, regions) in terms of the events found in a gold standard database E R , for a given time period. In the end, the obtained results enable us to know to what degree geographic zones are represented by E.
Timeliness We start with the first comparison, which is based on the concept timeliness  [2, 4, 6, 52, 63]. Barboza et al. [6]  define this concept as the ability of identifying disease events in a timeframe enabling utilization of the information by decision makers to mitigate potentially dangerous situations as soon as possible. In the literature, timeliness is measured as the time difference between the publication date of an event in an EBS system and that of the same event in a gold-standard database. Nevertheless, we model it with an exponential decay function in order to obtain a normalized score, as proposed in  [13, 40, 41]. Its calculation for an event database E of an EBS system is performed with respect to a gold standard database E R. This requires to know the binding of the events between E and E R , which is unknown in advance.
The spatial closeness between the considered countries is defined in Table  4. than the user-specified minimum period-support. For instance, if we take the same example illustrated in Table  3 , all partial periodic spatial patterns for ι = 2 and = 2 are illustrated in Table  5. It is worth noticing that the parameter ι has a positive effect on the generation of partial periodic spatial patterns, while has a negative effect on the number of patterns being generated from the database. Moreover, note that the input parameters ι and can be both expressed in percentage or in count, respectively. For instance, when we set = 1. 0 (resp. < 1. 0), this amounts to generate full (resp. partial) periodic spatial patterns. In this work, given spatial and temporal scales l Z and l T , we discover all full and partial periodic spatial patterns in E l Z l T with respect to the input parameters ι, and α through the method ST-ECLAT (ST for short), proposed in  [31].
Particularly, we are interested in two use cases for obtaining these patterns. First, we want to know what spatial entities (e. g. countries) have consistently epidemiological events throughout the year. We call the obtained results continuous periodic patterns. Second, we want to know what spatial entities (e. g. countries) have a seasonal effect and are exposed to disease events only for some period of time every year. We call the obtained results seasonal (or yearly ) periodic patterns. For space matters, we explain in the Appendix how we quantitatively evaluate the performance of E in terms of its ability to detect these continuous and seasonal periodic patterns with respect to a gold standard database E R (Section C. 3. 2).
Thematic Dimension In this section, we aim to evaluate EBS systems in terms of thematic entities they extract from the events. In other words, we want to know whether the dimensions D D and D H in an event database E provides fine-or coarse-grained information. Ideally, we expect E to provide very detailed information, as in a gold standard database E R. Note that this aspect is related to one of the relevant characteristics of an EBS system in the CDC's guideline, so-called completeness  [6]. In our evaluation, we want to discover the rich data relations between spatial, temporal and thematic entities with two use cases. In the first use case, we totally omit the temporal aspect, and we propose to find out what thematic entities characterize most a spatial entity. For instance, if we take the same example illustrated in Table  1 , we might want to know where the specific Avian Influenza serotype H7N9 is more prevalent.
Time Multidimensional pattern t 1 (Paris, AI, bird), (Île de France, AI, bird), (France, AI, bird) t 2 (Italy, AI, wild bird), (Italy, AI, bird) t 3 (Spain, H5N1, wild bird), (Spain, HPAI, wild bird), (Spain, AI, wild bird), (Spain, H5N1, bird), (Spain, HPAI, bird), (Spain, AI, bird) Table  6 : Illustrative example of the modified version E + of a subset of an event database E with three events. The first column indicates the timestamp of the events, and the second column describes the multidimensional patterns. Note that the second column contains all ancestors in the associated hierarchy of every multidimensional pattern. For the sake of clarity, we show in bold the elements in E, before obtaining its modified version E +. In this work, we perform the two use cases discussed above by discovering frequent multidimensional patterns, accompanied by the partial periodicity condition.
When the temporal aspect is omitted, we simply calculate the support sup(Y ) of each multidimensional pattern Y in E + , as such pattern always corresponds to a single tuple (d Z , d D , d H ) in this context. We call them static multidimensional patterns. When we take the partial periodicity condition into account, this amounts to find partial periodic multidimensional patterns, as it ensures that two multidimensional patterns appear in the same time interval in E +. We call them temporal multidimensional patterns. In practice, we use the ST algorithm described in Section 3. 4. 2 to generate these static and temporal multidimensional patterns. The flexibility of ST is that when we set a very large inter-arrival time value to ι, this allows us to omit the partial periodicity condition. For space matters, we explain in the Appendix how we quantitatively evaluate the performance of an event database E of an EBS system in terms of its ability to detect these static and temporal multidimensional patterns with respect to a gold standard database E R (Section C.
Identification of Important News Outlets We say that a news outlet needs to fulfill two conditions in order to be considered as important. First, it reports epidemiological information that are reported by both local and international news media. In other words, if someone follows the news reported by an important news outlet, it means she receives sufficiently necessary epidemiological information for her country and nearby. Second, it also reports the events that are reported by important news outlets. We perform the task of identification of important news outlets through network analysis. We design our approach in two steps. First, we extract the news outlet network G E from an event database E, where nodes represent news outlets and edges describe the relations for node pairs. We do this process on the whole or a subset of data for each considered EBS platform.
Timely Detection One specific criteria that one may want to optimize in event detection is to minimize detection time (i. e. capturing an epidemiological event as soon as possible). Our second objective is related to this timeliness capability of the news outlets. We want to identify the news outlets, which are timely in event detection, and not those detecting as many events. To do so, we follow the work of Leskovec et al. [36]. Their method first creates the news outlet network G E from E, then finds through their method CELF a set A of news outlets, which minimizes detection time, while covering all the event set E. First, we extract our news outlet network G E as follows. Let us suppose that the set S e of news outlets reports through their news documents the same event e in an event database E.
The marginal gain is expressed for a subset A of news outlets in terms of the function R(A), which is used as a penalty reduction function. It is defined as in Equation  5. R(A) = e∈E P (e) T max -min s∈A T (e, s) , ( 5 ) where T max is time horizon, P is a (given) probability distribution over the events and T (e, s) represents the time delay in days, until news outlet s participates in the event path P e. Note that T (e, s) equals T max , if s does not report event e. Moreover, in our context each event has uniform probability, therefore we omit P (e) from the definition of R(A). In the rest of the work, we denote the first k most timely news outlets obtained from the CELF method for G E by CELF(G E , k). For space matters, we explain in the Appendix how we quantitatively evaluate the performance of an event database E in terms of its ability to detect the important and timely news outlets (Section C.
Selected EBS systems We show the relevancy of our framework on two well-known EBS systems PADI-web  [67]  and ProMED  [11]. Moreover, we use the reference gold standard database Empres-i from the World Organisation for Animal Health (WOAH) and the Food and Agriculture Organization of the United Nations (FAO)  [18]  to evaluate the performances of PADI-web and ProMED. Our choice of the Empres-i database is based on the fact that it is a well-populated official database for the main animal diseases, such as Avian Influenza and African Swine Fever  [3, 20]. ProMED collects and organizes its disease events through 50 subject matter expert moderators from 34 countries 6 , who provide written commentary, giving the reader the necessary historical context and/or clinical background to understand the importance of the information being reported.
Event Data and Processing The event datasets we use concern the Avian Influenza (AI) cases affecting bird species from 2019 to 2021. These AI cases can be high pathogenic AI (HPAI) or low pathogenic AI (LPAI). They are retrieved from PADI-web, ProMED and Empres-i, respectively. Regarding the PADI-web data, we rely only on those considered as relevant from PADI-web's automatic process. We chose a three-year study period (2019-2021) to sufficiently capture the space-time epidemiological characteristics of the AI events around the world. In order not to penalize ProMED we keep all its data provided by official and unofficial data sources, although PADI-web relies only on online news outlets. Nevertheless, for the sake of completeness, we compare both systems in Table  8  and Figure  8  by discarding the ProMED data provided by official data sources (i.
e. WOAH reports). Furthermore, it is also worth noticing that automated EBS systems, such as PADI-web, might report false event information due to their automated location detection and extraction strategies. Evaluating the rate of reporting false events for such systems is not the scope of this work. We process the collected raw event datasets by transforming them into normalized event databases, as explained in the Appendix (Section B). During this processing, we deal with the different event definitions that PADI-web and ProMED have, which do not exactly match the one proposed in Section 3. 1, as the definition of an event can be different from one EBS system to another. Regarding ProMED, we only extract the information regarding news outlets from the raw news documents. Regarding PADI-web, the events are essentially disease-location pairs.
Moreover, PADI-web extracts event-related thematic information for each collected news document, without relating them to any event. For this reason, we complete the minimally defined events with the extracted thematic entities, as detailed in Section B. 1. In the end, we obtain three normalized event databases for PADI-web, ProMED and Empres-i, denoted by E P W , E P M and E EI , respectively. These normalized data are not publicly available due to third party restrictions, nevertheless, they are available on request 7.
Results We now assess, compare and discuss the performances of the considered EBS systems when applied to our framework on the Avian Influenza event databases E P W , E P M and E EI. In our experiments, the number of events by year for those event databases are shown in Table  7. In total, there are 1515, 786 and 5229 events for E P W , E P M and E EI , respectively. We present the results in line with our four evaluation objectives: 1) spatial (Section 5. 1), 2) temporal (Section 5. 2), 3) thematic (Section 5. 3) and 4) source (Section 5. 4) dimensions. Our source code is publicly available 8.
Spatial Dimension We evaluate through spatio-temporal representativeness how accurate EBS systems describe the distribution of events found in the gold standard Empres-i database in terms of place and time. We calculate  γ Here, we focus only on the countries, where the spatio-temporal representativeness score is the maximum value of 1 for both PADI-web and ProMED. If we focus on the countries covered both PADI-web and ProMED instead, there are 58 countries in common. the spatio-temporal representativeness scores for each region and country with monthly time intervals. In the following, we discuss only these scores at country scale and leave those at region scale in the Appendix, due to lack of space. We plot the scores at country scale in Subfigures 4a and 4b (see Figures  9  and 10  in the Appendix for region scale).
Timeliness analysis We study how timely the EBS systems PADI-web and ProMED are compared to the Empres-i events, as well as the assessment of timeliness between them. Note that we perform this assessment based on the identification of the putatively associated events between a pair of event databases, as explained in Section 3. 2. The obtained statistics regarding these putatively associated events are shown in Table  10. In Figure  5 , we plot the time lag values for each pair of systems. In these plots, when the first (resp. second) system reports an event earlier than the second (resp. first) one, then this results in a negative (resp. positive) value. Moreover, we summarize the statistics in terms of timeliness in Table  11. We can see from Figure  5  and Table  11  that although both PADI-web and ProMED can be timely depending on events, PADI-web is more timely with respect to the Empres-i events compared to ProMED (49% vs.
Periodicity Analysis We now study how accurate EBS systems detect full or partial periodic continuous and seasonal patterns with different temporal scales based on the evolution of the epidemiological events. We compare the obtained results based on the Empres-i dataset to see to what extent PADI-web and ProMED can capture similar patterns. To ease our discussion, we visualize the evolution of the events occurring in some countries of interest with fine-grained temporal scale from 2019 to 2021 in Figure  6. We describe it generically here, for matters of convenience. This figure is in a form of heatmap matrix. The columns represent distinct events provided by PADI-web (in pink), ProMED (in gray) and Empres-i (in yellow). The rows correspond to the bi-weeks of 2019, 2020 and 2021. Each cell of the matrix indicates the absence or presence of at least one event for a given time period and country.