dataesr/scientific-paragraphs-categorization huggingface dataset:
The articles should have enough cases of systematic names, common or generic names of compounds and drugs, trade names, identifiers, acronyms, reference numbers of compounds and even formulas. In case of the CHEMDNER corpus the document selection criteria took into account primarily the scientific discipline of the journals and publication dates. The following steps were used to select abstracts for the CHEMDNER corpus. Step Step 2: Selection of the top 100 journals for each category based on the journal impact factor. Step 3: Selection of journals that had at least 100 articles. Step 4: Selection of articles that were published in 2013 in English, with abstracts and links to full text articles in the PubMed database. Step 5: Selection of articles that belonged to the various subject categories.
Step 6: Randomization of the abstracts and selection of 10,000 records Step 7: Splitting into three datasets: 3500 (training set), 3500 (development set) and 3000 (test set) abstracts. The CHEMDNER corpus therefore contains representative articles for a range of chemistry-related fields. It is sufficiently large to cover the most relevant mention types and naming variability that are encountered in the scientific literature, allowing both to generate a predictive model and train an NER recognizer on a subset of abstracts as well as evaluate the performance on a distinct test collection. We selected recent publications to make sure that the corpus would be useful for the detection of chemical entities in new abstracts as soon as they get published. It also covers journals with an high impact in the field based on its impact factor and the number of published articles by that journal.
The selection of text spans was done at the character level, we did not allow nested annotations and distinct entity mentions should not overlap. Each text span was selected according to the annotation guidelines and classified manually into one of the CEM classes. Figure  2  provides a very general flowchart of the CHEMDNER annotation process resulting in the annotations used for the BioCreative CHEMDNER task. The annotation modus operandi itself relied on the chemical background knowledge of the curators (and comprehension of the guidelines) during the labeling of the chemical entity mentions. We recommended the curators to consult existing chemical knowledgebases in case of doubts. They could crosscheck information from reference sources such as Wikipedia, and chemical databases (ChEBI, DrugBank, etc.
) or even carry out online searches to make sure that the annotations were compliant with the guidelines. Annotators knew that the text collection corresponded to PubMed abstracts and they were provided with pointers to the original PubMed records. No additional meta-data or automatically pretagged text was provided. We initially experimented with a pre-tagging strategy using a specially adapted version of the MyMiner system  [42]  that included the option of pre-tagging the text with Oscar4 and then manually correcting the labels. The automatic pre-tagging strategy had limitations in terms of performance and had the potential of biasing the curation results. Moreover, as we also requested the classification of mentions into one of the seven chemical mention classes, we finally chose to use an exclusively manual annotation approach.
Annotation format In principle, the information represented in annotated textual data can be represented in various alternative formats reflecting how the annotations look like. For choosing the annotation format of the CHEMDNER corpus, several criteria were important. First of all, the format should be easy to use for building NER systems, thus it should be simple and easy to modify. There was a clear separation of the entity annotation format and the exchange (dump) format of the released CHEMD-NER corpus. This means that we kept the annotations separate from the actual text (the information on the location of the entity mentions is stored in a different file from the actual raw text). We used a standoff annotation format by specifying in a separate file the character location. Using character offsets instead of token location was particularly important for the CHEMDNER corpus because it makes it easier for the corpus consumers to use their own text tokenization strategy.
We avoided using a complicated XML schema for the initial baseline release. We examined some basic recommendations provided by the Linguistic Annotation Framework (LAF) for data distribution  [44]. All records used for the CHEMD-NER corpus were distributed as plain text, UTF8encoded PubMed abstracts in a tab-separated format with the following three columns: article identifier (PMID, PubMed identifier), title of the article, and abstract of the article. The baseline entity annotation file had a tab-separated format with columns corresponding to the article identifier, the part of the document processed (T: title, A: abstract), the start and end characters offsets of the chemical, the text string of the chemical entity mention and the corresponding chemical entity mention class. Example cases of the entity annotation file can be seen on Figure  2.
CHEMDNER corpus overview The CHEMDNER corpus is currently the largest chemical entity corpus annotated with a high degree of granularity for PubMed abstracts. A detailed summary of the total number of generated annotations of the entire CHEMD-NER corpus as well as divisions according to each of the three corpus subsets (training, development and test set) can be seen in table  1. The CHEMDNER corpus contains a total of 84,355 manual chemical mention annotations; corresponding to 19,805 unique chemical name strings extracted from 10,000 exhaustively examined abstracts. Although the majority of the abstracts did contain at least a single chemical mention (a total of 8,301 abstracts), this table also shows that a fraction of the abstracts did not have any chemical mention at all. This smaller subset can be used as a true negative dataset of abstracts that do not mention SACEMs.
Until now, such a true negative dataset for chemical entity mentions was missing. This table also shows that the annotation density across the various datasets is coherent and this in turn reflects that the CHEMD-NER corpus is balanced and that overall the three subsets have a comparable (relative) number of chemical mentions. The used abstracts were derived from a total of 203 different journals from heterogeneous chemically related topics (see the subsection on Document selection and sampling). When examining the annotations according to the chemical mention classes, as shown in the lower part in table 1 the quantitative importance of two chemical mention classes becomes obvious, namely of the mention classes TRIVIAL (30. 36%) and SYSTEMATIC (22. 69%). These two classes make up more than half of all the annotations.
In case of the CHEMDNER dataset, the vocabulary transfer was of 36. 34% when uniting both the training and development set names before comparing them to the test set entity list. It was 27. 77% when using only the names from the training set, and 27. 70% when using only those from the development set. In order to get a general idea on what the CHEMDNER corpus contains we carried out a simple statistical corpus analytics to summarize the corpus content. Figure  3  shows the statistical profile of the chemical entities contained in the CHEMDNER corpus by examining the distribution of the chemical mentions. It illustrates the CHEMDNER corpus rank/frequency profile, reflecting the relation between chemical entity mention frequency and the corresponding entity rank when ordering chemicals according to the resulting absolute frequency.
The simplest IAA score is the percentage agreement experts. The IAA analysis of the CHEMDNER corpus was conducted using a random sample of 100 abstracts chosen from the entire dataset, asking the curators to annotate the data set independently. The result of the IAA study constitutes a sort of upper boundary for the expected automated prediction performance. An interannotator agreement of 91% was obtained when exact matching of the chemical mentions was used without considering the label of the SACEM classes. When the SACEM class annotation of the mentions was also considered, the IAA was of 85. 26%. Manual inspection of the conflicting annotations showed that the main source of discrepancies were missed annotations by either one or the other annotator and not true annotation errors or differences in the mention boundary definition.
CHEMDNER corpus test set predictions Most of the existing biomedical corpora are not distributed together with the results of automated systems predictions trained or tested using these datasets. This makes it impossible to do a more exhaustive and detailed analysis of the differences between various methods at the level of concrete annotations. When a corpus was used to generate multiple predictions, for instance by different teams of a community challenge, it is interesting to check various run combinations or construct an ensemble systems with improved performance over the best single run. The competitive performance of ensemble systems has been demonstrated for instance for the recognition of gene mentions  [46]  or the detection of protein interactions  [47] , showing in some cases that even low scoring runs can positively contribute to the ensemble system performance.
CHEMDNER silver standard corpus Due to the considerable workload required for the construction of manually annotated corpora, some efforts have been made to construct automatically tagged text collections generated by different systems. Despite obvious limitations when relying on automated tagging, one advantage of this strategy is that they can generate very large datasets. When assuming that the automated tools have an acceptable performance, the combination of multiple systems can generate labels with an acceptable quality. The BioCreative metaserver constituted a pioneering work in the integration, alignment and visualization of multiple automated predictions, including the annotation of gene/protein mentions and handling their character overlaps  [49]. The use of silver standard corpora as training data was explored for the implementation of chunkers of biomedical text  [50]  and NER systems  [51].
The CHEMDNER silver standard corpus can be interpreted as a sort of collaborative effort to annotate chemical entities. For this dataset there are still aspects that would benefit from further analysis, such as alternative harmonization strategies of the mentions or a comparative analysis on the performance of systems trained on silver-standard corpora versus gold standard corpora. The release of automatically extracted chemical mentions from the entire PubMed database (of systems trained the CHEMDNER corpus) would demonstrate how scalable those methods are and help in the curation of chemical data from PubMed. Moreover, determining ways to differentiate those mentions that are of practical relevance for curators still needs additional analysis, but some preliminary studies that took into account simply the position of the chemical names in the text and restricting the selection to certain sections of the abstract showed interesting outcomes  [26].
The performance of these tools on other documents, including patents and full text articles could potentially highlight both the adaptability as well as challenges associated with each particular document type. For an enhanced version of the CHEMDNER corpus, aspects that could improve the impact of this resource include a more granular classification of the SACEM classes. With this respect, a simple ontology or hierarchical classification of chemical entity mention classes would be important. The underlying classes would have to be useful to improve automatic detection of entities and to facilitate the normalization of mention to either structures or chemical databases. Some mention classes can only be normalized using a dictionary based approach, others using name to structure software. Some of the current entities contained in the CHEMDNER dataset cannot be directly normalized without some more granular mention subtypes (e.
g. in case of the SACEM class FAMILY). Well-specified, generally used workflows of the underlying normalization process of chemical entity mentions to structures/databases are currently missing. We also think that a more granular annotation strategy could help to improve the recognition of other entity mentions such as genes and proteins. In addition to a more detailed chemical mention classification, some annotations would benefit from a more granular labeling at the level of substrings, for instance in case of hybrid chemical mentions (e. g. chemical mentions that are formed by strings belonging to different SACEM classes like SYSTEMATIC and TRIVIAL). In the case of chemical mentions of the class MULTIPLE, which cover chemical entities that appear in form of separated or unconnected expressions (discontinuous) they are being annotated together in order to generate integrated forms.
CHEMDNER abstracts, split into chemical disciplines (subject categories, first column; MULTIDISCIPL. CHEM. : Multidisciplinary Chemistry) On average, the number of chemical mentions per abstract was of 4. 39, almost half when compared to the chemistry-related test set abstracts. The number of predicted unique compound names per abstracts was 0. 83 (compared to the 2. 10 of the test set). These numbers partially reflect also the fact that in case of this random background set, on average only 52. 80% of the abstracts did contain chemical mentions. When extrapolating these numbers to the entire PubMed database, of over currently 14,8 million records with abstracts, we would obtain over 12 million unique chemical names with more than 65 million mentions. However these numbers have to be taken with care, because the background set corresponded to recent articles, while the PubMed database hosts a considerable number of older publications.
BINoculars is a tool for data reduction and analysis of large sets of surface diffraction data that have been acquired with a two-dimensional X-ray detector. The intensity of each pixel of a two-dimensional detector is projected onto a three-dimensional grid in reciprocal-lattice coordinates using a binning algorithm. This allows for fast acquisition and processing of high-resolution data sets and results in a significant reduction of the size of the data set. The subsequent analysis then proceeds in reciprocal space. It has evolved from the specific needs of the ID03 beamline at the ESRF, but it has a modular design and can be easily adjusted and extended to work with data from other beamlines or from other measurement techniques. This paper covers the design and the underlying methods employed in this software package and explains how BINoculars can be used to improve the workflow of surface X-ray diffraction measurements and analysis.
The result of these developments is that the data acquisition rate has increased by six to seven orders of magnitude, from typically 1 point per second to millions of points per second, and so the amount of data collected during one experiment has increased dramatically. Today's computer hardware can keep up with that increased demand, but the development of software to analyse these large data sets has been lagging behind, which has kept most users from exploiting the full potential of modern surface diffraction beamlines. BINoculars aims to fill this gap by taking a novel approach to data reduction in surface X-ray diffraction (SXRD) experiments. Currently, data reduction is typically performed by integrating a region of the image from a two-dimensional detector, sometimes coupled with another integration to determine the background level.
BINoculars takes a series of images from a two-dimensional detector, calculates for each pixel the corresponding reciprocal-lattice coordinates ðhklÞ, and reduces the image collection to a single data set by averaging the intensities of pixels taken at identical ðhklÞ positions (within a user-specified resolution). This transformation and averaging are illustrated in Fig. 1. To allow online analysis during data acquisition, BINoculars can run on a high-performance cluster to process large data sets. For example, the result of a full hour of continuous scanning, adding up to a total of approximately 10 10 pixels, can be processed in a matter of minutes. In addition, BINoculars provides tools to further process the data, including visualization, curve fitting and crystal truncation rod integration.
Implementation BINoculars has been written in Python, an open-source scripting language that is very suitable for scientific software, thanks to its powerful and clear syntax and the extensive support for numerical calculations via the NumPy and SciPy libraries  (Perez et al. , 2011). BINoculars has been designed to process large data sets and its operation is usually CPU bound. An ordinary desktop computer can easily take many hours to deal with a data set obtained in 1 h (e. g. 10 10 pixels with 16 bit per pixel). To allow online analysis during data acquisition at a beamline, BINoculars can use a computing cluster to distribute the load over multiple computers. To keep BINoculars modular and flexible, the workflow for processing data is separated into four modules: the dispatcher is in charge of the whole process and handles job parallelization and distribution, the input class gathers the experimental data, and the projection class converts the raw data into the coordinates of choice.
Finally, the processed data are binned on a discrete grid and stored in a space class, which provides generic tools for further analysis. The specific behaviour of the dispatcher, input and projection modules can be A graphical overview of the process performed by BINoculars. The data displayed here are a rocking scan through ðh; k; lÞ ¼ ðÀ2; 0; 1:85Þ, the diffraction pattern from a Pt(110) surface. Upper panel, raw data acquired by the two-dimensional detector with the corresponding angles !, , from a six-circle diffractometer  (Vlieg, 1997). Middle panel, the diffractometer angles are mapped onto reciprocal-space coordinates ðhklÞ. Lower panel, the averaged intensity of pixels on a regular grid of volumetric bins (voxels) is calculated. Part of the bins are removed for clarity. Note that the data in reciprocal space in the lower panel are constructed from a series of images over a larger range and with a denser sampling in ! than shown in the other panels.
The input class takes care of all these technicalities, and writing a new input is the most important task when adding support for another experimental setup. In addition, the work done by the input class is often the most computationally intensive step in the entire process of BINoculars. Although it is likely that a significant performance gain can be obtained by implementing this part of the code in C rather than Python, the code is intimately related to the continuously evolving experimental practices of the beamline, and as such a high-level language is a better choice. The next step is performed by the projection class, which converts the data collected by an input class into the appropriate coordinate system for binning. For an SXRD experiment, the input class will typically return a series of detector images with corresponding diffractometer angles for each pixel, and the projection class will convert the angles into reciprocal-space coordinates ðhklÞ for each image.
Then the pixel intensities are accumulated at every discrete grid location, using the histogram operation bincount from NumPy. In addition, the number of contributions per coordinate is stored in order to calculate the mean intensity per bin rather than the integrated intensity. This binning operation is the essential data reduction step performed by BINoculars: hence the name of the program. The dispatcher orchestrates the entire process: it asks input for the sequence of images from the two-dimensional detector, delegates it to the appropriate projection and performs the binning operation by feeding the projection result into a space. Two dispatcher implementations are currently present: one for local processing using multiple processor cores on a single computer, and one that distributes tasks over a high-performance cluster managed using OAR (https://oar.
BINoculars provides various tools for analysis, including mathematical operations, plotting and exporting. For ultimate flexibility it is possible to directly manipulate a BINoculars space from a Python script, but in many cases the standard tools are sufficient. In addition, a fitting and integration tool is available to calculate the structure factors of a crystal truncation rod to be directly inserted into the fitting program ROD  (Vlieg, 2000). It takes as input a reciprocal mesh which it slices by a user-specified resolution and the resulting data are either fitted (typically with a two-dimensional Lorentzian) using a least-squares optimization or simply integrated. The error is estimated from equation (  2 ), as will be described in more detail in the next section.
The differences between this diffraction pattern and the pattern reported in the literature for the same surface in ultra-high vacuum are not fully understood; however, they are likely to be due to the sample preparation procedure in the electrochemical cell. The data set was acquired in just 111 min. Fig. 5  shows a data set that is strongly affected by background intensity. Like Fig. 4 , the data set is built up from a series of ! scans, and in this case those scans are clearly visible as arcs after processing by BINoculars. The problem is that the background intensity depends not only on the position of a pixel but also on the position of the camera. In other words, when moving the camera by only a small amount, such that a certain feature remains in the field of view (in this case a move in between the consecutive ! arcs), the contribution of the background intensity to that feature can change significantly, as illustrated in Fig.
3. This particular data set has been obtained using the high-pressure flow reactor setup at ID03 (van  Rijn et al. , 2010) , which has a beryllium dome around the sample. The dome acts as a strong X-ray scatterer only 14 mm away from the sample. It is not possible to lower the resulting background intensity using slits without dramatically reducing the aperture of the detector. However, for this sample it proved possible to estimate and subtract the background level, the result of which is shown in the middle panel of Fig. 5. For each pixel of the detector, the background was estimated as the average intensity of that pixel in all images in a single ! scan. The average is calculated by fitting a Poisson distribution, as this turned out to give better rejection of outliers (which are in fact the diffraction peaks) than a simple mean or median calculation.
The data set is then segmented into small intervals along l. Each section is analysed separately by a numerical integration algorithm to calculate the structure factors as a function of l. This method has recently been described by  Drnec et al. (2014). At lower l the detector is more perpendicular to the surface, and for some samples it might be useful to augment the l scan with rocking scans at low l. BINoculars can easily deal with such a hybrid data set, since it starts by processing the data into the three-dimensional rod, after which the integration procedure (taking place in reciprocal space) is performed completely independently of the original character of the raw data. The fourth example demonstrates that BINoculars can be used with other coordinate systems than ðhklÞ. Fig. 7  shows the reflected intensity from a (PbSe) 4+ (TiSe 2 ) 4 sample.