dataesr/scientific-paragraphs-categorization huggingface dataset:
These outcomes, which include spin system matrix,  [7]  spectral peak lists,  [5, 8, 9]  and spectral peak assignments,  [10]  can be represented in different predefined tags in NMReDATA format. We are encouraging providers to expand the list of software programs that export their computational workspaces to NMReDATA format. For the example compound, the collected experimental data were processed using the Bruker CMC-se software for spectral analyses (initial evaluation, spectral peak picking, and assignment). CMC-se includes an option to export its results to NMReDATA format. After assigning peaks in CMC-se, the NMReDATA file will contain spectral peak lists of all 1-D and 2-D spectra and their associated assignments. Figure  2  shows the assignment as carried out in CMC-se. If the spectrometer software available at an NMR facility does not export NMReDATA, or as an alternative option to perform the assignment and to export it as NMReDATA, three online tools are available.
All are free to use. One is the "Quick Check" option of nmrshiftdb2, which does an automatic assignment (but allows editing this) and can export an NMReDATA file. The "Quick Check" module is available at the www. nmrshiftdb. org website. This needs a manual input of the structure and the shift lists (Figure  3  left) and produces an assignment from those (Figure  3 right ). An NMReDATA file can be exported once the assignment is finalized, using manual correction if necessary (Figure  3.
bottom). Another option to explore the contents of an NMReDATA file in a visual and interactive way is F I G U R E 1 A screenshot of the nuclear magnetic resonance (NMR) record of 5α-cyprinol. The nuclear magnetic resonance extracted data (NMReDATA) file (60004113. sdf) sits in the root directory of the record, and the Bruker data are contained in their original form. Each of the directories 10-15 contains the raw data for one spectrum. The processed data (or pdata) directories are part of the Bruker output, alongside files not visible in the file browser NMReDATA J_reader, a Hypertext Markup Language (HTML)-based tool. [11] It is multiplatform and can be operated either online or off-line. All the contents in the file are exposed in a structured view, and their information is presented according to their format (Figure  4 ).
The molecular structure is presented in an interactive 3-D display onto which chemical shifts, assignments, and couplings may be overlapped. JSmol, the JavaScript variant of Jmol,  [12]  is used for display of the structure as well as for data and file operations. JSpecView is used for display of spectra if they use the JCAMP format. Apart from its function as a viewer, NMReDATA J_reader can also be used to edit the NMReDATA tags composing the file. A special tool is included for adding implicit hydrogens F I G U R E 2 A screenshot of the data of 5α-cyprinol opened in Bruker CMC-se. [4] The data had been acquired using Bruker equipment, and the data were processed using Topspin, then opened in CMC-se and saved as NMReDATA from there and generating a 3-D structure that may be appended to the original 2-D structure.
| Formal validation For a formal validation of an NMReDATA file, two tools were developed. One is a Java-based software, called NMReDATA javatools, which is available as a library and also as a stand-alone Java program from https://github. com/NMReDATAInitiative/javatools. Figure  5  shows the NMReDATA file for 5α-cyprinol sulfate opened with the stand-alone version of the javatools. The second tool is a JavaScript-based software available from https://github. com/cheminfo/nmredata. Both apply a syntactical validation of the file, ensuring that all required elements are contained and that the format of the tags is correct. The javatools also do some basic logic checking, for example, whether atoms used in the assignment exist in the structure. There is no check for the chemical validity, for example, whether the structure is compatible with the given NMR data.
| Validation of chemical shifts The module for the validation of chemical shifts is conducted in collaboration with nmrshiftdb2  [13]  project. For the current example, the "Quick Check" module of the nmrshiftdb2 was used to verify the chemical shift lists and their assignments against the corresponding information calculated by nmrshiftdb2. The "Quick Check" module is available online on the "QuickCheck" tab of the https://www. nmrshiftdb. org website. This module accepts an NMReDATA file as its input and generates a validation report as shown in Figure  6. Of course, a validation report is also directly generated along with the NMReDATA file when shifts are entered manually as mentioned in Section 2. 2. 1. The report for each  13  C and 1 H shift gives a predicted value and calculates how close this is to the shift in the file.
| Validation of 2-D spectra correlations Further verification of the results can be conducted by using the logic for structure determination (LSD) software  [14]  to compare the archived molecular structure in an NMReDATA file to those suggested by LSD. The NMReDATA editor, included in NMReDATA javatools, allows exporting an NMReDATA file in the LSD input format. Because LSD requires 1-D 1 H and 13 C as well as 2-D COSY, heteronuclear single quantum coherence spectroscopy (HSQC), and HMBC spectra, those must be contained in the NMReDATA file. It will then list all possible structures compatible with these spectra. Figure  7  shows the three structures LSD suggests as a fitting solution for the measured spectra of the example compound. The structures are very similar, differing only in the -OH group positions, with the middle one being the correct structure.
| Publishing and deposition Because NMReDATA are data format, it cannot provide by itself a full solution for the problem of NMR data handling. For this, it needs to be integrated with repositories, databases, and search interfaces. We here sketch an ideal data deposition workflow to enable a full Findable, Accessible, Interoperable, and Reuseable (FAIR)compliant data handling (see Section 3). A requirement for deposition is that the proposed molecule and assignments pass all described validations. The data, together with the reports and the original NMReDATA file, will be saved in a repository and proposed for review. The selection of a suitable repository is left to the data producer. It could be a repository managed by a publisher, an institutional repository, or a third-party database.
Besides data integrity, persistent unique identifiers (e. g. , DOIs), versioning and query facilities for the data sets then improve findability and accessibility. Nmrshiftdb2 is an example of a database accepting NMReDATA uploads. The data for 5α-cyprinol sulfate have been uploaded to nmrshiftdb2 and are available at F I G U R E 7 The three candidate structures generated by logic for structure determination (LSD) for the example data. The structure in the center is the correct one for 5α-cyprinol https://nmrshiftdb. nmr. uni-koeln. de/molecule/60004113/ dataset/MRC_Methanol-D4%2B%28CD3OD%29. Figure  9  shows the deposited data in nmrshiftdb2. The raw data for each spectrum are available on the "Download" tab. Ideally, the submission of a spectral assignment article and of its associated data will be a seamless process.
We have decided to base NMReDATA on an existing format to make adaption easier by use of existing tools (e. g. , any molecular structure editor should be able to open an NMReDATA file and display the structure). This advantage outweighs that of an explicit KR language but will consider an Extensible Markup Language (XML) or linked-data serialization for the future. As our format leverages on the open nmrML raw data standard (XML with ontology support), this data section comes readily FM-F2 compliant. FM-F3-resource identifier in metadata: The NMREDATA_ID tag allows inclusion of IDs generated by repositories in the metadata of a file. FM-F4-indexed in searchable resource: This goal is achieved by the interplay of NMReData and repositories. Search functions are provided by the repositories (e.
g. , nmrshiftdb2 allows search by structure, spectrum, author, solvent, etc. ). FM-A1. 1-access protocol, FM-A1. 2-access authorization, and FM-A2-metadata longevity: These issues are mainly dealt with by the repositories. NMReDATA provide an important aspect of longevity, namely, a defined, vendor-independent format. Full metadata longevity has yet to be proven, but the community is building a rigid sustainability plan, which will contribute to NMReDATA metadata longevity. Longevity for the standard ensures, in turn, longevity for the data using the standard. Submission as NMR processed data standard to FAIRsharing is under discussion. FM-I1-use a knowledge representation language: See FM-F2. FM-I2-use FAIR vocabularies: Common terminology of the field has been used, and the format is published.
| EXAMPLE AND TEST DATA In order to enable testing of tools and to exemplify the format in practice, we have created a repository of NMReDATA files at https://github. com/NMReDATA Initiative/Examples-of-NMR-records. This repository contains various examples, which cover a wide range of use cases. It comes in conjunction with the NMReDATA javatools, which can be used to check all NMReDATA files in the repository for their compliance to the standard. Any additional file can be checked as well.
| CONCLUSION We have shown how the NMReDATA format streamlines the process of NMR processing, data handling, verification, and archiving of the results. We also showed how the NMReDATA facilitate the fulfillment of the FAIR principles and, together with appropriate repositories and journal publication policies, ultimately contribute to a fully FAIR compliant NMR data handling process in the future. The NMReDATA format is readable for both humans and machines. This ensures that the format can be widely used, even if appropriate software is lacking, and will always be readable. Apart from firmly establishing the format in the community, we plan to have a serialization of NMReDATA as linked data (for example, XML or Resource Description Framework [RDF]). NMReDATA also form the core of a wider initiative for chemical data, called CHEMeDATA.
NMReDATA levels: Most files are NMREDATA_ LEVEL 0, but in level_1, there are examples for ambiguous assignments. These are taken from the nmrshiftdb2 database. In line with many other repositories, nmrshiftdb2 can only hold unambiguous assignments, and text provides a hint that other assignments are possible. In contrast, the NMReDATA can hold it in a defined format. The files were manually edited to include the ambiguous assignment. The NMReDATA tools only read one assignment, which is checked in the java test files. A better handling of such assignments in processing software is encouraged by the NMReDATA project but not enforced. • Explicit hydrogens: Asunaprevir, 1,2-bis(pyridylethynyl).
ACKNOWLEDGEMENTS DS work was financed by  Phenomenal  ( H2020 654241 ) at the initiation phase of this effort, current work in kind contribution. HD is supported by  National Heart Lung and Blood Institute  grant  T32 HL007575. This project made use of the  NMR Uppsala infrastructure , which is funded by the  Department of Chemistry -BMC  and the  Disciplinary Domain of Medicine and Pharmacy. NES  gratefully acknowledges funding by the  Deutsche Forschungsgemeinschaft (DFG ,  IDNMR  project, grants  SCHL 580/3-1  and  SCHL 580/3-2 ).
An overview of the software mentioned in this paper (order as mentioned in the paper) License/availability Commercial, academic licenses available Commercial Commercial Free online Free online Free online LGPL v3. 0 URL https://www. bruker. com/service/support- upgrades/software-downloads/nmr. html https://mestrelab. com/software/mnova/ https://www. acdlabs. com/products/adh/ spectrusprocessor/index. php https://www. nmrshiftdb. org https://www3. uah. es/nmr_e_data/reader/ reader. htm http://nmredata. com/ https://github. com/NMReDATAInitiative/ javatools Name Vendor Language/operating system Functions Topspin/CMCse Bruker Windows, Linux, macOS Export: NMRRecord+NMReData Import/ Visualize/Validate: NMReData Mnova MestreLab Windows, Linux, macOS Export/Import: NMR Record/NMReDATA Spectrus Processor ACD/Labs Microsoft Windows 10 Export NMR Record/NMReDATA nrmshiftdb2 nmrshiftdb2.
• NMReDATA levels: Most files are NMREDATA_ LEVEL 0, but in level_1, there are examples for ambiguous assignments. These are taken from the nmrshiftdb2 database. In line with many other repositories, nmrshiftdb2 can only hold unambiguous assignments, and text provides a hint that other assignments are possible. In contrast, the NMReDATA can hold it in a defined format. The files were manually edited to include the ambiguous assignment. The NMReDATA tools only read one assignment, which is checked in the java test files. A better handling of such assignments in processing software is encouraged by the NMReDATA project but not enforced. • Explicit hydrogens: Asunaprevir, 1,2-bis(pyridylethynyl).
As for the RoBoost-PLSR framework, it proposes to apply the basic idea of gradient boosting: i. e. combining an ensemble of weak learners. The weak learners are defined here as weighted one-latent variable PLSR models. The weights are defined iteratively in order to reduce the contribution of outliers on the calculated model. The weak learners are then combined using an unweighted sum of the predictions of each weak learner. This strategy enables the weighting of samples in the calibration set independently of the number of latent variables (LVs) while considering the multivariate nature of the samples. The objective of this paper is to provide a study of the proposed new RoBoost-PLSR method using simulated and real data. These data represent different types of outliers that could be present in spectral databases.
Data and software RoBoost-PLSR was evaluated on three simulated datasets and one real dataset. Simulations were used to introduce controlled disturbances while the real dataset was used to confirm and support the simulations results. The algorithms were developed using the R software packages. RoBoost-PLSR was developed on the basis of "rnirs" (https://github. com/mlesnoff/rnirs) functions. The functions and data associated with RoBoost-PLSR are available on Github "RoBoost-PLSR" (https://github. com/maxmetz/RoBoost-PLSR). PRM was implement with the "prms" function available in the "sprm" (https://cran. r-project. org/web/packages/sprm/sprm. pdf) package.
Simulated data The three simulations were generated according to the generic framework proposed by Ref. [27]. Contrary to the simulation strategies usually used to evaluate robust methods, the data were not simulated from a real model. The data are simulated from a combination of spectral signatures, some of which are related to one or more variables to be predicted (Y matrix). The simulations were based on a combination of pure artificial spectra and controlled noises. The aim of each simulation was to reproduce the common external disturbances that can occur when calibrating a predictive model. It consisted of adding to the dataset an additional set of predefined outliers that have a negative effect on the performance of the models. The first simulation introduced pure Y outliers. The second simulation introduced contaminant induced outlier i.
Real dataset The real dataset consisted of NIR spectral samples acquired from two types of feed materials: soybean and meat and bone meal. Each sample-spectrum was associated with its Y-response i. e. the chemical reference measurement of its protein content. The spectra were measured with a Foss spectrometer in the spectral range [1100 À 2498 nm] with a 2 nm spectral resolution. These data were extracted from the "PROT" database provided by the CRA-W (Agronomic Research Centre of Wallonia, Belgium). This database was already used for the development and comparison of local methods  [28].
Evaluation strategies The purpose is to evaluate the behaviour of the newly introduced RoBoost-PLSR methods in presence of outliers during calibration. The calibrated model is then evaluated on a validation set. The reference against which all models were compared was a PLSR calibrated on a dataset without outliers (and will be designated as such). Roboost-PLSR was evaluated and compared with two standard regression algorithms: PLSR and PRM. In the case of the simulations, the weight parameters of PRM and RoBoost-PLSR were optimised according to the validation set. Only the results of the optimal (i. e. the parameters that provide the minimum value of RMSEP) parameters of RoBoost-PLSR and PRM were presented in the following section. The calibration sets were generated from 500 samples (400 inliers and 100 outliers).
The resulting models were studied with validation sets containing 500 inliers. The prediction performance of the RoBoost-PLSR method was studied also as a function of the proportion of outliers. It varied from 10% to 40%. These performances were compared to the reference model (PLSR without outliers). This study was carried out with the three simulated datasets. In the case of the real dataset, the weights parameters of PRM (using the Hampel function) and RoBoost-PLSR were optimised according to the validation set. Only the results of the optimal parameters of RoBoost-PLSR and PRM were presented in the following section. The calibration set was composed of 457 soybean protein (TTS) samples and 100 animal-protein (ANF) samples that represent the outliers. The validation was conducted on 50 additional samples of soybean and results were evaluated through Root Mean Square Error of Prediction (RMSEP).
Simulation 1: pure Y-outliers Fig. 1  presents the properties of the simulated dataset with Youtliers. Fig. 1a  shows that inliers spectra (in red) blend perfectly with the rest of the population. Likewise, there is no separation of the two populations of spectra when projected on the two first principal components of a PCA (see Fig. 1b ). The same behaviour is observed up to the 10 th component. In this simulation, the outliers are simulated to display significant differences in terms of Y. Fig. 1c  shows that the distribution of the outliers is not similar to the distribution of inliers. The samples are distinguished only by their response values (y) and not by their explanatory variables (X). Fig. 2  presents four curves showing the RMSEP evolution as a function of the number of LVs, for PLSR calibrated with outliers, the reference, PRM and RoBoost-PLSR, both calibrated with outliers.
Simulation 2: contaminant induced outliers Dataset 2 introduces X-outliers. The purpose is to simulate the impact of a contamination of samples during spectral measurements without any anomaly for the reference measures y. Fig. 5a  shows that such outliers (in red) overlap with standard observations. The difference between the two groups is very faintly apparent on the spectra plot. Fig. 5b  shows a separation of outliers from inliers on a projection onto the two first principal components of PCA. The two groups are contiguous though, which implies that some outliers could be confounded with inliers. Fig. 5c  shows the distribution of Y reponses for both outliers and inliers. Data are simulated so that the outliers responses match the same distribution as inliers. In practice, this situation corresponds to the possibility of conducting rigorous reference measurements in controlled laboratory conditions for chemical measures, while the spectral  measurements are high-throughput and possibly conducted in outdoor or uncontrolled conditions.
Simulation 3: X-outliers induced by microvariations of the measuring environment Dataset 3 introduces further X-outliers. The purpose is to simulate the effect of microvariations of the measurement environment, such as temperature or hygrometry shifts e. g. when there is a timelapse between spectral measurements. The occurrence of such minor disturbances can alter the resulting spectra in imperceptible ways, yet, sufficiently to deteriorate PLSR models. Fig. 9  shows the similarities between the outliers and the inliers. Spectra overlap so that the two populations are indistinguishable. Fig. 10  presents their projection PCA axis. The first two components cannot help to differentiate outliers. It is only from the fourth component that the two groups are discriminated. However, this axis represents less than 2% of the total variability which describes the difficulty to determine the presence of such outliers beforehand.
Real dataset and application: prediction of protein content The present section intends to deal with real agronomic data, with the example of a common animal nutrition application: the prediction of the protein content of feed materials and the presence of incorrectly categorised samples. In this database the samples resulting from animal bonemeal (noted ANF) represent the outliers polluting the regular soyabean cakes (noted TTS). In the proposed application, the proteins contained in ANF outliers, present spectral similarities with soya proteins (TTS). Therefore, even in minor proportions, these outliers can alter PLSR models. Fig. 15  shows the similarities between the outliers (ANF) and the inliers (TTS). Spectra overlap so that the two populations are indistinguishable. There is supposedly an overall difference in baselines, yet insufficient to separate the data into consistent clusters.
Fig. 17  presents the repartition of the weights attributed within the calibration of PRM. ANF samples weights are not distinguished from the TTS samples weights. This result explains the poor prediction performances of PRM on this real dataset. Fig. 18  presents the weights attributed to samples for the eight considered LVs within the calibration of RoBoost-PLSR. From the first LV, most ANF samples are assigned null weights along with a few TTS samples. From the second latent variable, all ANF samples weights are close to 0.
Conclusion & perspectives This article showed the potential of the RoBoost-PLSR method. This method offers a relevant solution for the calibration of PLSR models in the presence of various types of outliers. At this stage the proposed Algorithm is mainly based on weighting strategies within a of series unidimensional PLS. The method is designed to detect outliers within the calibration, through iterations where dissimilarity measurements take into account the hypothesis of robust linear models. The four evaluated applications demonstrate that the introduced outliers are predominantly detected and discarded from the model. As a result, RoBoost-PLSR is able to attain performance on par with the reference. One dataset was found to be particularly difficult to process by the PRM method. This dataset was the one with X-outliers.
It would be interesting to integrate within PRM a weighting criterion related to the X-residuals (as in RoBoost-PLSR). This would enable to observe the benefit of considering X-residuals compared to the benefit of estimating weights based on the score space alone. Eventually, RoBoost-PLSR proved to be a promising framework to deal with various practical issues. Further studies should be carried out in practical context for more diverse applications; Including smaller datasets, where it is yet undetermined if the estimation of weight criteria is still relevant/functional. Indeed, the observations carried out in this paper are based on large learning databases. This implies that it is potentially possible to apply stricter weights without degrading the prediction quality of the method. To this end, the method requires further studies on the following issues: Firstly, a comprehensive study regarding the weight functions and their optimisation should be carried out, in order to better adjust the models.