dataesr/scientific-paragraphs-categorization huggingface dataset:
Improved Workflows for End-to-End Systems The early steps of a workflow (telecommunication, quality assurance, and communication) are usually the responsibility of the owner of the platform/instrument. JCOMM and other relevant bodies and initiatives, such as the OBP project, strive to standardize these steps and to facilitate efficient pathways from instrument/platform to databases. The workflow for ocean data is at its most intense during this period: many applications are run in real-time and demand rapid processing steps from the point of measurement to the point of use (assimilation, analysis, and monitoring). The workflow must accommodate high-volume data streams, but often with repeated and similar characteristics (e. g. , satellites), and at relatively low volume with more individual characteristics (e.
In reality there are multiple manifestations of such workflows, all aimed at similar outcomes but with slightly different assumptions and implementations, and often with different spatial-temporal windows and variables. As with ocean and climate models, such diversity can be a good thing, but it also comes with risks in terms of the efficiency and integrity of the database as a whole. In summary, the explosion in observing technology over the last decade has exposed weaknesses in current global data management practices. The ever-increasing number of sensors and the volume of data they are producing demonstrates a need for improved data acquisition and distribution systems to meet the current and future needs of users, which equally applies to near-real time, delayed mode, and reprocessed data.
Furthermore, integrated data and metadata will be available that contains comprehensive dataset information, such as historic details on current and past data codes and formats. The MCDS data flow is based on a three-tier structure (Figure  8 ) of Data Acquisition Centers (DAC), Global Data Assembly Centers (GDAC) and Centers for Marine-Meteorological and Oceanographic Climate data (CMOC) that standardize data flows from operators to end users. The general structure coordinates data flow through different levels of the MCDS providing a traceable and robust flow of data of known quality: • DAC is the basic level of the MCDS and receives data directly from the measurement source, applies basic quality/duplicate checks, and forwards the data to an associated GDAC. The definition of this type of center encompasses several of the IODE structural elements, including the NODC and ADU; • GDACs combine the streams of associated DACs to produce a global aggregation for a specific observation system, apply further quality/duplicate checks, and provide the data to an associated CMOC; • CMOCs collect data from GDACs, aggregate all the relevant data types for a specific set of environmental variables, and produce integrated datasets and products for end users.
The World Ocean Database (WOD; https://www. nodc. noaa. gov/OC5/WOD/pr_wod. html, Boyer et al. , 2013 , also available through http://wod. iode. org) was endorsed as a CMOC for ocean profile data in 2017. The WOD is a database operated from the U. S. National Centers for Environmental Information (NCEI) and is also a project of the IOC International Oceanographic Data and Information Exchange (IODE). The WOD aggregates historic and recent subsurface ocean profile EOVs. The WOD role as CMOC is to provide all available ocean profile data in a uniform way to the public through both machine-to-machine discovery and individual discovery and download. As a CMOC, the WOD can flexibly prepare metadata and discovery options, which feed into the WIS and ODIS systems for dissemination with other types of ocean and marine meteorological data.
The JCOMM MAN has carried out a Strength and Weaknesses, Opportunities and Threats (SWOT) analysis (Table  2 ) to attempt to devise a better fit-for-purpose collaboration between IOC and WMO, in view of the changes that will involve JCOMM in 2020. In general, the weaknesses outlined in Table  2  are due to the technical nature of the three program areas, which have not included any periodic review of stakeholder needs. A bottom-up approach has been taken in the program area activities, which has been adequate over their 20 years joint activity development period, but they now must be revisited. In particular, there is a weakness in the connections between the observational program area and the ocean infrastructure of IODE and MCDS. Many real-time OPA networks have defined both their real-time and delayed mode data management strategies separately from the IODE structural elements described in section Data Management Systems, From Near-Real Time to Delayed Mode.
Furthermore, there has been little cross-fertilization between the IODE system and the MCDS itself. The technical institutions represented by the WMO members and the IOC members states require international best practices across communities, so they can take advantage past experiences in building a fit-for-purpose met-ocean data exchange system. JCOMM DMPA began to define a strategy for the met-ocean archiving system but failed because information about the working practices of IODE networks was not fully considered. In an attempt to bridge the gap, IODE is developing an ODIS system that will initially be in the form of a "Catalogue of Sources" (http://catalogue. odis. org), aimed at being an online searchable catalogue of existing ocean-related web-based data and information sources/systems of as well as products and services.
It will also provide information on products and visualize the landscape (entities and their connections) of ocean data and information sources. This may lead to a design for combining real-time and delayed mode data flows and the ongoing storage of the data. However, this process is far from being accomplished and will require future activities to focus on the strategic objectives. The future ODIS will be an eenvironment in which users can discover data, information, and associated products or services provided by member states, projects, and other partners associated with IOC and WMO. The system will aim to align itself with accepted community data management principles, such as the FAIR principles  (Wilkinson et al. , 2016)  and, where appropriate, interoperate with existing data exchange solutions.
ODIS will also allow additional metrics to be defined that record the data flow and if the data are securely archived within the IODE network and links with the satellite data centers. To achieve this objective, JCOMM began defining standards for networking between IODE, OPA networks, and WIS (https://www. jcomm. info/index. php?option=com_oe&task= viewDocumentRecord&docID=20327). The Inter-Program Expert Team on Integrated Marine Meteorological and Oceanographic Services (IPET-MOIS) will continue this work (Figure  11 ). This group will define a formal procedure to interconnect all the relevant WIS to the IODE centers and eventually ODIS, using best practices with standards-based data and metadata exchange. With real-time data as its initial focus, IPET-MOIS will identify marine observing assets that have real-time value and facilitate the inclusion of these data on the GTS and WIS.
Another major weakness of the system is the lack of strong links between the three pillars of the met-ocean value chain Strengthen global, regional and sub-regional mechanisms 0. 1. 1 5. A: Strengthen the work of global and regional centers 5. B: Strengthen global, regional and sub-regional mechanisms to provide support for weather, climate and hydrological services 6. Increase education and research opportunities 0. 1. 2 6. A: Improve access to and provision of fellowships 6. B: Strengthen applications of research findings (Figure  2 ). Neither the OPA networks nor the IODE/MCDS infrastructure get requirements from the forecasting and/or downstream services. As outlined in the user survey, it is clear that maritime safety services do not as yet benefit from the existing new observation data streams and the advanced metocean forecasts.
FUTURE FOCI The areas identified as weaknesses and opportunities in Table  2  are considered the starting points for the focus of the new Joint IOC and WMO collaborative activities, which will begin in 2020. The WMO reform advice was to abolish JCOMM and establish a new joint collaborative board between WMO and IOC that will take on all the activities of the present JCOMM but within a new governance structure and with new working practices. The new Joint Collaborative Board should develop and provide strategic advice, directly derived from the overall strategic plans of IOC and WMO, to achieve joint objectives with the mandate to propose decisions directly to IOC and WMO governing bodies. It will also provide a forum for WMO and IOC bodies and external partners, to negotiate and fast-track projects and joint work plans, which include objectives that cut across meteorology and oceanography and WMO and IOC regional structures to enhance implementation and capacity development, and the value chain connecting observations, data management, forecasting systems, services, and research.
In this future structure, the expert teams will become involved in active IOC programs, such as GOOS, IODE, the IOC Tsunami Program, and the new two restructured commissions of the WMO (to be approved in June 2019). One of the major tasks of the new Joint Board will be to ensure the sustainability of the JCOMMOPS service, its connections with the IODE centers, and the full development of a fit-for-purpose data management system for the global met-ocean data and information system. The following goals form the backbone of the JCOMMOPS roadmap over the next 5-10 years: (i) Reach its metadata "gold" standard for all networks itemized in its initial Terms of Reference (Argo, OceanSITES, DBCP, SOT, GO-SHIP, GLOSS) and develop standard metadata services. (ii) Expand its support and monitoring capacity; to new OCG networks (such as OceanGliders) and develop a regional pilot on e.
g. , Mediterranean Sea. (iii) Enabling a responsive and truly global monitoring system. Over the long term, JCOMMOPS could expand its monitoring capacity to most of GOOS/WIGOS regions, and give visibility and monitoring capability to a cluster of coastal observing systems, with the cooperation of national focal points. The Joint Collaborative Board will oversee the work of the expert teams from an overall strategic point of view, and will organize the necessary exchanges with other programs, including GODAE OceanPredict, WCRP, GeoBluePlanet, etc. IMO and IHO should be invited to engage in the Joint Board, and particularly in the activities of maritime safety and bathymetric mapping, with an eye to the synergistic opportunities. A major focus of the new Joint Board will be to organize projects that will advance the sharing of data and training on met-ocean products and information, including the new ocean forecasts, analyses and reanalyses, and the early warning systems, navigation safety services, and emergency response decision support systems.
ACKNOWLEDGMENTS Several projects and national Institutions supported the authors of this paper. NP has received funding from the  European Union 's  Horizon 2020 research and innovation program  under grant agreement No. 633211  (AtlantOS). DL and SB acknowledge support from  NOAA  ( FundRef 100007298 ). KO'B was partially funded by the  Joint Institute for the Study of the Atmosphere and Ocean (JISAO)  under NOAA Cooperative Agreement NA15OAR4320063, Contribution No. 2018-0176.
INTRODUCTION One of the most critical aspects of handling large biological datasets is identifying and accurately quantifying similarities and differences in the data. Clustering is one of the most popular ways to do this, and many clustering algorithms with specific biological applications have been developed over the last two decades. However, despite the availability of numerous clustering algorithms, three key issues still remain unaddressed. Most clustering methods use heuristics to assign clusters and do not come with rigorous statistical performance guarantees. Second, existing clustering methods work well only for specific datasets. A comprehensive benchmarking of 13 well-known methods across 24 datasets revealed that there was no universal best performer; rather, methods typically worked best for the datasets that they were specifically designed for  (Wiwie et al.
, 2015). Even in our own experience, the choice of an appropriate clustering method is highly specific to the dataset being analyzed  (Das et al. , 2012 (Das et al. , , 2013 (Das et al. , , 2015;; Vo et al. , 2016). Furthermore, clustering methods typically work for generating either overlapping or non-overlapping clusters, but not both. These led us to envision a clustering approach that comes with rigorous statistical guarantees regarding both cluster identification and assignment of variables to clusters. The method would be generically applicable across a wide range of datasets, as there would be no assumptions regarding how the data were generated. Furthermore, the method is designed to estimate the type of clusters that best fit a dataset, which in some cases may be overlapping and in other situations non-overlapping.
Here, we report LOVE, a robust and scalable (Table  2 ) latent factor model-based clustering method with all the above properties. We apply LOVE to three datasets with very different properties and correlation structures (Table  1 ) and show that LOVE generates stable, biologically meaningful, and accurate clusters in all three cases, outperforming state-of-the-art methods. Furthermore, we demonstrate that LOVE consistently outperforms 13 previously benchmarked state-of-the-art clustering methods  (Wiwie et al. , 2015)  across datasets (Table  3 ).
The unmixed variables are then used to identify the unique clusters (Figure  1B ). The final step comprises assignment of the mixed variables to multiple clusters based on the membership matrix A (Figure  1C ). There are three primary tuning parameters in LOVE-determination of cluster anchors using delta, membership matrix determination using lambda, and thresholding during assignment of variables to clusters using mu. The mathematical details of each step, as well as the associated parameter tuning, are provided in the Transparent Methods. We also provide code to implement each step of LOVE as well as associated detailed documentation (Data S1). LOVE is highly scalable as a method. The determination of pure and mixed variables is an O(n^2) algorithm, where n is the number of variables to be clustered.
The estimation of the membership matrix A involves solving K linear programs (where K is the number of latent factors). To practically test the scalability of LOVE across datasets with different numbers of variables, we tested LOVE on a wide range of datasets from a few hundred (10^2) to a million variables (10^6). Runtimes on a single core ranged from under a second to $140 h (Table  2 ). Thus even for the largest dataset of a million variables, this extrapolates to a runtime of a few hours on a typical server or cluster node. Thus LOVE is highly scalable and can efficiently cluster even ultralarge datasets in hours.
Overlapping Clustering Using LOVE To test overlapping clustering using LOVE, we used a previously described compendium of human gene expression data  (Das et al. , 2012). The dataset corresponds to expression measurements for 16,134 genes across 114 different points in the cell cycle. Using LOVE, we obtained >1,000 overlapping clusters, which corresponded well with prior biological expectation. For example, we obtained three overlapping clusters, where cluster 1 contained the genes RWDD3, BRD30S, and IRF2; cluster 2 contained the genes BRF3OS, IRF2, CTGF, INHBA, and INHBB; and cluster 3 contained the genes INHBA, INHBB, and COLA1A2 (Figure  2A ). Based on KEGG pathway annotations, RWDD3 is associated with nuclear factor (NF)-kB signaling and CTGF is associated with leishmaniasis, whereas the genes shared between clusters 1 and 2-BRD3OS and IRF2-are known to be associated with both NF-kB signaling and leishmaniasis (Figure  2A ).
We found that hubs were assigned to significantly more clusters by LOVE than they were by Cmeans clustering or ClusterOne (Figure  2C , p < 10 À10 using a Mann-Whitney U test), suggesting that the overlaps detected by LOVE are more consistent with prior biological expectation than the overlaps detected by other methods. We also checked whether overlapping genes are also assigned to appropriate clusters, i. e. , the clusters from LOVE coincide with prior biological expectation as defined by protein network modules. We found that the network distances (i. e. , minimum path length between the two nodes) between overlapping genes and non-overlapping genes from the same cluster were low. This distribution of network distances had a median of 3. 3, and 75% of network distances were under 3. 6. Thus most overlapping and non-overlapping genes from the same cluster are within four hops (<25% of the network diameter) away in the protein network.
Non-overlapping Clustering Using LOVE Most methods are good at either generating overlapping or non-overlapping clusters  (Wiwie et al. , 2015). However, due to the inherent formulation of LOVE, it can be used for either purpose. To test the effectiveness of LOVE in generating non-overlapping clusters, we chose a recently published dataset of humoral immune measurements from 19 human subjects from two distinct clinical phenotypes-long-term HIV controllers and chronic progressors  (Sadanand et al. , 2018). For each subject, 18 different measurements of antibody-effector functions and titers were available at four different time points, corresponding to a total of 72 measurements  (Sadanand et al. , 2018). This dataset is different with regard to several key aspects from the earlier gene expression dataset.
First, the desired clusters here are non-overlapping as HIV controllers and chronic progressors are clinically distinct groups and are known to be very different in terms of their humoral responses  (Alter et al. , 2018; Sadanand et al. , 2018). Second, the sources of biological and technical variance in the two datasets are different. In terms of biological variability, the modulation of transcript expression levels across time points in the cell cycle is structurally very different from variation across human subjects with different clinical phenotypes. The extent of technical noise is also different as microarray measurements are relatively noisy, whereas this dataset comprises serological measurements collected using modern methods. Finally, the number of entities being clustered (number of input variables for LOVE) is also very different.
The gene expression dataset had >16,000 genes profiled over 114 different points in the cell cycle. The dataset of controllers and progressors have 19 human subjects. The differences across these two datasets reveal the inherent variation across different biological datasets. Testing LOVE on two extremes provides an opportunity to benchmark how the clustering method performs at different ends of the spectrum. The 19 human subjects were split into two clusters-one of that comprised eight progressors and two controllers and the other comprised seven controllers and two progressors (Figure  3A ). As each cluster primarily comprises individuals from one clinical phenotype, the latent factors in this case can be interpreted as the average humoral signature corresponding to each phenotype. Thus LOVE comes up with biologically meaningful latent factors for this dataset too, illustrating that the model formulation is both intuitive and interpretable.
Next, we evaluated the performance of LOVE in terms of accuracy, true positive rate, and true negative rate. Although clustering by definition is unsupervised, we were able to measure these metrics for this dataset as we already know the clinical outcomes for each human subject. We assumed that the ideal result would be two clusters-one comprising only controllers, and the other comprising only chronic progressors. Based on this definition of ground truth, we obtained an accuracy, a true positive rate, and a true negative rate each of $80% for LOVE (Figure  3B ). We contend that an accuracy of $80% is really high, especially for an unsupervised method, as previously even a supervised approach had a median classification accuracy of $75%  (Sadanand et al. , 2018). We also found that LOVE outperformed both Cmeans clustering and ClusterOne in terms of all three metrics-accuracy, true positive, and true negative rate (Figure  3B ).
LOVE on High-Dimensional Data Finally, we sought to evaluate LOVE on a high-dimensional dataset of vaccine-induced humoral immune responses. Recently, we found that the route of immunization, even for the same immunogen, can modulate mechanisms of protection in the context of vaccination against simian immunodeficiency virus (SIV)  (Ackerman et al. , 2018). Our study had three vaccination arms-IM239 (administration of the SIVmac239 immunogen intramuscularly), IM mosaic (administration of a mosaic envelope immunogen intramuscularly), and AE239 (administration of the SIVmac239 immunogen via inhaled aerosol). We found that each vaccine arm induced a distinct profile of humoral immune responses  (Ackerman et al. , 2018). We sought to evaluate whether these differences that we had captured using a supervised approach  (Ackerman et al.
, 2018). Together, these results show that LOVE is very accurate even when clustering a high-dimensional dataset and significantly outperforms existing methods. We also evaluated how stable LOVE is across the three key tuning parameters-delta (cluster anchor tuning), lambda (membership matrix tuning), and mu (latent variable tuning). A grid search around the optimal parameters revealed that both the sensitivity and specificity of LOVE are stable across a range of parameter values (Figures  4C-4E ). As for the previous two datasets, these results confirm that LOVE is able to accurately cluster even if somewhat less than optimal parameter choices are made.
Benchmarking LOVE against a Wide Range of Clustering Methods Our previous results demonstrate that LOVE works well across datasets with different properties both in terms of size and correlation structure. We also showed that LOVE outperforms two state-of-the art methods that use different approaches-fuzzy Cmeans clustering and ClusterOne. To comprehensively compare LOVE's performance in a wide range of existing methods, we used previously established benchmarks for 13 different clustering methods across datasets  (Wiwie et al. , 2015). Also, both the methods to benchmark against and the datasets were chosen by an independent study  (Wiwie et al. , 2015). Based on F1 scores (harmonic mean of precision and recall), LOVE outperforms the 13 existing methods on two very different datasets (Table  3 ).
The first dataset quantified similarities between 293 The Cancer Genome Atlas (TCGA) clinical samples across three different cancer types-breast cancer, lung cancer, and glioblastoma. Specifically, the data were a pairwise similarity matrix of dimension 293 3 293 (n 3 n, where n = 293, the number of variables/samples). For this dataset, LOVE outperformed 11 of the 13 other methods, in terms of the F1 score, and had a rank of 3 across the 14 methods (Table  3 ). The other dataset on which we benchmarked LOVE was synthetic and low dimensional (we have already evaluated LOVE on a high-dimensional dataset earlier), purposely chosen to be very different from the previous dataset. Here, the dataset consisted of 250 variables, and each variable had two corresponding features (i. e. , each variable could be represented by a dot on a two-dimensional dot-plot).