dataesr/scientific-paragraphs-categorization huggingface dataset:
To be consistent with satellite-derived chlorophyll values, which are derived from the light emerging from the upper layer of the ocean, all chlorophyll observations in the top 10 m (replicates at the same depth, or measurements at multiple depths) were averaged if the coefficient of variation among observations was less than 50 %, otherwise they were discarded. The averages were then assigned to the surface. The depth of 10 m was chosen as a compromise between clear oligotrophic and turbid eutrophic waters. Other methods, such as chlorophyll depth averages using local attenuation conditions  (Morel and Maritorena, 2001) , require observations at multiple depths, which, given our decision to use only in vitro measurements, would have reduced considerably the final number of observations. With regard to the inherent optical properties (aph, adg, bbp), if not already calculated and provided in the contributed datasets, they were computed from related variables that were available: particle absorption (ap), detrital absorption (ad), coloured dissolved organic matter (CDOM) absorption (ag) and total backscattering (bb).
The following equations were used: adg = ad + ag, ap = aph + ad, and bb = bbp + bbw. For the latter equation, the variable bbw was computed using bbw = bw/2, where bw is the scattering coefficient of seawater derived from  Zhang et al. (2009). The diffuse attenuation coefficient for downward irradiance (kd) did not require any conversion and was compiled as originally acquired. Observations of inherent optical properties (surface values) and the diffuse attenuation coefficient for downward irradiance were acquired in total from six data sources designed for ocean-colour validation and applications (SeaBASS, NOMAD, MERMAID, AWI, COAST-COLOUR, TPSS), thus already subject to the processing routines of these datasets. Concerning total suspended matter, these data were compiled as originally available from MER-MAID and COASTCOLOUR.
The merged dataset was compiled from 27 sets of in situ data, which were obtained individually either from archives that incorporate data from multiple contributors (SeaBASS, NOMAD, MERMAID, ICES, ARCSSPP, BIOCHEM, BODC, COASTCOLOUR, MAREDAT, SEA-DATANET) or from particular contributors, measurement programmes or projects (MOBY, BOUSSOLE, AERONET-OC, HOT, GeP&CO, AMT, AWI, BARENTSSEA, BATS, CALCOFI, CCELTER, CIMT, ESTOC, IMOS, PALMER, TPSS, TARA) and were subsequently homogenized and merged. Data contributors are listed in Table  2. There were methodological differences between datasets. Therefore, after acquisition, and prior to any merging, each set of data was preprocessed for quality control and converted to a common format. During this process, data were discarded if they had (1) unrealistic or missing date and geographic coordinate fields, (2) poor quality (e.
g. original flags) or method of observation that did not meet the criteria for the dataset (e. g. in situ fluorescence for chlorophyll concentration), and (3) spuriously high or low data. For the last item, the following limits were imposed: [0. 001-100] mg m -3 for chla_fluor and chla_hplc; [0-0. 15] sr -1 for rrs; [0. 0001-10] m -1 for aph, adg and bbp; [0-1000] g m -3 for tsm; and [aw(λ) -10] m -1 for kd, where aw is the pure water absorption coefficients derived from  Pope and Fry (1997). Also during this stage, three metadata strings were attributed to each observation: dataset, subdataset and contributor. The dataset contains the name of the original set of data and can only be one of the  following: aoc, boussole, mermaid, moby, nomad, seabass, hot, ices, amt, gepco, arcsspp, awi, barentssea, bats, biochem, bodc, calcofi, cc, ccelter, cimt, estoc, imos, maredat, palmer , seadatanet, tpss or tara.
The subdataset starts with the dataset identifier and is followed by additional information about the data, as <dataset>_<cruise/station/site> (e. g. seabass_car81). The contributor contains the name of the data contributor. An effort was made to homogenize the names of data contributors from the different sets of data. These three metadata are the link to trace each observation to its origin and were propagated throughout the processing. Finally, this processing stage ended with each set of data being scanned for replicate variable data and replicate station data, which when found were averaged if the coefficient of variation was less than 50 %, otherwise they were discarded. Replicates were defined as multiple observations of the same variable, with the same date, time, latitude, longitude and depth.
Replicate station data were defined as multiple measurements of the same variable, with the same date, time, latitude and longitude. For the latter case, a search window of 5 min in time and 200 m in distance was given to account for station drift. A small number of observations that were identified as replicates had different subdataset identifiers (i. e. different cruise names). These observations were considered suspicious if the values were different and discarded. If the values were the same, one of the observations was retained. This possibly originated from the same group of data being contributed to an archive by two different data contributors. Once a set of data was homogenized, its data were integrated into a unique table. This final merging focused on the removal of duplicates between the sets of data.
Although some duplicates are known (e. g. MOBY, BOUS-SOLE, AERONET-OC and NOMAD data are found in SeaBASS and MERMAID), others are unknown (e. g. how many data of GeP&CO, ICES, AMT and HOT are within NOMAD, SeaBASS and MERMAID). Therefore, duplicates were identified using the metadata (dataset and subdataset) when possible and temporal-spatial matches, as an additional precaution. For temporal-spatial matches, several thresholds were used, but typically 5 min and 200 m were taken to be sufficient to identify most duplicated data, which reflected small differences in time, latitude and longitude, between the different sets of data. Larger thresholds were used in some cases as a cautionary procedure. This was the case when searching for NOMAD data in other datasets, because NOMAD includes a few cases where merging of radiometric and pigment data was done with large spatialtemporal thresholds  (Werdell and Bailey, 2005).
A large temporal threshold was also used when integrating observations from the three data sources that did not have time available  (ESTOC, MAREDAT and TPSS). In regard to all data, if duplicates were found, data from the NOMAD dataset were selected first, followed by data from individual projects or contributors (MOBY, BOUSSOLE, AERONET-OC, AMT, HOT,GeP&CO, AWI, BARENTSSEA, BATS, CALCOFI, CCELTER, CIMT, ESTOC, IMOS, PALMER, TPSS and TARA), and finally for the remaining datasets (SeaBASS, MERMAID,ICES, ARCSSPP, BIOCHEM, BODC, COAST-COLOUR, MAREDAT and SEADATANET). This procedure was chosen to preserve the NOMAD dataset as a whole, since it is widely used in ocean-colour validation. It should be noted that, by this procedure, data from individual projects or contributors may be listed under NOMAD (e.
Data processing thus included two major steps: preprocessing and merging. The first step was related to each set of contributing datasets in particular and aimed to identify problems and convert the data of interest to a standard format. The second step dealt with the integration of data into one unique file and included the elimination of duplicated    MOBY is a fixed mooring system operated by the National Oceanic and Atmospheric Administration (NOAA) that provides a continuous time series of water-leaving radiance and surface irradiance in the visible region of the spectra since 1997. The site is located a few kilometres west of the Hawaiian island of Lanai where the water depth is about 1200 m. Since its deployment, MOBY measurements have been the primary basis for the on-orbit vicarious calibrations of the SeaWiFS and MODIS ocean-colour sensors.
SeaWiFS Bio-optical Archive and Storage System (SeaBASS) SeaBASS is one of the largest archives of in situ marine bio-optical data  (Werdell et al. , 2003). It is maintained by NASA's Ocean Biology Processing Group (OBPG) and includes measurements of optical properties, phytoplankton pigment concentrations, and other related oceanographic and atmospheric data. The SeaBASS database consists of in situ data from multiple contributors, collected using a variety of measurement instruments with consistent, community-vetted protocols from several marine platforms such as fixed buoys, handheld radiometers and profiling instruments. Quality control of the received data includes a rigorous series of pro-tocols that range from file format verification to inspection of the geophysical data values  (Werdell et al.
In the pigment data used in this work, a large number of in situ fluorometric measurements from continuous underway instruments were identified and discarded. These data were initially identified from cruises with more than 50 observations per day and then rechecked in the SeaBASS website to confirm whether indeed they were continuous underway measurements. A total of 120 412 such measurements were identified and discarded. Given the large volume of this group of data, it is possible that some chlorophyll a observations from in situ methods may have escaped the scrutiny and persisted into the final merged dataset. The Pigment search tool was recently discontinued, and, instead, the File search tool can be used, which was also used here to acquire chlorophyll observations for more recent years.
NASA bio-Optical Marine Algorithm Dataset (NOMAD) NOMAD is a publicly available dataset compiled by the NASA OBPG at the Goddard Space Flight Center. It is a high-quality global dataset of coincident radiometric and phytoplankton pigment observations for use in ocean-colour algorithm development and satellite-data product-validation activities  (Werdell and Bailey, 2005). The source of the biooptical data is the SeaBASS archive; therefore, many dependencies exist between these two datasets, which were addressed during the merging. The current version (version 2. 0 ALPHA, 2008) includes data from 1991 to 2007 and an additional set of observations of inherent optical properties. The current version was used in this work, but with an additional set of columns of remote-sensing reflectance corrected for the bidirectional effects  (Morel and Gentili, 1996; Morel et al.
, 2002). This additional set of columns was provided directly by the NOMAD creators. The compiled variables were rrs, chla_hplc, chla_fluor, aph, adg, bbp and kd. Conversion was necessary only for aph, adg and bbp and fol-lowed the procedures described in Sect. 2. 1. For the calculation of bbp the variable bb was used with a smooth fitting to remove noise. A portion of the NOMAD data were optically weighted (for methods see  Werdell and Bailey, 2005). These data are not consistent with the protocols chosen in this work, but these observations were retained since NOMAD is a widely used dataset in ocean-colour validation.
MERIS Match-up In situ Database (MERMAID) MERMAID provides in situ bio-optical data matched with concurrent and comparable MERIS Level 2 satellite oceancolour products  (Barker, 2013a, b). The MERMAID in situ database consists of data from multiple contributors, measured using a variety of instruments and protocols from several marine platforms such as fixed buoys, handheld radiometers and profiling instruments. Comprehensive quality control and protocols are used by MERMAID to integrate all the data into a common and comparable format  (Barker, 2013a, b). Access to MERMAID data is limited to the MERIS Validation Team, the MERIS Quality Working Group and to the in situ data contributors. For this work, access has been granted to the MERMAID database through a signed service level agreement.
The MERMAID data includes subsets of several datasets used in this compilation (MOBY, AERONET-OC, BOUSSOLE, NOMAD). These observations were removed from the MERMAID dataset to avoid duplication (as discussed in Sect. 2. 1). The compiled variables were rrs, chla_hplc, chla_fluor, aph, adg, bbp, kd and tsm. Remote-sensing reflectance was calculated by dividing the original fully normalized water-leaving reflectance (Rw_ex), which is the water-leaving reflectance (Rw = π Lw/Es), with a correction for the bidirectional nature of the light field  (Morel and Gentili, 1996; Morel et al. , 2002) , by π. Conversion was also necessary for aph, adg and bbp and followed the procedures described in Sect. 2. 1.
Hawaii Ocean Time-series (HOT) HOT programme provides repeated comprehensive observations of the hydrography, chemistry and biology of the water column at a station located 100 km north of Oahu, Hawaii, since October 1988  (Karl and Michaels, 1996). This site is representative of the North Pacific subtropical gyre. Cruises are made approximately once a month to the deep-water station ALOHA (A Long-Term Oligotrophic Habitat Assessment; 22 • 45 N, 158 • 00 W). Pigment data (chla_hplc and chla_fluor) were extracted directly from the project website. Radiometric measurements from the HOT project are also available, but observations of rrs and kd from the HOT project were acquired in this work as part of the SeaBASS dataset.
International Council for the Exploration of the Sea (ICES) ICES is a network of more than 4000 scientists from almost 300 institutes, with 1600 scientists participating in activities annually. The ICES Data Centre manages a number of large dataset collections related to the marine environment covering the northeastern Atlantic, Baltic Sea, Greenland Sea and Norwegian Sea. The majority of data originate from national institutes that are part of the ICES network of member countries. Data were provided (on 28 April 2014) from the ICES database on the marine environment (Copenhagen, Denmark) following a specific request. The ICES data were made available under the ICES data policy, and if there is any conflict between this and the policy adopted by the users, then the ICES policy applies. The compiled variables were chla_hplc and chla_fluor.
Arctic System Science Primary Production (ARCSSPP) The ARCSSPP database is a synthesis of observations between 1954 and 2006 from the Arctic Ocean and northern seas  (Matrai et al. , 2013)  In this work, the AWI data source refers to the group of observations that were provided to the OC-CCI project by Astrid Bracher. These are bio-optical observations collected during several cruises in the Atlantic and Pacific Ocean. were corrected for the bidirectional nature of the light field  (Morel and Gentili, 1996; Morel et al. , 2002)   Taylor et al. , 2011b) , and cruises ANT-XXVI/3 and ANT-XXVIII/3 (https://doi. org/10. 1594/PANGAEA. 819617; Soppa et al. , 2013). The compiled variables were chla_hplc, rrs and aph.
CoastColour Round Robin (COASTCOLOUR) COASTCOLOUR datasets were designed to evaluate the performance of ocean-colour satellite algorithms in the retrieval of water quality parameters in coastal waters  (Nechad et al. , 2015a). Three types of COASTCOLOUR datasets are available: (1) a matchup dataset where in situ biooptical observations are available simultaneously with a cloud-free MERIS product, (2) an in situ reflectance dataset where an in situ reflectance is available simultaneously with an in situ measurement of chlorophyll a concentration and/or total suspended matter, and (3) a simulated dataset where reflectances were generated by a radiative transfer model. This work used the matchup dataset, which includes most of the in situ measurements and is available at https://doi. org/10. 1594/PANGAEA.
841950 (Nechad et al. , 2015b). The matchup dataset provides optical, biogeochemical and physical data collections at 17 sites across the globe. From this dataset, observations of reflectance, chlorophyll a, total suspended matter and IOPs were compiled. The remote-sensing reflectances were corrected for the bidirectional nature of the light field  (Morel and Gentili, 1996; Morel et al. , 2002). The compiled variables were rrs, chla_hplc, chla_fluor, aph, adg, bbp and tsm.
MARine Ecosystem DATa (MAREDAT) The MAREDAT database is a global assemblage of pigments measured by HPLC  (Peloquin et al. , 2013a)  from the combination of 136 independent field datasets, solicited from investigators and databases. The database provides high-quality measurements of taxonomic pigments including chlorophyll a and b, 19'-butanoyloxyfucoxanthin, 19'hexanoyloxyfucoxanthin, alloxanthin, divinyl chlorophyll a, fucoxanthin, lutein, peridinin, prasinoxanthin, violaxanthin and zeaxanthin. The database is available through PAN-GAEA (https://doi. org/10. 1594/PANGAEA. 793246; Peloquin et al. , 2013b). For this work only measurements of total chlorophyll a flagged with high quality were used. The time of day was unavailable and was set to 12:00:00 UTC. These observations were flagged with "1" in the column flag_time.
The empty (white) squares indicate no data for that month. 4d583713667a0f52b9d2937a26d0d82e; Schofield et al. , 2018c; and https://doi. org/10. 6073/pasta/c479b922 d42ace1ce37f9a977e214952;  Schofield et al. , 2017). The compiled variables were chla_hplc and chla_fluor.
SeaDataNet archive (SEADATANET) SeaDataNet is a Pan-European infrastructure for ocean and marine data management. It aims to develop a standardized To maximize the number of ratios per dataset a search window up to 12 nm was used, when the four wavelengths  (412, 443, 490, 555)  were not simultaneously available. The effect of different search windows was negligible in the ratio distribution. system for managing large and diverse datasets collected by oceanographic cruises and automatic observation systems. For this work, discrete chlorophyll a concentration observations with an access restriction set to academic and unrestricted were acquired from the SeaDataNet platform with guidance from the help desk. Only data from the Institute of Marine Research -Norwegian Marine Data Centre (NMD), Norway, which comprised most of the acquired data, were used.
For each observation in a given station, there are three metadata strings: dataset, subdataset and contributor. The columns of the table take the form described in Table  1. The data contributors are indicated in Table  2. Regarding spectral variables, all original wavelengths were preserved, which requires a large number of unique wavelengths to be maintained in the database. No band shifting was performed (though some archived data in some data sources may have been merged with nearby wavelengths) and no minimum number of wavelengths per observation was imposed. This allows further manipulation of the table for different purposes. In the following paragraphs, the table is analysed and the final group of observations is described for each contributing dataset; however, the numbers reported here do not reflect the original numbers in each dataset, since duplicates across contributing datasets were removed (e.
g. NOMAD and others were removed from MERMAID). Observations of remote-sensing reflectance are available at 611 unique wavelengths (i. e. columns), between 404. 7 and 1022. 1 nm (Fig. 1 ). In total there are 59 781 observations (i. e. rows) with remote-sensing reflectance in the table. The total number of observations are partitioned per contributing datasets as follows: AERONET-OC (31 574), BOUS-  SOLE (17 364), MOBY (5466), NOMAD (3326), MER-MAID (885), SeaBASS (698), AWI (54), COASTCOLOUR (307) and TARA (107). Data from AERONET-OC, BOUS-SOLE and MOBY correspond to continuous time series, and, hence, the higher number of observations. Data distribution at 44X and 55X nm is provided in Fig. 2a  and b , respectively. Data were first searched for at 445 and 555 nm and then with a search window up to 8 nm to in- clude also data at 547 nm.
Median values at 44X nm range from 0. 003 m -1 (AERONET-OC) and 0. 009 m -1 (MOBY), whereas at 55X nm the median values lie between 0. 001 m -1 (AWI) and 0. 007 m -1 (COASTCOLOUR). The observations are unevenly distributed between each month of the year in both hemispheres, with a higher coverage in summer months (Fig. 3 ). There are fewer data in the Southern Hemisphere than in the Northern Hemisphere (Fig. 3 ). For additional analysis, rrs band ratios were plotted against each other (490 : 555 versus 412 : 443, Fig. 4 ). Most points are within the boundaries of the NOMAD dataset, but some scattered points were found. These points were retained in the table to allow further manipulation with different quality control criteria. Complementary analysis of remote-sensing reflectance data is made when other variables are concurrently avail-    For chlorophyll a concentration, two types of observations were compiled, one measured by fluorometric or spectrophotometric methods (chla_fluor) and the other measured by HPLC methods (chla_hplc).
The total number of chla_hplc observations is 23 550, ranging from 0. 002 to 99. 8 mg m -3 (Fig. 7 ), with contributions from NOMAD (1309), SeaBASS (9478), MERMAID (707), ICES (2994),  HOT (193), GeP&CO (1536) , BOUSSOLE (397), AMT (902), AWI (750), BATS (334), BODC (735), COASTCOLOUR (848), IMOS (103), MAREDAT (1024), PALMER (1077), TPSS (1002) and TARA (161). The combined chlorophyll dataset (all chlorophyll data considered, but for a given station, HPLC data were selected if available) has a total of 79 731 observations, with 10 %, 49 % and 41 % respectively from oligotrophic (< 0. 1 mg m -3 ), mesotrophic (0. 1-1 mg m -3 ) and eutrophic (> 1 mg m -3 ) waters. When compared with the proportions of the world ocean in these trophic classes, 56 % oligotrophic, 42 % mesotrophic and 2 % eutrophic  (Antoine et al.
, 1996) , oligotrophic waters are underrepresented relative to eutrophic waters in the compilation. The combined chlorophyll dataset is unevenly distributed between each month of the year in both the Northern and Southern Hemisphere, with higher coverage in summer months (Fig. 3 ). There are fewer data in the Southern Hemisphere than in the Northern Hemisphere (Fig. 3 ). The spatial distribution of the chlorophyll values for the combined dataset (Fig. 8 ) shows a good agreement with known biogeographical features, such as lower chlorophyll values in the subtropical gyres and higher values in temperate, coastal and upwelling regions. Many regions show a good spatial coverage (e. g. Atlantic and Pacific Ocean), while others are less well sampled (e. g. Southern and Indian Ocean). Of the contributing datasets, NOMAD and SeaBASS provide a good spatial coverage in many regions (Fig.
9 ). Other datasets also provide coverage from several locations across the globe (GEPCO, MAREDAT, TARA). The ICES, MERMAID and BODC data are mainly located along the coastal regions of Europe. The AMT and many AWI data mainly cover the central part of the Atlantic Ocean, other AWI data cover the Atlantic sector and the Amundsen to Bellingshausen Sea of the Southern Ocean and the western subtropical and tropical Pacific. The SEADATANET, ARCSSPP and BARENTSSEA provide coverage for the Arctic region and northern seas of the North Atlantic. The observations from BIOCHEM and TPSS are mostly from the northwestern Atlantic, while CALCOFI, CCELTER and CIMT provide data for the western coast of North America. The remaining datasets provide observations for fixed locations: PALMER (western Antarctic Peninsula), COASTCOLOUR (17 coastal sites across the world), BATS (Bermuda, North Atlantic), BOUSSOLE (Mediterranean), HOT (Hawaii, North Pacific), IMOS (coastal sites around Australia) and ESTOC (Canaries, North Atlantic).