dataesr/scientific-paragraphs-categorization huggingface dataset:
1 water leaving reflectance (ρ w hereafter) data: LAND, CLOUD, HIGH_-GLINT, MEDIUM_GLINT, and LOW_SUN. A final quality control step was applied to remove data points with negative ρ w values at 443 nm. No additional quality control was applied to in situ Chl-a data as the validity and consistency of the data were previously verified by the contributors and managers of the CCRR and MERMAID database  (Nechad et al. , 2015; Barker et al. , 2008). The resulting dataset (Table  1 ) contained 348 Chl-a/MERIS matchups from July 2002 to July 2011, with 235 points from the MERMAID dataset and 113 points from the CCRR dataset with in situ Chl-a ranging from 0. 05 mg per cubic meter (mg m -3 ) to 31 mg m -3. Data points are mostly distributed in North European waters and to a lesser extent along North America coasts (Fig.
Scatterplots show expected patterns (see Section 3. 2 for a detailed discussion) with an overestimation of CHL_OC4 and to a lesser extent CHL_OC5. A high scatter of CHL_RED when in situ Chl-a is low is also observed. This is not surprising as NIR_RED algorithm is known to be applicable only for eutrophic waters  (Gons et al. , 2002). Finally, no impact of the data source (HPLC, fluorometry and spectrophotometry) on the quality of the matchup can be observed suggesting a good coherence of the dataset.
Methodology overview Fig. 3  provides a general overview of the approach used to determine the algorithm-specific application criteria. Starting from a priori knowledge on the OC4, OC5 and NIR-RED algorithms, a set of conditions for the optimal performance of each algorithm was defined and Fig. 2. Chl-a estimated from MERIS reflectance with OC4 (panels A and D), OC5 (panels B and E) and NIR-RED (panels C and F) algorithms, compared to in situ Chla. For a better visualisation scatterplots are presented in log-log (upper panels) and linear-linear scale (lower panels). For the NIR-RED algorithm (right panels), the dotted grey line represents the limit Chl-a = 3 mg m -3 that it is recommended to consider when applying this algorithm. Performance metrics for these algorithms applied to the whole CCRR-MERMAID dataset are given in  Table 4 (Lines 1 to 3 ).
translated to algorithm specific QC tests. The QC tests consisted of different band and band-ratio thresholds. These thresholds are represented by linear lines (hereafter "boundary lines") which are defined with slope and intercept coefficients (hereafter, "threshold coefficients"). The determination of these threshold coefficients was based on the CCRR-MERMAID dataset. This dataset was divided into two equally large, but randomly selected subsets for the determination of the threshold coefficients on one side and their validation on the other side. The dataset division was repeated 501 times to ensure the robustness of the results. This number of repetitions has been chosen to ensure that most of the splitting combinations are represented while keeping a reasonably short processing time and the odd number facilitate the extraction of the median boundary lines.
Determination of threshold coefficients In order to parameterize and validate the QC tests with two independent datasets, the initial CCRR-MERMAID dataset consisting of 348 data points was randomly divided into two datasets of 174 data points, named training and validation dataset respectively. Then, using the training dataset, threshold coefficients for boundary lines (i. e. L R12_OC4 , L logR5_OC4 , L R12_OC5 , L logR5_OC5 , L Chl/NIR_RED and L R6/NIR_RED ) were determined. To obtain optimal and objectives values, a benefit function (BF) was defined as: BF = ∑ i y i with ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ y i = +5 if APD i ≤ 30 y i = +2 if 30 < APD i ≤ 50 y i = -2 if 50 < APD i ≤ 100 y i = -5 if APD i > 100 (6) where i stands for any points passing QC tests for OC4, OC5 or NIR-RED models and APD is the absolute percent difference and is defined as.
APD = 100 ⃒ ⃒ ⃒ ⃒ Chla sat -Chla situ Chla situ ⃒ ⃒ ⃒ ⃒ (7) BF increases when the number of low ADP matchups (APD < 50%) passing the QC increases and decreases when the number of high ADP matchups passing QC (APD > 50%) increases. By attempting to maximize the BF, one identifies the best compromise between including the largest number of low APD matchups in the selection while avoiding high APD matchups. For each training set, BF was computed for a large range of threshold coefficients in order to test all the potential boundary lines and retain the best one. The range of threshold coefficients tested is given in Table  3  (numbers in brackets) and is motivated in Section 3. 1. To achieve a robust choice of the best thresholds the random division of the whole dataset was repeated 501 timeseach of the 501 training sets gives the threshold coefficients which optimize the BF and, applied to the validation subset, it provides different results for the validation metrics.
Hence, data points with a R12 band ratio higher than 1. 25 were flagged as "possible atmospheric correction error". In addition to these previous tests, a limit range for Chl-a values was also introduced  (Lee and Hu, 2006). Indeed, OC4 and OC5 algorithms apply only for oligotrophic and mesotrophic waters  (Odermatt et al. , 2012; Matsushita et al. , 2015; Smith et al. , 2018)  as these algorithms tend to saturate in eutrophic condition. Saturation is marked by constant CHL_OC4 values when in situ Chl-a increases  (See Fig. 5aMatsushita et al. , 2015). Such a saturation pattern is not observed in our dataset probably due to weak representation of eutrophic and hyper-eutrophic waters. However, to take into account all water conditions, a supplementary quality control test was introduced to verify that CHL_OC4 is lower than 10 mg.
QC test for OC5 algorithm The OC5 quality control test was designed using the same approach as for OC4 and the final QC procedure is also displayed in the same flowchart as OC4 (Fig. 5 ). Fig. 6  shows that OC5 performs well for a wider range of R12 and R5 at a given R53 compared to OC4. The valid (green) points are more scattered and the boundary lines, L R12_OC5 and L logR_OC5, are much less restrictive than for the OC4 algorithm, confirming that OC5 performs well for a larger range of CDOM and SPM values compared to OC4. For very low R12 and high log10(R5), OC5 tends to overestimate Chl-a motivating the need to also apply a QC test on OC5 Chl-a products. To maximize the number of pixels processed especially during turbid conditions and because of the overall good performance of the OC5 algorithm on the present dataset, we propose an optional relaxed boundary line for the high SPM OC5 test (orange line in Fig.
6 panel B ) computed by modifying the original BF (Eq. (  6 )) to a value of +5 for points with APD < 30%, +3 for points with APD between 30 and 50%, 0 to points with APD between 50 and 100% and -2 to points with APD higher than 100%. The boundary line for the R12 test has not been modified as the standard version already accepts most of the dataset. The coefficients of the relaxed boundary line as well as the regular boundary lines are given in Table  3  and Section 3. 2 presents the performance of OC5 when applying the relaxed QC.
Performances of the OC4, OC5 and NIR-RED algorithms without quality control The performances of the OC4, OC5 and NIR-RED algorithms were initially tested using the whole CCRR-MERMAID matchup dataset without application of quality controls (Fig. 2, Table 4 ). The results show that without QC the OC5 algorithm performs best with a median absolute percent difference of 39% and a slight overestimation (median ratio = 1. 17). Most of the points were situated close to the 1:1 line and the RMSD was 2. 36 mg m -3. The OC4 algorithm clearly tends to overestimate Chl-a concentration as most of the points were above the 1:1 line and the median ratio is 1. 84, resulting in a MAPD of 84. 5%, which is significantly greater than in case 1 waters (i. e. 35%,  Bailey and Werdell, 2006) , with a rather scattered distribution (RMSD = 5.
2  panels C and F (grey dotted lines) and used for calculation of validation statistics (Table  4 ). Although this limit allows to eliminate all negative and low estimates, it also shows that without a priori information on a study area the elimination only of CHL_NIR less than 3 mg m -3 can still led to very high overestimation errors (Fig. 2C ; Table  4 ). The moderate to bad performances of the OC5, OC4 and NIR-RED algorithms on the whole coastal dataset demonstrate the importance of defining quality control tests for each of these algorithms and for preventing use of their outputs when not appropriate.
Performances of the OC4, OC5 and NIR-RED algorithms after application of quality control Fig. 9  shows the distribution of each performance metric when applied to the validation dataset (50% of the whole CCRR-MERMAID dataset) before (blue boxplots) and after applying the QC (yellow boxplots) obtained for the 501 simulations. For the OC4 and NIR-RED algorithms, the boxplot comparison shows a clear improvement of all metrics after QC. Results for the OC5 algorithm also show a positive evolution as application of QC reduces the positive bias (MR is reduced), as well as the uncertainty (reduced RMSD) and the median percent difference. Only r 2 , the coefficient of determination of the linear Fig. 5. Flow chart showing the different steps driving to the decision of application of the OC4 or OC5 algorithms.
As QC steps are identical for OC4 and OC5 algorithms, only one chart is provided, the expression "OCx" in the chart can be replaced by either "OC4" or "OC5". regression between in situ and satellite measurements, as well as the slope and the intercept of the linear regression are not significantly improved when computed from QC positive data points only. This effect can be explained by the fact that the Chl-a range is slightly reduced (all points with CHL_OC5 > 10 mg m -3 are eliminated). As expected OC4 and NIR-RED quality controls are the most drastic as they retained only 20% and 13% of dataset respectively. This result is easily explained by the low occurrences of case 1 and hyper-eutrophic conditions in coastal waters in general and hence in the CCRR-MERMAID dataset. On the contrary, the OC5 algorithm is better adapted to moderately turbid waters characterizing most of the coastal waters and applies after QC to 64% of the dataset.
In addition, 50 additional points (13% of the dataset) can be added if relaxed OC5 QC is applied. For these extra points, Chl-a derived from OC5 is generally overestimated (MR = 1. 5) and MAPD increases to 57% which might remain acceptable for certain applications. Table  4  indicates performance of the final QC when applied to the full CCRR-MERMAID dataset in comparison to the absence of QC and Fig. 10  (panels A, B and C) show for each algorithm which datapoints have been pass QC tests (red dots). These graphs show that after QC a majority of dots are in the -50%/+50% percent difference which is positive. However, some "good datapoints" don't pass QC, suggesting the limits of the present QC tests. This could be explained by natural variability, uncertainty of in situ measurements or error in the retrieval of water leaving reflectance.
As expected, OC4 is mostly used in the clear oligotrophic waters of the Mediterranean Sea, Californian coast and South Portugal coast and NIR-RED algorithm mostly applied in very coastal regions of the North Sea, French Atlantic coast, North East American coast. The method of Chl-a measurement (i. e. HPLC, fluorometry or spectrophotometry) seems to be independent of the selected algorithm. These results tend to confirm that the developed QC tests are based on optical water types rather than strong regional specificities or inconsistencies between sub-datasets that made the CCRR-MERMAID database. 2 ). In panel A, the dashed blue line shows also the upper limit for R12 > 1. 25 (suspect atmospheric correction). The orange dotted line in panel B represent the boundary line in case of use of the relaxed version of the OC5 algorithm.
Merged algorithm After application of QC, 70% of the dataset (242 data points) can be processed by at least one of the OC4, OC5 and NIR-RED algorithms and this number increases up to 83% (290 data points) if the relaxed version of the OC5 QC is applied. Although each algorithm performs better in specific water types, some data points can pass two different algorithms QC. In fact, all points (69 points in the CCRR-MERMAID dataset) which pass QC for OC4 also pass QC for OC5. Additionally, 14 points passed the QC for both OC5 and NIR-RED and no point passed the QC for all three algorithms (i. e. OC4, OC5 and NIR-RED). To create a merged product, these specific points were naïvely treated by averaging over the two valid algorithms. Although many much more complex methods have been used to merge Chl-a algorithms  (Gons et al.
, 2008; Hieronymi et al. , 2017; Smith et al. , 2018) , we chose to use simple averages because (1) the main scope of this paper is to provide independent QC tests for a series of Chl-a algorithms and (2) our present dataset was insufficient for testing more sophisticated merging methods. This task of algorithm merging is further discussed in Section 4. 2. The performance of the final merged algorithm after QC is presented on Table  4  (lines 7 and 8) and Fig. 10 (panel D ). The standard merged algorithm has a median absolute percent difference of 30. 4% and the points are well aligned along the 1:1 line (see Fig. 10  and Table 3 ; r 2 of the linear regression = 77%). 68% of the data points are within the plus or minus 50% APD range. The majority of points out of this range present an overestimation of the in situ Chl-a measurements.
Application to MERIS data Four almost cloud-free images of the North-Western European waters were selected from the MERIS MEGS8 RR Level 2 archive to illustrate application of the QC tests. The QC test for each Chl-a algorithm and the proposed algorithm merging approach were applied to each image in order to generate final Chl-a maps (Figs. 12  and 13 ). The results show that the OC5 algorithm is used in most regions and for most times of the year with a few exceptions. Due to high turbidity, the image of 2006-03-06 shows that Chl-a concentration can be processed only if the relaxed condition for the OC5 algorithm is applied. In addition, some areas flagged as "high SPM" with the OC5 algorithm cannot be processed by any of the three Chl-a algorithms presented here. These areas generally correspond to very turbid zones visible on the RGB true colour image (see Fig.
Application to OLCI data A first application of the algorithm QC and merging approach applied to OLCI-A data is presented in Fig. 13  for an image acquired on 2017-06-18. L1 reduced resolution data were processed to level 2 with the POLYMER atmospheric correction processor  (Steinmetz et al. , 2011). This atmospheric correction algorithm is designed for coastal waters and has shown good results for OLCI data when compared to AERONET-OC data  (Zibordi et al. , 2009)  located in the North Sea  (Cardoso Dos Santos et al. , 2019). Although a future calibration of OLCI CHL_OC5 is required, OC5 look-up tables published for MERIS were applied to OLCI data, which has very similar spectral bands. The spectral similarity between MERIS and OLCI sensors was tested on a large number of hyperspectral in situ reflectance spectra measured in coastal waters and contrary to observations reported by  Wang et al.
(2020)  when comparing OLCI and VIIRS spectral band, no band shift was Fig. 9. Box-plots comparing the performances of the OC4 (top panels), OC5 (middle panels) and NIR-RED (bottom panels) algorithms on the initial validation dataset (blue box-plots, 50% of the whole CCRR-MERMAID dataset randomly defined in the division-training-validation phase) to the subset of points having passed the QC (yellow box-plots). In the case of the NIR-RED algorithm before QC (blue boxplots), statistics were calculated for datapoints with a NIR-RED Chl-a value higher than 3 mg m -3 following  Gons et al. (2002). By this way, negative values are removed. The red dotted lines represent for each metric its ideal values in case of a perfect algorithm. Note that the y-axis scales differ for some metrics between algorithms.
The Chl-a map shows values ranging between 0. 15 and 8 mg m -3 , with, similarly to the MERIS images, highest Chl-a value along the coast close to estuaries and higher Chl-a values in the southern North Sea (52 • N/3 • E) than in the English Chanel (49 • N/5 • W). The results presented here need to be validated against in situ measurements, which are not available yet, but as a first observation they are consistent with MERIS results and suggest that the present methodology should be easily applicable to OLCI and hence ensure reliable Chl-a data over the next decades.
Discussion on methodology We propose QC tests based on the relative percent difference between in situ and satellite estimates to prevent use of three different Chla algorithms outside their conditions of validity and to prevent the production of erroneous remote sensing Chl-a products in coastal waters. The QC tests have been designed here on the basis of MERIS/in situ matchups for coastal waters. In contrast to the simulated datasets generated from radiative transfer models  (Doerffer and Schiller, 2007; Fig. 10. In situ Chl-a observations as a function of satellite Chl-a estimations. In panels A to C, red dots represent datapoints which pass the final OC4, OC5 and NIR_RED QC respectively, whereas greys points didn't pass QC. Panel D shows In situ Chl-a observations as a function of satellite Chl-a estimations after application of QC tests and merging methodology presented in this paper.
Colour code indicates for each point which Chl-a algorithm has been selected. 292 points (83% of the full MERMAID-CCRR dataset) are represented here. The validation metrics are provided in Table  4  lines 7 and 8 for standard merged algorithm and extending version respectively. In each panel, dotted lines represent the ±50% error interval. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article. ) Nechad et al. , 2015; Hieronymi et al. , 2017)  which are often used for algorithm testing, the CCRR-MERMAID dataset used here is based on MERIS water reflectance. This choice of dataset solves any application transfer issue which may arise from algorithms developed based on synthetic datasets because of possible bias in the representation of optical processes in the radiative transfer models.
Nonetheless, as is generally the case for in situ/satellite matchup datasets, the CCRR-MERMAID dataset presents geographic limitation. Indeed, the CCRR/ MERMAID dataset is dominated by the North Sea and European Atlantic coast with respectively 38% and 24% of data points whereas the other regions are also represented (Californian coast, 22% of data points; Mediterranean Sea, 12. 5% of data points and North America east coast, 2%). However, the over-representation of North European waters in the dataset shouldn't be an issue for global application, as North European waters cover a large range of optical water conditions  (Hieronymi et al. , 2017Fig. 6; Mélin and Vantrepotte, 2015)  with the presence of very turbid, absorbing or eutrophic waters. Regarding the OC5 algorithm, although it has been parameterized from a regional dataset  (Gohin et al.
2. 2 and applied to MERIS and OLCI images (Section 3. 3). Although results showed coherent maps, the QC may produce discontinuities in Chl-a concentration when switching between two algorithms and creates white areas where no algorithm applies. To fill white areas, it is possible to relax the benefit function as it was done for OC5 while increasing uncertainty or to develop specific algorithms for optical waters that do not pass QC Fig. 14. Application of the Chl-a algorithm methodology developed in this paper to OLCI reduced resolution image data from 2017 to 06-18. The left panel shows algorithm selection and the right panel shows the resulting merged Chl-a map. The OLCI image has been processed from level 1 with the POLYMER atmospheric correction  (Steinmetz et al. , 2011). Corresponding RGB image is available in supplementary materials (Fig.
, 2017; Smith et al. , 2018; Neil et al. , 2019). For each pixel, a set of weights defines how the different Chl-a algorithms are combined to determine an optimized Chla product. These weights are generally defined according to optical properties such as Chl-a  (Smith et al. , 2018)  or the full reflectance spectra  (Hieronymi et al. , 2017; Neil et al. , 2019). Such an approach has been tested in our dataset without success as it requires a stronger dataset with more data points and larger SPM, CDOM and Chl-a gradients. In the future, a refinement of the coarse weights (0,0. 5,1) proposed here could be developed. The QC tests presented in this paper were designed to maintain a relatively low uncertainty on the subset of data points where they apply. Their design was based on the relative difference between in situ and satellite estimates, which contrary to other statistical index such as RMSD allows to keep in situ Chl-a measurements as a reference.
, 2008; Gohin et al. , 2019; Gohin et al. , 2020)  is a key driver for monitoring the coastal and offshore waters in Europe. While in situ data acquisition is still considered as the main monitoring tool, the European Commission highlighted the need for greater coherence with related EU legislations (Water Framework Directive and Habitats and Birds Directive) and for more coherent and coordinated approaches within and between marine regions and sub-regions (European Commission, 2014). During recent years there has been a growing tendency to use optical remote sensing as a supporting tool to achieve the monitoring requirements because Chl-a satellite data combine cost-effective data collection with a much improved spatial and temporal coverage compared to in situ measurements. The satellite data also facilitates transnational spatial coherence which for MFSD monitoring avoids discrepancies between EU countries due to different in situ methodologies and protocols.
The reliability of Chl-a estimates and the accurate assessment of uncertainty, which is important to users, is addressed by the methodology presented in this study. To test the relevance of this methodology in the framework of the MFSD eutrophication assessment, QC tests for OC4, OC5 and NIR-RED algorithms were applied in a North Sea wide eutrophication assessment and the merged Chl-a product was evaluated by a comparison analysis with in situ datasets for all assessment areas in the greater North Sea. The comparison of the yearly mean and P90 CHL products yielded a median difference of  35. 19% and 39. 05% (Van der Zande et al. , 2019)  validating the present methodology for eutrophication assessment in European waters. For future applications, it has been shown that the direct application of the present methodology on OLCI data is straight forward because of the good match between OLCI and MERIS spectral bands.
In addition, although a validation with in situ OLCI matchup is needed, the application of the present methodology on OLCI data (Fig. 14 ) shows coherent results. Regarding other optical sensors, there is an interest for coastal water monitoring to define QC tests for high resolution sensors such as Sentinel 2 -MSI or Landsat -8, OLI. In that case, some adaptations are needed to account for spectral resolution of these sensors: (1) to use the CCRR-MERMAID database, band-shifts coefficients are needed, (2) some variables used in QC tests will need to be modified if the MERIS band doesn't exist in the high resolution sensor and (3) some Chl-a algorithms will have to be abandoned (i. e. OC5, NIR-RED with Landsat8-OLI) or adapted (OC4 to OC3) when spectral bands are missing.