dataesr/scientific-paragraphs-categorization huggingface dataset:
Importantly, the measured values of a ph (λ) were used in the calculations of these IOPs. These calculations are also described in detail below. Thus, each combination of the five constituent IOPs in the synthetic IOP dataset consists of the measured a ph (λ) and the calculated a d (λ), a g (λ), b b-ph (λ), and b b-d (λ), where the results of these calculations depend on the measured a ph (λ). As a result of this approach, it would seem justifiable to refer to the created IOP dataset as a quasisynthetic dataset. For simplicity, however, we refer to it as a synthetic IOP dataset while bearing in mind that a ph (λ) spectra were derived from measurements.
Description of the in situ dataset Figure  1a  depicts the location of oceanographic stations where the near-surface measurements of a ph (λ) were made. As shown, these measurements were collected across diverse open-ocean and coastal environments. Their total number is 4382, constituting the initial field dataset of a ph (λ) considered in this study. Figure  1  also shows the location of stations where coincident measurements are available for the pairs of IOP coefficients, namely a ph (λ) and a g (λ) (Fig. 1b ), a ph (λ) and a dg (λ) (Fig. 1c ), and a ph (λ) and b bp (λ) (Fig. 1d ). We recall that while the in situ data of a g (λ), a dg (λ), and b bp (λ) were not used in the development of the synthetic IOP dataset, they were assembled for the purpose of comparison with corresponding coefficients that were calculated and included in the synthetic IOP dataset.
Many in situ data of the IOP coefficients used in the present study were collected in previous studies (e. g. ,  Reynolds et al. , 2001; Babin et al. , 2003; Loisel et al. , 2007; Claustre et al. , 2008; Huot et al. , 2008; Stramski et al. , 2008; Lubac et al. , 2008; Loisel et al. , 2009; Bricaud et al. , 2010; Loisel et al. , 2011; Antoine et al. , 2011; Neukermans et al. , 2012; Uitz et al. , 2015; Neukermans et al. , 2016; Reynolds et al. , 2016; Aurin et al. , 2018; Reynolds and Stramski, 2019; Stramski et al. , 2019). Some data are described in publications devoted to compilation of various datasets  (Valente et al. , 2019; Casey et al. , 2020)  and are included in several databases (e. g. , SeaBASS, CoastlOOC, BOUSSOLE, and GOCAD). As the IOP coefficients in the in situ dataset were measured over a broad range of trophic and environmental conditions, their spectral values span more than 3 or 4 orders of magnitude.
Generation of the dataset of hyperspectral a ph (λ) The first task necessary in developing the synthetic IOP dataset was to assemble data on the hyperspectral absorption coefficient of phytoplankton, a ph (λ), from field measurements collected across diverse open-ocean and coastal environments  (Figs. 1a, 2a). These a ph (λ) data were obtained with the filter-pad spectrophotometric method as a difference between the measurements of a p (λ) and a d (λ)  (Kishino et al. , 1985; IOCCG Protocol Series, 2018). Historically, most of these measurements were acquired with the transmittance configuration of the filter-pad method; such measurements are included in our dataset. However, some data in our dataset were obtained with the inside integrating-sphere configuration of the filter-pad method, which is superior to the transmittance configuration of measurement  (Stramski et al.
, 2015; IOCCG Protocol Series, 2018). A significant portion (23. 7 %) of the initial dataset of a ph (λ), consisting of 4382 measurements, covers a spectral range of 400 to 750 nm with a high spectral resolution of data reported at 1 nm intervals. In some cases, the original measurements extended to the near-UV spectral region and/or longer wavelengths in the near-infrared spectral region (800 or 850 nm). The data beyond 750 nm are not used in this study because our RT simulations target the spectral range of 350 to 750 nm. It is notable that the absorption measurements of marine particles and phytoplankton are generally unavailable or are not reported in the UV because of increased methodological challenges and uncertainties in this spectral region  (Stramski et al. , 2015; IOCCG Protocol Series, 2018; Kostakis et al.
, 2021). As a result, only a relatively small fraction of a ph (λ) measurements in the initial dataset were reported in the near-UV region. In addition, the initial dataset included a relatively large fraction of a ph (λ) measurements that were reported at wavelength intervals larger than 1 nm. These lower-resolution data (hereafter referred to as multispectral) ranged from a small wavelength interval of 2 nm to data reported at more limited numbers of wavelengths (as few as <10) within the visible spectral range. It is likely that the multispectral data available from some data sources that we used in this study were originally measured at higher spectral resolution but were eventually reported only for some selected wavelengths, such as those corresponding to the spectral bands available on satellite ocean color sensors.
The first objective of the analysis of a ph (λ) was to consider the initial a ph (λ) dataset within the 400-750 nm range and convert the measurements that were reported at lower spectral resolution to uniformly hyperspectral data at 1 nm intervals. In this analysis, all measurements originally available at 1 nm intervals were considered to provide reference spectral shape functions of a ph (λ). The originally multispectral data of a ph (λ) were converted to hyperspectral data using several different approaches depending on the spectral features of lower-resolution data. One approach utilized the reference spectral shape functions of a ph (λ) and was applied to multispectral a ph (λ) data if they were reported for fewer than 100 wavelengths. In this case, a given multispectral spectrum of a ph (λ) was converted to a hyperspectral spectrum using a specific hyperspectral measurement that exhibited the highest correlation with the multispectral measurement under consideration.
Generation of the complete IOP dataset In the next step of the analysis, the subset of 2204 a ph (λ) spectra that was created from the initial a ph (λ) dataset as described above was subject to additional modifications to ensure that the final a ph (λ) dataset was characterized by the probability distribution that resembles the distribution representative of the global ocean. This process and background information on the motivation for such adjustments in the probability distribution are described below. When the end goal is to achieve a high degree of representativeness of the global ocean as in this study, the process of assembling in situ datasets of IOPs is unavoidably subject to limitations, even if relatively large amounts of data from many field experiments and cruises are considered.
This is mainly because the global ocean is dominated by vast areas of open-ocean pelagic environments and the amount of IOP data collected in these environments is disproportionally limited compared to the amount of data collected in coastal re- gions, which represent a relatively small portion of the global ocean. Thus, the probability distributions based on in situ datasets, such as those presented in Fig. 2 , are expected to deviate from the probability distributions representative of the global ocean. In particular, the maxima of probability distributions and the measures of central tendency, such as the median and mean values, obtained from compilations of a relatively large amount of in situ IOP data (such as in Fig. 2 ) are expected to be shifted to larger values compared to actual global distributions because the IOPs exhibit a general tendency of higher values in coastal regions compared to openocean environments.
While this issue has been recognized, it has not been addressed or resolved in various studies that focus on global ocean color applications. For example, current global ocean color algorithms for estimating chlorophyll a concentration (Chla) are based on relatively large amounts of in situ data whose probability distribution is shifted significantly to higher Chla compared with the global Chla distribution (O'Reilly and  Werdell, 2019). Similarly, in the development of previous synthetic optical databases with RT simulations (e. g. ,  IOCCG Report, 2006) , no special attempt was made to ensure consistency between the probability distributions of input IOP data and the distributions expected for the global ocean. In the recent development of refined global ocean color algorithms for estimating the concentration of particulate organic carbon (POC), the in situ dataset was assembled with a goal to achieve reasonable consistency with a global POC distribution  (Stramski et al.
, 2022). This goal was, however, achieved at the expense of a significant reduction in the amount of accepted in situ data compared to the size of the overall pool of available in situ data. In this study, our goal was to create a relatively large synthetic IOP dataset based on the initial dataset of several thousand measurements of spectral a ph (λ) so that the probability distributions of IOPs in the final synthetic dataset are reasonably consistent with the expected distributions representative of the global ocean. As described above, the initial field dataset in support of this process consisted of 4382 spectra of a ph (λ); this number was further reduced to 2204 spectra that were accepted as a result of analysis and some criteria applied to the initial dataset. This reduced dataset of accepted a ph (λ) spectra was then further modified to ensure that the final probability distribution of a ph (440) resembles the global distribution of a ph (440).
, 2007; Claustre et al. , 2008; Stramski et al. , 2008)  and in some areas of the Mediterranean Sea in summer  (Loisel et al. , 2011). This 17th OWC is described in  Jorge et al. (2021). The 3SAA does not yield the separate contributions of CDOM, a g (λ), and non-algal particles, a d (λ), to the overall non-phytoplankton absorption coefficient, a dg (λ). Therefore, we also used another semi-analytical model (CDOM-KD2) described in  Bonelli et al. (2021)  to estimate a g (443) from OLCI-derived R rs (λ). Having a dg (λ) from the 3SAA and a g (λ) from CDOM-KD2, non-algal particulate absorption, a d (λ), was obtained as the difference a dg (λ) -a g (λ). As a result of this analysis, we obtained a dataset of satellite-derived constituent absorption coefficients, a ph (443), a g (443), a d (443), and a dg (443), as well as the particulate backscattering coefficient, b bp (550), where we focused on the spectral band near 440 nm for absorption and 550 nm for backscattering.
For illustrative purposes, Fig. 4a  depicts the spatial distribution of 17 optical water classes (OWCs) over the global ocean obtained from satellite OLCI data following the methodology of  Mélin and Vantrepotte (2015). For further illustrative purposes, these 17 OWCs were grouped into three optical water groups (OWGs). Group 1 consists of OWC1 and OWC2, which are characterized by high water turbidity such as in coastal areas affected by discharge from large rivers. Although the focus of this study is to create synthetic datasets representative primarily of open ocean and moderately turbid coastal waters, an explicit identification of Group 1 data that represent very turbid waters is of interest for comparisons with the database developed specifically for coastal waters by  Nechad et al. (2015).
The second OWG, Group 2, includes six OWCs: from OWC3 through OWC8. This group represents mainly productive waters in both coastal and open-ocean environments, such as those encountered in the North Atlantic during phytoplankton bloom periods  (Lévy et al. , 2005). Finally, Group 3 included the remaining nine OWCs: from OWC9 through OWC17. These water types are observed mainly in mesotrophic and oligotrophic regions of the global ocean. Based on this classification, 79. 6 % of OLCI water pixels in Fig. 4a  belong to Group 3, 10. 8 % to Group 2, and 9. 6 % to Group 1. The histograms of OLCI-derived R rs (443) associated with these three groups of data are shown in Fig. 4b. For comparative purposes we also assembled a dataset of in situ measurements of R rs (λ), which were collected at various locations within the global ocean (Fig.
4c ). The histograms of in situ R rs (443) associated with Groups 1, 2, and 3 are depicted in Fig. 4d , which show a similar pattern to that in Fig. 4b. For the in situ dataset of R rs (λ), 69. 2 % of data belong to Group 3, 15. 7 % to Group 2, and 15. 1 % to Group 1. The probability density functions (PDFs) of global satellite-derived a ph (440), a g (440), a dg (440), and b bp (550) are depicted in Fig. 5. We refer here to satellite-derived absorption coefficients at 440 nm although they were derived from OLCI reflectances at 443 nm, which is a minor difference that is inconsequential for the purpose of this study. Comparison of Figs. 2a  and 5a  indicates that the distribution of measured a ph (440) from our initial field dataset (Fig. 2a ) is shifted towards higher values compared to the global distribution of satellite-derived a ph (440) (Fig.
5a ). The probability distribution of reduced dataset of measured a ph (440) (N = 2204) that was created from the initial field dataset of a ph (λ) shows similar deviations from the global distribution (not shown). Thus, to create the final dataset of a ph (λ) that has the probability distribution of a ph (440) consistent with the global satellite-derived distribution, we adjusted the number of a ph (440) measurements in each bin of the histogram of the reduced dataset either by removing the measurements from any given bin or by adding the measurements to this bin. The removal or addition of a ph (440) measurements associated with any given bin was done by subjecting all a ph (440) measurements originally contained within a given bin to random selection. Specifically, in the case of addition the randomly selected a ph (440) was added as a replicate of a ph (440) to a given bin.
In the case of removal, the randomly selected a ph (440) was simply removed from a given bin. As a result of this process, we obtained a modified distribution of measured a ph (440) that is fairly consistent with the satellite-derived distribution of a ph (440). Both the modified histogram and the corresponding modified PDF of measured a ph (440) are depicted in Fig. 5a  for comparison with the global satellitederived distribution. In total, this modified distribution consists of 3320 measurements of a ph (440); obviously, each of these measurements at 440 nm has an associated full spectrum of a ph (λ) values between 300 and 750 nm. These 3320 spectra of a ph (λ) represent one IOP component of the final synthetic IOP dataset. The full synthetic IOP dataset created in this study consists of 3320 combinations of measured a ph (λ) and synthetically generated a d (λ), a g (λ), b b-ph (λ), and b b-d (λ).
Below is a de-  We note that all IOP coefficients are expressed in units of m -1 and the light wavelength is given in nanometers. The four IOP coefficients were calculated using a similar methodology to that applied in previous studies aiming at generation of synthetic ocean optical databases (IOCCG  Re-port, 2006; Craig et al. , 2020). Specifically, we used the measured values of a ph (440) as the main driver of calculations of a g (λ), a d (λ), b b-ph (λ), and b b-d (λ). Thus, the variability in the measured a ph (440), as depicted by the probability distribution of measured a ph (440) in Fig. 5a , is the main source of variability in these four co-existing IOP coefficients. a g (440) = 10 (P 1 + γ ) , (1) where P 1 is a parameter related to a ph (440) and γ is randomly selected from a predetermined range of values (Table  1 ).
The spectral values of a g (λ) are subsequently determined from a g (λ) = a g (440) e -S g (λ-440) , (2) where the spectral slope parameter, S g in units of nm -1 , is randomly selected from a predetermined range of values (Table  1 ). The absorption coefficient of non-algal particles was modeled in a similar fashion: a d (440) = P 2 a ph (440) , (3) a d (λ) = a d (440) e -S d (λ-440) , (4) where P 2 is a parameter related to a ph (440) and the spectral slope parameter S d (nm -1 ) is randomly selected from a predetermined range of values (Table  1 ). The parameterizations of P 1 and P 2 were chosen to match relationships observed in the in situ dataset assembled in this study. Particulate backscattering is not modeled in terms of the single coefficient b bp (λ) but instead as separate contributions by phytoplankton, b b-ph (λ), and non-algal particles, b b-d (λ), so that their sum yields b bp (λ).
These results are in good agreement with previous studies  (Morel and Maritorena, 2001; Loisel et al. , 2006; Antoine et al. , 2011). The variability of measured a ph (440) illustrated in Fig. 5a , along with the dynamic range of parameters P 1 , P 2 , P 3 , P 4 , the spectral slopes S g , S d , S c-ph , and S b-d , and the degree of randomness in the selection of these parameters for any given value of a ph (440) that initiates the process of calculating a g (λ), a d (λ), b b-ph (λ), and b b-d (λ), resulted in the generation of the synthetic dataset of IOP coefficients covering a wide dynamic range consistent with in situ and satellite observations over the global ocean. Figure  5b , c, and d compare the probability distributions of satellite-derived a g (440), a dg (440), and b bp (550) with the distribution of these coefficients from the final synthetic IOP dataset.
This comparison supports the general consistency of the distributions of these IOP coefficients, which is in line with the desired consistency achieved for a ph (440) (Fig. 5a ) as discussed earlier in this section. It is also noteworthy that in contrast to this newly created synthetic IOP dataset, the previous synthetic datasets exhibit significant differences between the probability distributions of synthetic IOPs and global distributions based on satellite observations (Fig. 6 ). Overall, the above-described synthetic IOP dataset includes 3320 scenarios of non-water IOPs, i. e. , IOPs associated with the variable contributions of phytoplankton, nonalgal particles, and CDOM to the optical properties of seawater. In addition to the non-water absorption coefficients, a ph (λ), a d (λ), and a g (λ), as well as the non-water scattering coefficients, b ph (λ) and b d (λ), the radiative transfer simulations required input of scattering phase functions of particles, specifically for phytoplankton and non-algal particles.
We assumed the particulate phase functions proposed by  Fournier-Forand (1994)  with the backscattering ratio bb-ph = 0. 01 for phytoplankton and bb-d = 0. 018 for non-algal particles. Note that while the backscattering ratios are assumed to be spectrally constant, the phase functions vary with light wavelength because of spectral variations of b ph (λ) and b d (λ). All IOP data in the final synthetic IOP dataset cover the spectral range of 300 to 750 nm with a 5 nm interval. This wavelength interval is consistent with the intended output of our radiative transfer simulations. The radiative transfer simulations also required input of the absorption and scattering properties of pure seawater. For the spectral absorption coefficient of pure seawater, a w (λ), we used the values recommended in the IOCCG Protocol Series (2018).
This recommendation includes the values from  Jonasz and Fournier (2007)  for the spectral range 300-330 nm,  Morel et al. (2007)  for 340-415 nm,  Pope and Fry (1997)  for 420-725 nm, and  Kou et al. (1993)  for 730-750 nm. The spectral volume scattering function of pure seawater (from which the spectral scattering coefficient and scattering phase function can be obtained) was calculated following  Zhang et al. (2009) , assuming a water temperature of 18 • C and a salinity of 35 ‰. The temperature of 18 • C is consistent with the mean sea-surface temperature (SST) calculated from the monthly global NOAAv2 SST database at 1 • spatial resolution from December 1991 through November 2021  (Jérôme Vialard, personal communication, 2023). The salinity of 35 ‰ is also consistent with the global surface average  (Durack et al.
Radiative transfer simulations The IOP dataset described in Sect. 2, which includes 3320 combinations of non-water IOPs, provided the key input to radiative transfer (RT) simulations that were performed with the HydroLight v5. 0 radiative transfer code  (Mobley and Sundman, 2008). All RT simulations were run assuming vertically homogeneous IOPs within the water column and an infinitely deep ocean, i. e. , no seafloor effect on the light field within the water column. For all simulations, the computed https://doi. org/10. 5194/essd-15-3711-2023 Earth Syst. Sci. Data, 15, 3711-3731, 2023 radiometric and AOP variables were saved into the output data files at 10 cm depth intervals between the ocean surface and 1 m depth, and at 1 m intervals between 1 and 50 m depth. Thus, the primary focus of our RT simulations is on the ocean surface layer that can potentially contribute to light leaving the ocean and which has significance to remote sensing by spaceborne or airborne optical instruments.
02 in the HydroLight code. For each scenario of sun zenith angle and inelastic processes, we performed 3320 RT simulations, each for a different combination of seawater IOPs. Thus, given the three sun zenith angles, the three scenarios of inelastic processes, and 3320 combinations of IOPs, overall we performed 29 880 simulations. The combination of the synthetic IOP dataset used as input to RT simulations (Sect. 2) and the results of the radiance, other radiometric quantities, and AOPs obtained from these 29 880 simulations (described in this section) constitute the synthetic ocean optical database developed in this study.
Comparisons of the synthetic database with in situ data In this section, we compare the selected spectral IOP coefficients from the synthetic IOP dataset with the in situ data of the IOPs and the selected spectral AOPs from the synthetic database generated with the RT simulations with the in situ data of the AOPs. In these comparisons, we also include some empirical relationships between the IOPs and between the AOPs that were established in previous studies based on the analysis of in situ data. Figure  7  depicts scatter plots of IOP coefficients, specifically a g (440) vs. a ph (440) (Fig. 7a ), a dg (440) vs. a ph (440) (Fig. 7b ), a d (440) vs. a ph (440) (Fig. 7c ), and b bp (550) vs. a ph (440) (Fig. 7d ). The scatter plots include two datasets, the in situ dataset and the synthetic dataset, as described in Sect.
2. We recall that in both types of datasets, a ph (440) plotted on the x axis is the same because the phytoplankton absorption data used in this study were obtained from field measurements with no modeling involved. The scatter plots show a significant degree of overlap which indicates general consistency between the synthetic and in situ datasets. Similar patterns are observed when the a g (440)/a ph (440), a dg (440)/a ph (440), a d (440)/a ph (440), and b bp (550)/a ph (440) ratios are plotted as a function of a ph (440) (not shown). For illustrative purposes, the data from the synthetic IOP dataset are color coded to indicate the partitioning of data into the three OWGs, i. e. , Groups 1, 2, and 3, which were defined using the synthetic spectra of R rs (λ) generated through RT simulations with input of the synthetic IOP data.