dataesr/scientific-paragraphs-categorization huggingface dataset:
, 1988). Other collected data were provided by Robyn Tuerena and Anne Lorrain. Robyn Tuerena provided a data contribution coming from the data set mentioned in  Tuerena et al. (2019) , which we will refer to as the Tuerena data set. This contains 595 data points including 501 from  Young et al. (2013)  and covers samples within the euphotic zone and an observation time frame of 1964-2012. Moreover, we included 69 unpublished data points provided by Anne Lorrain, covering the years 2012-2015 and sampled during the cruises CASSIOPEE, PANDORA, OUTPACE, NECTALIS-3, NECTALIS-4 and KH-13. We refer to this data set as the Lorrain data set. A recent collection of 303 measurements of δ 13 C POC has been provided by  Close and Henderson (2020) , largely based on data gathered from individual publications referenced therein.
Otherwise, we tried to include the author, title, publication year and platform, and DOI. For unpublished data like Harrison's (unpublished data, quoted from  Goericke, 1994)  from the  Goericke (1994)  data set or those included by the co-authors, we specified from where we took the data. Coordinates are given in decimal degrees over [-90 • , 90 • ] × [-180 • , 180 • ]. The sample depth is given in meters measured positively from the ocean surface downwards. Data published as measured at 0 m were included Table  1. Changes that were introduced into data taken from  Goericke (1994) : the first column names the publication or author of the primary data set. The second column lists in which part of the data we applied changes. The third and fourth columns show what these changes were, and the last column gives the reason for this.
The time axis (TIME) increases in increments for each year from 1964 to 2015 by 1 and has a size of 52. This interpolation includes only δ 13 C POC data with full spatio-temporal metadata coverage; i. e. , additionally to latitude, longitude and depth, we also required and included year and month information. Ferret scripts were used for the interpolations. These averaged the irregularly measured data points within the ocean grid to one single data point representing each covered grid cell. The interpolation function SCAT2GRIDGAUSS by NOAA's Pacific Marine Environmental Laboratory (2020) performed the spatial averaging under PyFerret v7. 5. Calculations in this function are based on a work by  Kessler and McCreary (1992)  and can be summarized as follows: let (x 1 , y 1 ),. , (x n , y n ) ⊆ R 2 be an equidistant grid and ( x1 , ỹ1 ),.
Main data set characteristics The final data set includes 4732 individual δ 13 C POC measurements of seawater samples. We show the distribution of δ 13 C POC values by Gaussian kernel density estimation (KDE) in Fig. 1. KDEs are non-parametric density estimations  (Silverman, 1986)  for the approximation of probability density functions, which are theoretically similar to histograms but with continuous curves not dependent on rigid intervals. We applied a Python implementation from the SciPy stats package  (Virtanen et al. , 2020)  to create the results presented here. Likewise, we derived conditional probability densities of δ 13 C POC values, given the different measurement method applied (Fig. 3 ).
All sample devices provided data over all sample depths. Deeper samples were mainly taken from traps and pump systems and the upper samples from bottle and net data. Most data sampled deeper than 2600 m were collected by sediment traps. At 3800 m there were several trap contributions by  Calvert (e. g. , Calvert, 2002) , mostly from the late 1980s. Data sampled by a deep-sea manned submersible were given at locations down to 2520 m  (Lein and Ivanov, 2009). For resolving differences between sampling methods we chose data from the Atlantic Ocean which comprise all four major methods (with data embracing a region between 45 • S and 80 • N and 70 • W and 20 • E). In addition, data were distinguished by tropical, temperate and polar subregions. By crudely sorting the data according to their sampling locations, we gain some insight into methodological variability within a subregion and may relate this to variations between the three subregions (Fig.
Decadal variations The decadal UVic grid NetCDF files are the basis for showing long-term changes in the δ 13 C POC data. An overview of where the data within the individual decades were sampled is given in Fig. 10. This shows that the sparsest coverage was obtained in the 1960s, located close to the central American continent. Most data in the Indian Ocean were sampled in the 1970s. A cruise across the southern part of the Atlantic Ocean up to 30 • N and some samples close to Iceland were also measured in this decade. The 1980s is similarly sparse in spatial coverage to the 1960s. Measurements of the 1980s were taken at locations in the Southern Ocean, in the Arctic and in the Atlantic close to the Equator. The 1990s has the best coverage including most ocean basins. Most Southern Ocean data were sampled within the 1990s.
Conclusions The aim of this work was to construct the largest publicly accessible δ 13 C POC data set. The starting point of our collection and analyses was the readily available data collection of  Goericke (1994) , which comprised 467 data points. Our primary objective was to elaborate this set of data by adding useful meta-information from the original publications and by introducing additional δ 13 C POC measurements, as recorded in the world ocean database PANGAEA and made available by Robyn Tuerena and Anne Lorrain. This way we could expand the data collection substantially, from the original 467 to 4732 data points. This new δ 13 C POC data set provides the best coverage to date and will be a useful tool to help constrain many marine carbon cycling processes and pathways from ocean-atmosphere exchange to marine ecosystems, as well as to better understand observations and validate models.
To ensure dynamic growth of our data collection, the corresponding author will provide annual updates of the data set. Furthermore, he may be contacted by any interested researcher who would like to add their data to this collection. The data are provided in a csv structure and interpolated onto two different global grids in NetCDF format. The csv file contains the δ 13 C POC values, their anomalies to their mean and all available meta-information. The interpolations are provided on a coarse 1. 8 • × 3. 6 • grid of a δ 13 C POCsimulating model and a finer 1 • ×1 • grid by the World Ocean Atlas. We have provided a detailed description of our data collection procedure, all added meta-information and data coverage as well of the interpolation procedure carried out. We took the utmost care to make all data coherent, comparable and back-trackable and all adjustments transparent.
Assumptions, changes and deletions of the used data sets have been described in detail. We have described the general spatial and temporal trends of the sampled δ 13 C POC data of the raw data file. Distributions were always approximated by Gaussian kernel density estimators. The data range from 1964-2015 with by far the best coverage in the 1990s. Sample locations reach down to a depth of nearly 5000 m and best cover the uppermost 10 m, especially in the Atlantic and Indian Ocean. We were able to   The interpolated data provide insights into geographical behavior of the sampled δ 13 C POC data. We showed a good general coverage of all global oceans by δ 13 C POC but observed a lack of data in PANGAEA that cover North Pacific regions. Since the Atlantic Ocean provides the best coverage, corresponding data were used for a north-south trend anal-ysis, where we observed that the lowest values ( -28 ‰) can be found in the Southern Ocean, whereas the highest ( -22 ‰) are restricted to low-latitude regions.
Acknowledgements. We would like to thank  Tronje Kemena  for providing the basic global biomes masks, used for analyzing the interpolated data sets on the coarse grid. We thank the referees and editor for their constructive feedback regarding the initial version of the manuscript. Financial support. This research has been supported by the  Helmholtz School for Marine Data Science (MarDATA)  (grant no. HIDSS-0005 ) and by the  Deutsche Forschungsgemeinschaft (DFG)  (project no. 445549720 ). Review statement. This paper was edited by  Attila Demény  and reviewed by  Anne Morée  and one anonymous referee.
The results indicate that our best performing model had a balanced classification accuracy score of ≈ 90% when classifying active wave breaking in the test dataset. An example of a direct application of the method includes a statistical description of geometrical and kinematic properties of breaking waves. We expect that the present method and the associated dataset will be crucial for future research related to wave breaking in several areas of research, which include but are not limited to: improving operational forecast models, developing risk assessment and coastal management tools, and refining the retrieval of remotely sensed ocean properties.
Due to a lack of observed data, the constants involved in these models have been derived from limited datasets that may not adequately represent the natural environment. For example, a recent study has shown that popular surf zone parametric wave breaking models incorrectly represented the fraction of broken waves in their formulations with errors >50%, despite these models being able to adequately represent surf zone energy dissipation possibly due to parameter tuning  10. This paper aims to start addressing the data unavailability issue by providing a reliable and reproducible method to detect and track waves that are actively breaking in video imagery data. Wave breaking directly affects several environmental phenomena. For example, wave breaking is considered as the main driver for air-sea exchanges  11  , being often described as a function of  Phillips' 12  Λ(c)dc parameter.
The foam generated by breaking waves at the sea surface also influences on the measurement of microwave brightness temperature signatures for moderate to high wind speeds  19, 20. To date, no study has, however, systematically quantified such temperature signatures for large field datasets and most of our knowledge is empirical. Such limitation could be addressed by extending wave breaking detection and tracking methods to infrared wavelengths, which is directly correlated to water temperature  21. Advancements in this regard could lead to improvements in global climate models given that the inclusion of wave-generated heat transfer has shown to reduce cold biases in such models for non-breaking waves  22. Breaking waves are expected to transfer more heat than non-breaking waves  23  and, when considered, could improve climate models even further.
With the recent explosion in the usage of machine learning, particularly deep neural networks, it was only a matter of time before these techniques were adapted to wave research. Recent studies have shown that deep neural networks can accurately be used to classify different types of surf zone breakers  24  , obtain wave heights from video data  25  , track waves in the surf zone  26  and the shoreline position  27  and, when applied to pressure transducer data, distinguish between broken and unbroken waves  10. In this paper, we describe a robust and extensible method to identify and track breaking waves in video imagery data using a combination of classic machine learning algorithms and deep learning. More importantly, we make the present dataset and code base fully available for future researchers who can either use it directly, re-train the models adding more training data, or expand the present method to other cases (for example, object segmentation).
The development and standardization of a general framework to detect wave breaking in video imagery data should help to provide the wave breaking statistics database that is currently needed to assess, or further develop, our current understanding of several processes related to breaking waves. Ultimately, the efforts initiated here aim to lead to improvements in several areas of research such as wave modelling and forecasting, wave energy tracking and harvesting  28  , ocean-atmosphere interaction  15  , remote sensing of the ocean  16  , and safety at sea and coasts  29.
This step was easily parallelized with performance increasing linearly with number of computing threads. The second step of the method consisted of training a deep convolutional neural network to distinguish between active wave breaking and passive foam (and all other non-desired occurrences). This step can be understood as a binary classification step. Figure  1  shows a schematic representation of the present method. From the original images, randomly selected subsets of size 256x256 pixels centered on the fitted ellipses were extracted and manually labelled as either active wave breaking (label 1, or positive) or otherwise (label 0, or negative). The present training dataset was generated using raw video imagery data from Guimaraẽs et al. 36  and consisted of 19000 training and 1300 testing images (see Table  1 ).
The training dataset is further split into 80% training and 20% validation data  37. The validation dataset is reserved for future hyper-parameter fine-tuning. Note that the present dataset has a class imbalance of ≈90% towards the negative label, that is, for each active wave breaking sample (label 1, or positive) there are nine instances that were not active wave breaking (label 0, or negative). Data augmentation  38  (rotation, vertical and horizontal flips, and zoom) was employed during training to increase the variety of samples of the positive label. Five state-of-the-art neural network architectures, or backbones (VGG16 39 , ResNet50V2 40 , Incep-tionResNetV2  41  , MobileNetV2 42 and EfficientNetB5  43  ), were implemented and can be chosen by end users (see our Github code repository https://github.
Although VGG16 was chosen for presenting the results, the performance of the other architectures is nearly identical to VGG16 on real-world applications. Finally, note that user-tunable parameters for the neural networks (for example, learning rate and neuron activation thresholds) were kept unchanged from their default values in the TensorFlow library  48. This implicates that more aggressive parameter optimization could improve the results presented here even further. For sake of brevity, we refer the reader to the project's code repository for guidance on how to select hyper-parameters. Table  1. Data characterization summary table. f is the sampling frequency in Hertz, D is the duration of the experiment in seconds, H s is the significant wave height computed from the wave spectrum, T p1 and T p2 are, respectively, the peak wave period of the first and second spectral partitions computed following Hanson & Jensen 50 , D p is the peak wave direction, and U 10 is the wind speed measured or converted (denoted by the * ) to a height of 10m above the sea surface using Large & Pond's  51   The last step of the method consisted of grouping the active wave breaking events in space and time In the case of a grey-scale image, the single grey channel was triplicated.
The final classification layer has one unit and uses sigmoid activation with a threshold of 0. 5. (note that at this stage bright pixels are only grouped in space). Time-discrete wave breaking events had their bounding ellipse region filled in pixel space (i, j) and were stacked in time (t) resulting in a point-cloud-like three-dimensional (i, j,t) structure. The DBSCAN  34  algorithm was then used to cluster these data and obtain uniquely labelled clusters. The two parameters for the DBSCAN algorithm, that is, the minimum number of samples in each cluster (n min ) and the minimum distance allowed between two samples (eps), were set to be equals to the minimum number of points inside a fitted ellipse among all ellipses (n min ) and equals to the sampling frequency in Hertz (eps). These values for n min and eps were constant among the analyzed datasets.
Note that this final step can be replaced by any other density-based clustering algorithm or other more sophisticated algorithms such as SORT  52  (which is also available on the project's code repository but not was used here). Note that up to the clustering step, all calculations were done in pixel domain and "time" was obtained from sequential frame numbers. To convert between pixel and metric coordinates, the grids constructed using the stereo-video dataset available from Guimarães et al. 36  were used in the present study. If stereo video data are not available, such conversions could be done knowing the camera position in real-world (that is, metric) coordinates, which is usually done using surveyed ground control points  53, 54. Conversion from sequential frame numbers to time (in seconds) can be done by knowing the sample rate (in frames per second, that is, Hertz) for the data.
Evaluation Metrics Due to the imbalanced characteristic of the dataset, the classification accuracy score in isolation is not an appropriated metric to evaluated the present test dataset. For instance, a classifier that guesses that all labels are negative (that is, 0) would automatically obtain a high score (≈ 90%). To properly assess the performance of the classifier, more robust metrics were defined. These metrics are defined as follows: • True Positives (T F) and True Negatives (T N) are samples that were correctly classified. False Positives (FP) or Type I errors are false samples that were incorrectly classified as true, and False Negatives (FN) or Type II errors are true samples that were classified as false. • Accuracy is the percentage of examples correctly classified considering both classes, that is, Accuracy= T P+T N T +N , where T and N are the total number of positive and negative samples, respectively.
Classifier performance From the analysis of all training curves, confusion matrices, and from   2-a  and b , for example). In general, as the loss value decreased, the number of false positives decreased, which made the precision and recall to increase. This behavior was consistent for all models. Given that different architectures may perform better for other datasets and further optimization could change the model ranking presented here, all pre-trained models and their training metrics evolution are made available in the project's code repository. Finally, it is worth mentioning that larger models (for example, VGG19, ResNET152 and EfficientNetB7) could achieve better results but this was not attempted there due to hardware limitations (that is, these models required more memory than what was available on the NVIDIA GTX 1080 GPU used in this study).
confirmed the ability of the neural network to correctly classify the naïvely identified wave breaking candidates and only keep instances that represented active wave breaking. Moreover, the same neural network was able to correctly classify active wave breaking events for the rogue waves seen at La Jument  29  and for the small wind-generated breakers seen in the Black Sea data. This result highlights the ability of the neural network to generalize well on the dataset, which is a difficult result to achieve. From the analysis of the training curves, the averaged classification error (accounting for the imbalance in the data) should be of the order of ≈10%, which to the authors' knowledge it was not assessed by other wave breaking detection methods. We strongly recommend that future research should report active wave breaking detection errors when used to assess or further develop models that depend on wave breaking data.
Comparison with Mironov & Dulov (2008) To highlight the improvements obtained by the present method, this section presents a comparison between Mironov & Dulov's (2008)  30  automatic active wave breaking detection method and the present method. To perform this task, data from the 2013 Black Sea experiments in Table  1  that had previously been classified and investigated in detail by Guimarães  55  were used. All the active breaking events (that is, considering Animations of these results are available at https://github. com/caiostringari/deepwaves. In this particular example, the image was subdivided into blocks of 256x256 pixels for processing. Note that identical results were seen using other architectures other than VGG16 to classify these data. all 900s of data) detected using Mironov & Dulov's (2008)  30  were manually classified as true or false and compared to the labels predicted by our method.
To the authors' knowledge, these data are the only currently available data that has been classified by both methods as well as manually. Figure  4  shows the result of the compassion between models. On average, our method has relatively ≈50% less error then Mironov & Dulov's (2008) method with an averaged absolute reduction in error of ≈15%. The results in Figure  4  are also consistent with the results seen in Table  2  which showed that our model had errors in the order of ≈15% when considering the validation and test datasets. Note that all tested neural network architectures performed very similarly with only a slight advantage for InceptionResnetV2. It is also worth mentioning that Mironov & Dulov's (2008) method was designed and optimized to work specifically with data from the Black Sea and it is not guaranteed that it will generalize to other datasets.
Wave Breaking Statistics In this section we briefly present examples of wave breaking statistics that can be directly derived from the classified wave breaking data. For brevity, the analysis is limited to data from the Black Sea because our classifier performed best at this location (classification errors < 10% considering both 2011 and 2013 experiments). Five quantities will be analysed: the wave breaking duration (T br ), the wave breaking area (A br ), the major (a) and minor (b) axis of the fitted ellipses (representative of the wave breaking lengths at their maximum during the active part of the wave breaking), and Phillips' distribution Λ(c)dc. These quantities were obtained directly from space-time clustered wave breaking events with the exception of the cumulative wave breaking area (A br ) which was calculated from the projections of pixels clustered in the first step of the method to metric coordinates.
Discussion We have presented a new method to detect active wave breaking in video imagery data that is robust and easily transferable to future studies. Overall, VGG16 was the best performing architecture which is a surprising result given that VGG is a considerably older architecture that has been superseded by more recent architectures such as ResNets and EfficientNets  43. Also surprisingly, EfficientNet was one of the worst performers considering the test dataset despite the state-of-the-art results that this architecture achieved in recent years  43. One explanation could be that given VGG16 is the model with the highest number of parameters (despite being the shallowest network), it adapted better to the characteristics of the present dataset. Another explanation could be that EfficientNet is currently overfit on the ImageNet  58  dataset that was used to train this network hence its high reported classification scores.
The simplest explanation could be that this type of events is under-represented in the training dataset. The solution for this issue consists of adding more data representative of these instances to the training dataset. The second possibility is that the image size becomes too small in the deeper layers of the network which makes it impossible for the network to learn these events (see below for further discussion). A solution for this issue could be to increase the input image size but this was not attempted here due to hardware constraints (that is, memory limitation, as discussed above). Neural networks have been historically seen as black boxes in which only the final classification outputs are relevant. There has been, however, an increase in interest to understanding how neural networks are learning.