dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2017) , was deployed in 2018 as part of the China-France Oceanography SATellite (CFOSAT) mission. The SWIM system uses a rotating radar beam and measures surface winds and waves, which will help achieve better calculations of surface fluxes at the air sea interface and provide more accurate wind and wave forcing. The second will be the wide-swath Surface Water and Ocean Technology (SWOT) altimeter scheduled to launch in 2021, which will provide two-dimensional observations of sea surface topography at kilometer resolution over a 120 km swath. This will be equivalent to having multiple Nadir altimetry satellites and provide much more detailed observations of sea surface height than are currently available. Wide-swath altimetry has the potential to provide a step change in ocean forecast performance through: (i) high spatial coverage with instantaneous two-dimensional snapshots of surface gradients leading to more dynamically balanced initial conditions for forecasts; (ii) high data coverage leading to a reduction in observation latency and thereby permitting higher forecast skill during the forecast period; and (iii) estimates of river discharge from major rivers leading to major improvements for ungauged rivers and estimates for international rivers.
The three new satellite systems combined with evolving in situ data programs will enable ocean prediction systems to better reproduce mesoscale features in the ocean and increase forecast accuracy. Finally, it is important to mention that satellite measurements of sea surface salinity (SSS) are becoming more reliable and that current missions are able to provide synoptic measurements of SSS over the global ocean with spatial scales that are much finer that those provided by the Argo array. Planned enhancements are to increase the spatial resolution to better resolve mesoscale features and to improve the accuracy of the satellite SSS observations, especially at high latitude (see Lee and Gentemann, 2018, for a review). Ocean data management is essential for the proper distribution of data to the operational systems.
This is addressed by several other papers in this issue (e. g. ,  Pearlman et al. , 2019; Tanhua et al. , 2019) , so it will not be addressed in detail here. Observations networks that follow the FAIR (findable, accessible, interoperable, and reusable) data management principles are an asset for operational oceanography but not necessarily sufficient, as will be discussed below. Observations are used in operational oceanography for validation, verification, real-time assimilation, and reanalysis assimilation. For validation (defined as assessing the quality of a system) and assimilation into reanalysis systems, there is the need for a standard set of quality-controlled observations, ideally perhaps with temporal consistency to ensure no aliasing from the observing network. In cases such as these, the FAIR principles are critical.
For verification (defined as monitoring the quality of products in operations) and assimilation in real-time there is the additional requirement for timeliness and robustness of observation availability. Realtime prediction systems normally operate on daily or more frequent cycles, ingesting observations from the near past. A data management system capable of delivering observations with associated information about the quality of the observation on a timescale suitable for real-time systems is also much needed. Timeliness is likely to become increasingly stringent as operational oceanography and operational meteorology come together in coupled ocean-atmosphere prediction systems. The rapidly evolving nature of the atmosphere and the need for rapid updates of information for users of these coupled systems will increasingly drive the timescale requirements and observations available within 3-6 h will become increasingly valuable.
As operational oceanography expands from the traditional surface waves and hydrography into chemistry and biogeochemistry where quality control is often less-easily automated and less observations are available in real-time, this will increasingly be an issue. Extending data management protocols similar to those used for drifting buoys and traditional Argo observations of profile temperature and salinity to other observation types is key to adding value to these observations from operational ocean systems. Additionally as indicated by  Smith et al. (2019) , all the above mentioned observation systems are needed to provide coverage in Arctic waters, for the prediction systems, as these are areas that are quickly evolving under global warming.
OCEAN MODELS Ocean numerical models are one of the essential building blocks of global and regional operational oceanography systems. A comprehensive operational system includes models that solve governing equations for ocean currents, sea levels, temperature, salinity, sea-ice, surface waves, and concentrations of tracers relevant to environmental or biogeochemical processes. Over the past decade, ocean circulation models have evolved through improved understanding, numerics, spatial discretization, grid configurations, parameterizations, data assimilation, environmental monitoring, and process-level observations and modeling (see companion paper by Fox-Kemper et al. , 2019, for a review). Increases in computing power and the improved physical consistency of their numerical formulation  (Griffies et al.
, 2000)  have allowed current state-ofthe-art ocean circulation models to routinely resolve oceanic flows down to mesoscales globally, sub-mesoscale regionally, and detailed structure in the coastal ocean down to harbors (e. g. ,  Álvarez-Fanjul et al. , 2018; Le Sommer et al. , 2018; De Mey-Frémaux et al. , 2019, this issue). Ocean models that do not include observational data assimilation are mostly used to scientifically explore underlying ocean mechanisms via idealized or realistic configurations. In conjunction with data assimilation, they are used to extrapolate in space and in time the available discrete oceanic observations to build a physically consistent estimate of the ocean state and its evolution. This, in turn allows for the creation of reanalysis products that can describe past evolution  (Ferry et al.
, 2012; Carton et al. , 2018; Wunsch, 2018) , or initial state to perform forecasts  (Chassignet and Verron, 2006; Dombrowsky et al. , 2009; Schiller and Brassington, 2011; Bell et al. , 2015; Chassignet et al. , 2018). Observational data via data assimilation sets the stage for model state estimates and forecasts  (Chassignet et al. , 2009) , with the quality of the estimates and the forecast being strongly dependent upon the ability of an ocean numerical model to faithfully represent the resolved dynamics of the ocean and the parameterized subgrid scale physics. Besides capturing large-scale ocean features, there is a strong need for the ability to predict the variety of spatial and temporal scales near the coastlines where a large part of the world's population lives. This poses both an observational as well as a modeling challenge and necessitates the development of suitable coastal ocean forecasting systems, capable of providing the backbone for societal and management needs  (Kourafalou et al.
, 2015). From such ensembles, one can then derive the uncertainty of the model forecast expressed often as the ensemble standard deviation. Alternatively, it is also possible to compute the probability of some events (e. g. , ocean currents) exceeding a given threshold or the probability of finding a drifter within a given search area. In order to provide a suitable ensemble forecast, it is also necessary to validate the reliability of the probabilistic forecast  (Candille et al. , 2007; Yan et al. , 2017). One of the main applications of ensemble forecast is to optimally reduce the model uncertainty using observations with e. g. , data assimilation (section Ocean Data Assimilation) or other approaches like Bayesian Model Averaging (e. g. ,  Salighehdar et al. , 2017). Dynamically, variations of ocean circulation are coupled with that of sea-ice, waves and atmospheric physics.
OCEAN DATA ASSIMILATION Incorporating observations into a prediction system via data assimilation is crucial for operational forecasting and reanalysis systems to obtain accurate estimates of the ocean state. Numerous data assimilation algorithms have been developed and applied in operational ocean forecasting systems. Most global operational global ocean forecasting systems apply threedimensional methods such as 3DVar or Ensemble Optimal Interpolation (and its variants)  (Martin et al. , 2015) , while regional and shelf-seas systems use a wide variety of methods including more advanced methods such as the Ensemble Kalman Filter (e. g. ,  Sakov and Sandery, 2015)  and 4DVar (e. g. ,  Edwards et al. , 2015). As the available computational resources increase, many of the global ocean forecasting centers are developing implementations of these more advanced methods  (Nerger and Hiller, 2013) , as well as hybrid ensemble/variational methods.
Below we describe some examples of where feedback from the assimilation to observing communities, and vice versa, should be improved. Assimilation systems require information about uncertainties in the observations and the model forecast. The observation errors are usually considered as two components: the measurement error, and the representation error (which accounts for the fact that the models do not represent all the physical processes and scales that a particular observation measures;  Janjić et al. , 2018). For some observation types there is considerable information available to the assimilation systems about the measurement errors; for instance, satellite SST datasets available through GHRSST provide per-observation uncertainty estimates. In other observing systems, and particularly for new satellite observations, there is a need to ensure that measurement uncertainty estimates are available to assimilation systems.
The representation errors estimated by the forecasting community for assimilation systems with different spatial and temporal characteristics could allow the observation community to better characterize the information content of the observations, although representation errors also depend on the specificity of the individual prediction systems (e. g. , in terms of spatial resolution and resolved processes). Quality control is carried out by many of the data assembly centers which provide observations to assimilation systems, both in real time and delayed mode. Real time assimilation systems also carry out an automatic quality control step using various information including short-range model forecasts (e. g. ,  Storto, 2016). This is a critical component of operational ocean forecasting systems since the assimilation of erroneous observations can severely degrade forecast quality.
, 1998). In this data set, three fronts are analyzed -the North Wall of the Gulf Stream, the Loop Current in the Gulf of Mexico, and the Kuroshio Current. The frontal positions of both the Navy analyses and the Global RTOFS data are mapped, and a metric is used to compute the geometric distance between the two fronts  (Dubuisson and Jain, 1994). Such observational programs require significant resources, manpower, and expertise. Class-4 metrics aim to measure the performance of a forecasting system, its capability to describe the ocean (in hindcast mode), as well as its forecasting skill (analysis and forecast mode). Class-4 metrics are model and observation products defined in the observational space, i. e. , the operational system is evaluated where observations are gathered. Class-4 metrics can be applied in delayed mode or in real time, knowing that in real time, less observations are available.
, 2004; Zebedin et al. , 2008; Lafarge et al. , 2010; Verdie et al. , 2015). Conversely, few works investigate the issue of assessing the quality of the derived models, especially out of a given reconstruction pipeline  (Schuster and Weidner, 2003). Usually, quality assessment is based on visual inspection  (Durupt and Taillandier, 2006; Macay Moreia et al. , 2013) , geometric fidelity metrics  (Kaartinen et al. , 2005) , or by extending standard 2D object detection criteria  (Karantzalos and Paragios, 2010) , without any semantic dimension. Only one benchmark dataset has addressed the issue  (Rottensteiner et al. , 2014). It remains focused on very few areas and a geometric comparison with manually extracted roof structures  (Li et al. , 2016; Nan and Wonka, 2017; Nguatem and Mayer, 2017).
Features used for this classification are extracted from very high spatial resolution (VHR, 0. 1 m to 0. 5 m) images and DSMs, like 3D segments or texture correlation score comparisons. In spite of their semantic contribution in quality evaluation, such taxonomies are prone to overfitting to specific urban scenes/ modeling algorithms or require the computation of complex features on This work aims to propose a new quality evaluation paradigm that detects and describes semantic errors that affect 3D building models. Two important characteristics must be taken into account. First, the definition of the semantic errors should not vary from one urban scene to another and guaranty independence to the underlying 3D reconstruction method. Secondly, the scalability of the method should be addressed in order to ensure the ability to correctly classify unseen areas and to define the minimal amount of data required.
In Figure  3. ii. d), we can observe how two cylindrical towers were reconstructed as a rectangular parallelepiped. • Imprecise geometry (FIG): inaccurate facet geometric estimation :e. g. , wrong height or inaccurate slope. The latter is depicted in Figure  3. ii. e 2. All these errors stem either from the modeling approach, or from the poor spatial resolution of the input data (DSM or point cloud). These errors can be related to state-of-the-art labels. For instance, "Missing Node" (resp. "False Node", "Missing Edge" and "False Edge") in  Xiong et al. (2014)  correspond to, or are   included in, the topological atomic errors from the Facet Errors family: FUS (resp. FOS, FIT, and FIT). The difference is that we distinguish flaws that can affect superstructure facets (LoD-3) from the ones that impair building facets (LoD-2).
Dataset 3D models from three different cities of France are selected in order to assess the performance of our framework: Elancourt, Nantes, and the XIII th district of Paris (Paris-13) (Figure  6. i). Elancourt is a small city exhibiting a high diversity of building types: residential areas (hipped roof buildings), and industrial districts (larger buildings with flat roofs). Nantes represents a denser urban setting with lower building diversity. In Paris-13, high towers, with flat roof, coexist with Haussmann style buildings that typically exhibit highly fragmented roofs. The Elancourt (resp. Nantes and Paris-13) scene contains 2,009 (resp. 748 and 478) annotated building models. The DSM and orthorectified image resolution is 6 cm while it is 10 cm for the two other areas. 3D models were generated using the algorithm described in  (Durupt and Taillandier, 2006) , out of existing building footprints and aerial VHR multi-view DSMs.
The second is made clear by the difference in roof colors, in dense uniform settings  (Figure 3. i. a). This helps identifying different instances of buildings. Figure  8  shows all "Building errors" family labels are better detected for Elancourt. It is also the case of FOS and FIT. A certain monotony can be noticed, at the exception of BOS. Better resultas are obtained for Paris-13 than for Nantes, while having around half the number of models to train on. This means that BOS cannot be easily learnt in Nantes. It is coherent with the fact that the dataset represents a part of the dense downtown of the city. The same monotony is observed, this time in reverse, with the rest of "Facet errors" defects. Paris-13 is much better with less training samples. For geometric defects  (FIG and FIB) , Nantes is comparable to Paris-13, but, with FUS, it is way much worse.
Scalability analysis It is established that the scene composition can affect greatly model defect detection. This fact motivates studying training the classifier and testing prediction on different scenes. The goal is to prove the resilience of the prediction to unseen urban scenes. As the annotation process require a lot of effort, this trait is crucial to guarantee the scalability of this method. Different configurations are possible (Figure  10 ). In a first experiment, we train on one urban scene and test on another one (transferability of the classifier model). In a second configuration, the classifier is trained on two scenes and tested on the last one: the goal is to investigate generalization. The last experiment targets the representativity of a single 3-area dataset by trying multiple train-test splits.
Generalization study We try to find out how omitting one urban zone from the training dataset affects the test results on that same area. An other way to look at it is, from an operational point of view, to find out how much learning on a union of many urban scenes is helpful when testing on an unseen one. We also We compare these experiments with the ablation study on the same area (cf. Table  4 ). We analyse result along the same criterea as the transferability study. We start again with a comparison depending on error families. Out of the 11 possibilities for the "Building errors" family, 8 yield worse results. For the "Facet" family, 6 out of 13 comparisons exhibit the same trend. This is worst than the transferability comparisons in ratio. This results from the fact that fusing more datasets, that are not tailored for a specific error detection, does not Similarly to the previous study, image and height modalities play a major role in error detection.
Image-based features are crucial for FIB, BIB, FUS and BUS detection (Table  4 ). Height-based attributes, however, induce a larger impovement in predicting FIG and BIT, while geometric ones are relegated to playing a minor role. Otherwise, a curiosity can be noticed: only when fusing all modalities together, in Paris-13 and Nantes, does predictions improve for BOS. We also confirm the observations about the best urban scene for error prediction training. In this case, the best zone should always give the worst scores. It is mostly the case with all atomic errors, with the exception of BIT. This outlier can be explained by the resemblance of the Paris-13 3D model to Nantes (which was established to be the best) samples. Indeed, for most labels, Nantes and Paris-13 reach the same scores. However, the discrepancy in F-scores proves the added value for each dataset.
Representativeness study The objective is to find out, after merging all training samples from all datasets, what is the minimal amount of data that can guaranty stable predictions. We can, thereafter, understand how much learning on one scene typology can affect results compared to a mixed training set. Figure  13  depicts F-score as a function of training ratios (between 20-70%) and atomic errors. We note the high stability of the F-score. This indicates that having a small heterogeneous dataset is not detrimental to the learning capacity and can be even the most suitable solution. ensured with a limited training set. No standard logarithmic behaviour can be found at the studied scales. 20% of the full label set is sufficient so as to retrieve results with a performance similar to the initial ablation study.
Conclusion A learning framework was proposed to semantically evaluate the quality of 3D models of buildings. For that purpose, errors were hierarchically organized into a novel flexible taxonomy. It aims to handle the large diversity of urban environments and varying requirements stemming from endusers (geometric accuracy and level of details). Based on the desired LoD, exclusivity and semantic level, an error collection is considered. Model quality is then predicted using a supervised Random Forest classifier. Each model provides intrinsic geometrical characteristics that are compiled in a handcrafted feature vector. Remote sensing modalities can be introduced. This helps better describing building models and detecting errors. Attributes can indeed be extracted by comparing the 3D model with optical images or depth data at the spatial resolution at least similar to the input 3D model.
Experiments shows it helps detecting hard cases both for geometrical and topological errors. This new framework was applied to the case of aerial urban reconstruction, where features are extracted from Very High Resolution airborne images and a DSM. A fully annotated dataset con-taining 3,235 aerial reconstructed building models with high diversity and from three distinct areas was used to test our method. It was associated with multimodal RGB optical and Digital Surface Model features. Although being mitigated over under-represented errors, results are satisfactory in the well balanced cases. More importantly, we proved that the urban scene composition affects greatly error detection. In fact, some predictions scores are not only stable, when training on a different urban scene, they even outperform when learning on the same scene.
Additionally, we reported how, for a heterogeneous training dataset, the size of the training set have, practically no effect, as test score stay stable for all errors. This demonstrates that the proposed framework can be easily scaled with the right choice of training samples with little manually generated data. This exactly answers to the raised problematic, contrarily to the present state-of-the-art literature. As a next step, more structure-aware features (based on graph comparison, for instance) could be proposed  (Boguslawski et al. , 2011 ) so as to be applied on a richer and more diverse dataset (potentially involving data augmentation) under a deep-based framework.
1A ). Data were then converted to CIE-xyY coordinates and plotted in a CIE-xy chromaticity diagram, which is a normalized representation of the colors perceived by trichromatic observers  (Stevens et al. 2009 ; Fig. S3 ). For each individual, we calculated the Euclidean distance between the color of the different pattern element components and that of the background of the wing. These distances were used as a proxy for internal contrast, or conspicuousness of pattern element components in relation to the wing background. We also calculated the distance from each pixel along the transect to the brown patches used in our behavioral assay (details below). The mean distances were used as an inverse proxy for color similarity or crypsis in our experimental setup.
STATISTICAL ANALYSES All statistical tests were done using R (R Core Team 2017) and the package afex  (Singmann et al. 2018). We examined the effects of developmental temperature (Td), adult temperature (Ta), and sex on wing pigmentation using linear mixed models (Table  S1 ), and on perching preference and activity levels using generalized linear mixed models with a binomial distribution and a logit link. For the behavioral data, the cohort and the time point (i. e. , consecutive photos) were included as random effects in the models. For both behavioral variables, minimum adequate models were obtained by backward elimination, starting from the full model and using the Bayesian Information Criterion (BIC). P-values were calculated via likelihood ratio tests and by parametric bootstrapping (nsim = 999).
ACKNOWLEDGEMENTS We are grateful to  Carolina Peralta  for technical assistance,  Filipa Alves  for developing a set of interactive Mathematica notebooks we based our pigmentation phenotyping scripts on,  Martin Stevens  for valuable advice on color quantification, and two anonymous reviewers for suggestions that greatly improved our manuscript. Financial support was provided by the  Portuguese science funding agency, Fundacão para a Ciência e Tecnologia, FCT  ( PTDC/BIA-EVF/0017/2014 ); French research funding agency,  Agence Nationale de la Recherche ,  ANR  ( Laboratory  of.