Increasing the maturity of measurements of essential climate variables (ECVs) at Italian atmospheric WMOGAW observatories by implementing automated data elaboration chains Abstract   In the framework of the National Project of Interest NextData, we developed automatic procedures for the flagging and formatting of trace gases, atmospheric aerosols and meteorological data to be submitted to the World Data Centers (WDCs) of the Global Atmosphere Watch program of the World Meteorological Organization (WMOGAW). In particular, the atmospheric Essential Climate Variables (ECVs) covered in this work are observations of nearsurface trace gas concentrations, aerosol properties and meteorological variables, which are under the umbrella of the World Data Center for Greenhouse Gases (WDCGG), the World Data Center for Reactive Gases, and the World Data Center for Aerosol (WDCRG and WDCA). We developed an overarching processing chain to create a number of data products (data files and reports) starting from the raw data, finally contributing to increase the maturity of these measurements. To this aim, we implemented specific routines for data filtering, flagging, format harmonization, and creation of data products, useful for detecting instrumental problems, particular atmospheric events and quick data dissemination towards stakeholders or citizens. Currently, the automatic data processing is active for a subset of ECVs at 5 measurement sites in Italy. The system represents a valuable tool to facilitate data originators towards a more efficient data production. Our effort is expected to accelerate the process of data submission to WMOGAW or to other reference data centers or repositories. Moreover, the adoption of automatic procedures for data flagging and data correction allows to keep track of the process that led to the final validated data, and makes data evaluation and revisions more efficient by improving the traceability of the data production process.

Unformatted, Certified Scientific Objects We present an approach for scientific data management systems to apply certificates to scientific objects, which are typically unformatted datasets, to facilitate analysis by climate scientists. Typically, for a program to process data, the program requires cleansed data in a form that supports automatic manipulation. Most systems require that data must adhere to a specific format to achieve that goal. The technique described in this paper takes the opposite approach; instead, any dataset may be imported and manipulated in the system. But upon initial import, however, only a subset of system functions may work with any given dataset. As the data is refined and transformed by system functions, more functions may become compatible. Certificates are associated with objects that pass constraint validation within the system to ensure that they conform to function requirements. The attached object constraints represent invariant properties of the object, which may be used by functions in the system as function preconditions. Furthermore, the functions defined in the system may associate certificates with the newly generated results. Certificates related to function results are effectively function postconditions, which in turn are used to associate certificates with the objects generated in the system. Additionally, attached object certificates reflect the refinement of data into a more pristine version. This paper describes the technique for modeling and enforcing the constraints for data scientists that have similar requirements.