#  Executive Summary

ODYSSEA intends to develop, operate and demonstrate an interoperable and cost-
effective platform that fully integrates networks of observing and forecasting
systems across the Mediterranean basin, addressing both the open sea and the
coastal zone.

The platform is prepared to deliver a set of services focused on different
coastal user’s needs (navigation safety, ports operations, water pollution
prevention and response, eutrophication risks, search and rescue missions,
etc.) enabling the exploitation of the added value of integrated Earth
Observation (EO) technologies (satellite, airborne and ground based),
Copernicus Marine Service and ICT to deliver customized and ready to use
information. These services will provide an easy way to access in-situ data,
local high-resolution forecasts and products and services (e.g. meteo-
oceanographic conditions at specific locations, identification of optimum or
critical working windows, support to sea pollution response actions, etc.) for
a broad range of different users.

Taking in consideration that this platform will gather a large number of
diverse data sets (from existing networks and platforms and the ODYSSEA-
produced data consisting of operational numerical models data, data from in-
situ sensors and remote sensing data), the issues of data management and data
quality control assume a central concern. One goal of the platform is to
ensure that data from different and diverse data providers are readily
accessible and useable to a wider community. To achieve that, the strategy is
to move towards an integrated data system within ODYSSEA that harmonizes work
flows, data processing and distribution across different systems.

The value of standards is clearly demonstrable. In oceanography, there have
been many discussions for processing data and information. Many useful ideas
have been developed and put into practice, but there have been few successful
attempts to develop and implement international standards in managing data.

This document intends to provide an overview of the best practices concerning
these aspects and define the guidelines to be followed in themes such as
catalogues, metadata, data vocabulary, data standards and data quality control
procedures. This implies taking actions at different levels:

  * Adopt proper data management procedures to implement metadata, provide an integrated access to data in order to facilitate the integration in existing systems and assure the adoption of proper data quality control. 

  

  * Enable integration of more data, improve the enhancement of the services (viewing, downloading, traceability and monitoring) to users and providers, facilitate the discovery of data through a catalogue based on ISO standards, provide OGC services (SOS, WMS, WFS, etc.) to facilitate development; and facilitate the visibility of existing data and the identification of gaps. 

  

This deliverable will provide the guidelines and the first version of the Data
Management Plan (DMP) for the ODYSSEA Platform including strategies for
improving data management, data privacy and data quality control. As an EU
supported Mediterranean- focused platform, the data management will tune in to
specific aspects of ocean data management within the European context such as
existing networks of data and requirements of European industry and other end-
users.

An updated version of the DMP will be delivered in M18 of the project. It will
describe the implementation process, lessons learned and barriers overcome in
data management while deploying the ODYSSEA Platform. It will further
elaborate specific relevant aspects for ODYSSEA namely the "new" data series
approach using SOS, further integration into the ODYSSEA Platform among
others.

#  Introduction

ODYSSEA aims to provide a set of services focused on different coastal user’s
needs (navigation safety, ports operations, water pollution prevention and
response, eutrophication risks, search and rescue missions, etc.) allowing to
exploit the added value of integrated Earth Observation (EO) technologies
(satellite, airborne and ground based), Copernicus Marine Service and ICT to
deliver customized and ready to use information. These services will provide
an easy way to get in-situ data, local highresolution forecasts and products
and services (e.g. meteo-oceanographic conditions at specific locations,
identification of optimum or critical working windows, support to sea
pollution response actions, etc.) to a broad range of different users.

This report describes the strategies to be implemented to improve data
management, data privacy and data quality control of ODYSSEA services. The
strategy has four main components:

  * Catalogues, vocabulary and metadata; 

  

  * Data integration and fusion; 

  

  * Data quality control; 

  

  * Data privacy policy. 

  

The issue of metadata, vocabulary and catalogues is of prime importance to
assure the interoperability and easy discovery of data. A proper data
management following widely accepted standards also contribute to reduce the
duplication of efforts among agencies; to improve the quality and reduce costs
related to geospatial information, thus making oceanographic data more
accessible to the public and helping to establish key partnerships to increase
data availability. Aiming to contribute to these objectives, ODYSSEA will
adopt the procedures already proposed in the most relevant EU initiatives such
as CMEMS, EMODNet and SeaDataNet, especially the standards in relation to
vocabularies, metadata and data formats. In practice the gridded data sets
addressing either dynamic data sets (similar to CMEMS) or static data sets
(similar to EMODnet) will follow procedures similar to the ones adopted by
these two services. Regarding the time series data, SeaDataNet procedures will
represent the main guidelines and NetCDF-CF format will be the standard to be
adopted. However, ODYSSEA will go one step forward and will use these NetCDF
files to feed an SOS service, supported by North 52 software to assure the
interface with the users of the platform.

The capability of serving time series through a standard protocol, such as
SOS, will represent a step forward in relation to the existing services
although, being a pioneer, it is foreseen that some barriers must be broken.
The service will be tested in the ODYSSEA platform V0 Edition and it will be
later the subject of a detailed assessment in Deliverable D3.3 or in an
updated version of this Deliverable.

The data integration and fusion policies to adopt in ODYSSEA it is another
relevant issue of the project. Data integration and fusion deals with the best
strategies to adopt when it comes to merge datasets obtained from different
data sources, to build the best available datasets or fuse different data
sources to produce aggregated data. Although not being an easy ground, a
proper address of this issue may represent a valuable contribution to improve
data accuracy and robustness of models’ initial and boundary conditions and to
provide to the users comprehensive data that merge together different data
sets based on reliable criteria.

The data quality control either related with the quality of observed in-situ
data (e.g. tidal gauges, wave buoys, weather stations, etc.) or the modelled
forecasts is another relevant aspect that will be addressed by ODYSSEA. In the
case of local acquired data automatic procedures will run regularly to detect
and remove anomalous values from observed datasets. In the case of the models,
the results will be automatically compared with observations (e.g., buoys and
CMEMS grid observation products) and the statistical analysis will be provided
on a daily basis to the end users.

In relation with data privacy (data protection and the rights of platform end-
users, customers and business contacts), it is apparent that ODYSSEA will
assure the respect of their personal data under the General Data Protection
Regulation (GDPR) (Regulation (EU) 2016/679) which will substitute Directive
95/46/EC on May 25, 2018. ‘Personal data’ means any information, private or
professional, which relates or can be related to an identified or identifiable
natural person (for the full definition, see Article 2(a) of EU Directive
95/46/EC).

In the following paragraphs a more detailed overview both of the state of the
art and the procedures to be adopted in ODYSSEA will be provided. Note that at
this stage this document mostly focuses in defining the guidelines to be
followed throughout ODYSSEA platform development, and although it does not
reflect yet a practical implementation of these guidelines which will be
subject of a later document.

#  Ocean Data Management: the European context

Delivery of data to users requires common data transport formats, which
interact with other standards  (Vocabularies,  data q  uality control).
Several initiatives exist within Europe for ocean data management, which are
now coordinated under the umbrella of EuroGOOS. EuroGOOS is the network
committed to develop and advance the operational oceanography capacity of
Europe, within the context of the intergovernmental Global Ocean Observing
System (GOOS). The scope of EuroGOOS is wide and its needs are partially
addressed by the on-going development within Copernicus, SeaDataNet and other
EU initiatives.

Therefore, to improve the quantity, quality and accessibility of marine
information, to support decision making and to open up new economic
opportunities in the marine and maritime sectors of Europe for the benefit of
European citizens and the global community, it was agreed at the annual
EuroGOOS meeting in 2010 that it is essential to meet the following needs
(AtlantOS, 2016):

  * Provision of easy access to data through standard generic tools; where “easy” means the direct use of data without concerns on data quality and processing and that adequate metadata are available to describe how the data were processed by the data provider. 

  

  * To combine in situ-observation data with other information (e.g., satellite images or model outputs) to derive new products, build new services or enable better-informed decision-making. 

  

The ocean data management and exchange process within EuroGOOS intends to
reduce the duplication of efforts among agencies; to improve the quality and
reduce costs related to geospatial information, thus making oceanographic data
more accessible to the public and helping to establish key partnerships to
increase data availability. In addition, a EuroGOOS data management system
intends to deliver a system that will meet European needs, in terms of
standards and respecting the structures of the contributing organizations.

The structure will include:

  * Observation data providers, which can be operational agencies, marine research centres, universities, national oceanographic data centres and satellite data centres. 

  

  * Integrators of marine data, such as the Copernicus in-situ data thematic centre (for access to near real-time data acquired by continuous, automatic and permanent observation networks) or the SeaDataNet infrastructure (for quality controlled, long-term time series acquired by all ocean observation initiatives, missions, or experiments), ICES and EurOBIS for biodiversity observations, and the new European Marine Observation and Data Network (EMODnet) portals. The integrators that will support both data providers, willing to share their observation data, and users requesting access to oceanographic data (historic, real-time and forecasts). Integrators develop new services to facilitate data access and increase the use of both existing and new observational data. 

  

  * Links with international and cross-disciplinary initiatives, such as GEOSS (Global Earth Observation System of Systems), both for technical solutions to improve harmonization in an interdisciplinary global context. 

  

##  Towards an integrated EU data system

ODYSSEA aims to contribute to improve data availability across the
Mediterranean basin, addressing both the open sea and the coastal zone. One
goal is to ensure that data from different and diverse insitu observing
networks and forecasting models are readily accessible and useable. To achieve
this, the strategy is to move towards an integrated data system that
harmonizes work flows, data processing and data distribution across the in-
situ observing network systems, and integrates in-situ observations into
existing European and international data infrastructures (the so called
“Integrators”). These include the Copernicus INS TAC, SeaDataNet NODCs,
EMODnet, EurOBIS, and GEOSS.

The targeted integrated system deals with data management challenges that must
be met to provide efficient and reliable data service to users. These include:

  * Common quality control for heterogeneous and near real time data; 

  

  * Standardization of mandatory metadata for efficient data exchange; 

  

  * Interoperability of Network and Integrator data management systems. 

  

##  Industry requirements

Presently, there is a need to change the way marine observatories and public
data-sharing initiatives engage with industry and users. Columbus project
proposes a set of recommendations designed to overcome some of the most
important gaps and barriers still faced by private data users. Taken together,
they represent the basic components of a strategy to open significant
opportunities for maritime industry to both benefit from and engage with
public marine data initiatives. This can ensure the optimum return of public
investments in the marine data sector, notably in support of meeting key EU
policy goals under the Blue Growth Strategy, the Marine Strategy Framework
Directive and the Maritime Spatial Planning Directive. Some barriers require
further analysis and discussion, but there are already many actions that can
be undertaken to improve the situation on the short and medium term (Columbus,
2017):

  * _Industry representatives should be included in the governance and take part in the entire cycle of decision making, development and operation of marine observation and data-sharing initiatives._

  

  * _There is a need for marine data-sharing initiatives to take a more pro-active approach and move out of the comfort zone of the traditional oceanographic marine monitoring and observing communities. This involves, among others, developing a more “service oriented approach”, learning new communication skills and language, being present and more visible in fora that attract industry and to exploit creative technologies._

  

  * _Data, products and services offered by marine observation and data initiatives should be presented in a user-friendly, attractive and intuitive way which is adapted to the target users. If users from different communities or sectors are targeted, options to adjust the interface depending on the visitor should be considered._

  

  * _Clear, succinct and open communication is critical: it should be instantly clear for industry what data, products and services are offered and what may be made available in the future. Equally important is to provide information on what is not available, and the limitations of the resources offered._

  

  * _More efforts should be made to build upon early achievements and successes: presenting use case examples can trigger interest where there may previously have been none._

  

  * _There is a significant role for maritime clusters in connecting marine data initiatives with industry and vice versa. Maritime clusters are an important bridge between private and public sector as they deal with both and have a good understanding of their culture, language, needs and concerns._

  

  * _At European level there is a need for defragmentation of the plethora of marine observation and data and information sharing initiatives, as well as the online data portals. In the longer term, there is a need for a joint roadmap, agreed by the responsible coordinating and funding bodies including at the European Commission level, to set out the strategic framework._

  

  * _Dedicated data-sharing policies to incentivise the private sector and address their specific needs should be developed. Ways forward could include: stating clearly the added-value or benefits of sharing data, moratorium on commercially sensitive data, provision of services in return for data which could support in-house data management, the development of a data-sharing ‘green label’ in recognition of corporate social responsibility. It is clear that implementation of the recommendations will require increased commitment and investment of time and resources, both from industry and from marine observation and data initiatives, but should provide both with significant returns over time_

  

##  The ODYSSEA approach

The procedures to follow in ODYSSEA regarding this issue of the data
management will preferentially follow the examples from CMEMS, EMODNet or
SeaDataNet. In practice two major data types will be addressed: the gridded
data and the time series data.

The gridded data may address dynamic data sets (similar to CMEMS) or static
data sets (similar to EMODnet). In both cases the procedures to follow will be
similar to the ones adopted by these two services.

Regarding the time series data, SeaDataNet procedures will represent the main
guidelines and NetCDFCF format will be the standard to be adopted. However
ODYSSEA will go one step forward and will use these NetCDF files to feed an
SOS service, supported by North 52 software to assure the interface with the
users.

#  Data quality control

The issue of data quality control will be addressed following the state of the
art recommendations of different projects such as SeaDataNet or AtlantOS.
SeaDataNet produced a comprehensive document presenting a set of guidelines to
be followed in marine data quality control. According to this document, from
which part is reproduced bellow, data quality control essentially and simply
has the following objective: “ _To ensure the data consistency within a single
data set and within a collection of data sets and to ensure that the quality
and errors of the data are apparent to the user who has sufficient information
to assess its suitability for a task_ ”. If done well, quality control brings
about several key advantages (SeaDataNet, 2010):

  * _**Maintaining Common Standards** : There is a minimum level to which all oceanographic data should be quality controlled. There is little point banking data just because they have been collected; the data must be qualified by additional information concerning methods of measurement and subsequent data processing to be of use to potential users. Standards need to be imposed on the quality and long-term value of the data that are accepted (Rickards, 1989). If there are guidelines available to this end, the end result is that data are at least maintained to this degree, keeping common standards to a higher level. _

  

  * _**Acquiring Consistency** : Data within data centres should be as consistent to each other as possible. This makes the data more accessible to the external user. Searches for data sets are more successful as users are able to identify the specific data they require quickly, even if the origins of the data are very different on a national or even international level. _

  

  * _**Ensuring Reliability** : Data centres, like other organisations, build reputations based on the quality of the services they provide. To serve a purpose to the research community and others their data must be reliable, and this can be better achieved if the data have been quality controlled to a ‘universal’ standard. Many national and international programmes or projects carry out investigations across a broad field of marine science which require complex information on the marine environment. Many large-scale projects are also carried out under commercial control such as those involved with oil and gas and fishing industries. Significant decisions are made, and theories formed, on the assumption that data are reliable and compatible, even when they come from many different sources. _

  

ODYSSEA services data flux will be managed automatically by the ODYSSEA
platform. The data quality control will start by the execution of automatic
procedures (independently of the adoption of more complex procedures). The
data quality control methodology will focuses on in situ observations and
modelled forecasts and it will be addressed from two perspectives: the data
**Quality Assurance** and the **Quality Control** .

Quality Assurance (QA) is a set of review and audit procedures implemented by
personnel or an organization (ideally) not involved with normal project
activities to monitor and evaluate the project to maximize the probability
that minimum standards of quality are being attained. With regard to data, QA
is a system to assure that the data generated is of known quality and well-
described data production procedures are being followed. This assurance relies
heavily on the documentation of processes, procedures, capabilities, and
monitoring. Reviews verify that data quality objectives are being met within
the given constraints. QA is inherently a human-in-the-loop effort and
substantial documentation must accompany any QA action. QA procedures may
result in corrections to data. Such corrections shall occur only upon
authorized human intervention (e.g., marine operator, product scientist,
quality analyst, principal investigator) and the corrections may either be
applied in bulk (i.e., all data from an instrument during a deployment period)
or to selective data points. The application of QA corrections will
automatically result in the reflagging of data as ‘corrected’.

Quality Control (QC) is a process of routine technical operations, to measure,
annotate (i.e., flag) and control the quality of the data being produced.
These operations may include spike checks, out-of-range checks, missing data
checks, as well as others. QC is designed to:

  * Provide routine and consistent checks to ensure data integrity, correctness, and completeness; 

  

  * Identify and address possible errors and omissions; 

  

  * Document all QC activities. 

  

QC operations include automated checks on data acquisition and calculations by
the use of approved standardized procedures. Higher-tier QC activities can
include additional technical review and correction of the data by human
inspection. QC procedures are important for:

  * Detecting missing mandatory information; 

  

  * Detecting errors made during the transfer or reformatting; 

  

  * Detecting duplicates; 

  

  * Detecting remaining outliers (spikes, out of scale data, vertical instabilities, etc); 

  

  * Attaching a quality flag to each numerical value to indicate the corrected observed data points. 

  

A guideline of recommended QC procedures has been compiled by project
SeaDataNet after reviewing NODC schemes and other known schemes (e.g. WGMDM
guidelines, World Ocean Database, GTSPP, Argo, WOCE, QARTOD, ESEAS, SIMORC,
etc.). The guideline at present follows the QC methods proposed by SeaDataNet
for CTD (temperature and salinity profiles), current meter data (including
ADCP), wave data and sea level data. SeaDataNet is also developing efforts for
extending the guideline with QC methods for surface underway data, nutrients,
geophysical data and biological data.

ANNEX I provides a detailed description of the implementation process
procedure to be followed for QA/QC in ODYSSEA.

##  Quality Control Flags

According to EuroGOOS (2016), an extensive use of flags to indicate the data
quality is recommended, since the end user will select data based on quality
control flags, amongst other criteria. These flags should always be included
in any data transfer (e.g., from ODYSSEA Observatories to the central ODYSSEA
platform) maintaining standards and ensuring data consistency and reliability
( _see Table 1_ ). The same flag scale is also recommended by SeaDataNet.

TABLE 1: QUALITY FLAG SCALE (REPRODUCED FROM EUROGOOS, 2016)

**Code Definition**  
  
<table>  
<tr>  
<th>

0

</th>  
<th>

No QC was performed

</th> </tr>  
<tr>  
<td>

1

</td>  
<td>

Good data

</td> </tr> </table>

  2. Probably good data 

  

  3. Bad data that are potentially correctable 

  

  
  
<table>  
<tr>  
<th>

4

</th>  
<th>

Bad data

</th> </tr> </table>

  5. Value changed 

  

  6. Bellow detection limit 

  

  7. In excess of quoted value 

  

  8. Interpolated value 

  

  
  
<table>  
<tr>  
<th>

9

</th>  
<th>

Missing value

</th> </tr> </table>

A Incomplete information

_Data with QC flag = 0 should not be used without a quality control made by
the user._

_Data with QC flag different from 1 on either position or date should not be
used without additional control from the user._

_If date and position QC flag = 1 only measurements with QC flag = 1 can be
used safely without further analyses_ _if QC flag = 4 then the measurements
should be rejected_ _if QC flag = 2 the data may be good for some
applications, but the user should verify this_ _if QC flag = 3 the data are
not usable, but the data centre may be able to correct them in a delayed_

_mode_

##  In situ observations quality control

The quality control of observations may be done in two phases. During the
download of in-situ observations automatic checks should be done such as those
proposed by SeaDataNet (2010) (e.g. global range test, date and time). After
quality control, only the valid data is stored in the database. At the second
phase a tool may be run to periodically perform a scientific quality control
check (SeaDataNet, 2010). This quality control aims to detect spikes, filter
high frequency noise (e.g. moving average or P50), data with abnormal
variability in time, etc. Specific tools will be running automatically with
this aim.

##  Forecasts quality control

The modelled forecasts quality control may be done by comparing time-series
forecasts with in situ observations (e.g., wave buoys, tidal gauge, weather
stations, etc.) through automatically-run algorithms. Also, gridded data
forecasts may be compared automatically with observations (e.g., CMEMS gridded
data observations). As a result, several statistical parameters may be
computed (e.g., correlation coefficient, bias, RMSE, skill, etc.) to assess
the quality of forecasts.

#  Data integration and fusion

##  Low level data integration and fusion

The issue of the best strategies to adopt when it comes to merge datasets
obtained from different data sources, to build the best available datasets or
fuse different data sources to produce aggregated data, indices and products
it is not an easy ground. A possible solution when we have different solutions
with different resolutions for the same area is to make a fusion of these data
and offer a unique integrated dataset. Another option is to provide all
datasets separately with an eventual option of an integrated solution. No
matter the adopted solution, the final objective of the data integration and
fusion is to contribute to improve data accuracy and robustness of models’
initial and boundary conditions and to provide to the users comprehensive data
that merge together different data sets based on reliable criteria.

For example if a user is interested in wave data for a specific site and it
realizes that for the period in which he is interested there exist different
time series from different wave buoys, he may be interested in getting a
unique time series merging together and make compatible the different time
series data. This process may require complex actions regarding the levels of
accuracy of the different measuring devices, the measuring time rate and
units, etc.

##  Semantic Information Integration and Fusion

Capacity for integration and fusion of semantic information will be provided
through the ODYSSEA platform. Semantic information is made of several
information items, potentially coming from different semantically rich
information sources. The main use of this capacity is for semantic network
enrichment and query.

The information processed is expressed through graphs of entities related with
each other and contains semantic metadata. The fusion is adapted to the domain
of application. This application domain is described through an ontology of
the domain. The fusion process is also adapted to the quality of the
information items, through the use of fusion heuristics.

The fusion heuristics integrate domain knowledge and user preferences. They
are the intelligent part of the semantic fusion system. They are end-users
defined functions used to express the confidence the users have in the
information sources, as well as specific strategies that must be followed in
order to fuse information coming from different sources. The two main semantic
information integration functionalities are:

  * Insertion of new information in a semantic information network (Synthesis), 

  

  * Query for information in a semantic information network (Mining). 

  

#  Data management

##  Providers code for data

Following the procedures adopted by AtlantOS, the Institutions providing data
to ODYSSEA platform should be reported and acknowledged following the EDMO
code recorded in the data file and the ODYSSEA platform cataloque. EDMO is the
European Directory of Marine Organizations, developed under SeaDataNet, and it
can be used to register any marine organization involved in the collection of
datasets (operators, funders, data holders, etc.). It delivers a code for the
organization to be included in the data or metadata leading to the
harmonization of information (compared to free text) and the optimization of
the datasets discovery. EDMO is coordinated by MARIS.

For EU Countries new entries are added by the National Data Centres (NODCs).
Through ODIP (Ocean Data Interoperability Platform) cooperation, there is also
a point of contact with the USA, Australia and some other non-EU countries.
The rest of the world is managed by MARIS, which also moderates the first
entrance in EDMO of new entries.

The request for a new entry in EDMO is sent to MARIS (current contact: Peter
Thijsse (peter@maris.nl), who verifies if the institution is already
registered. If a new entry is needed, the basic entry is made by MARIS, after
which the appropriate NODC is responsible for updating further details and
managing changes.

##  Data vocabulary

Use of common vocabularies in all meta-databases and data formats is an
important prerequisite towards consistency and interoperability with existing
Earth Observing systems and networks. Common vocabularies consist of lists of
standardised terms of reference covering a broad spectrum of disciplines of
relevance to the oceanographic and wider community. Using standardised terms
of reference the problem of ambiguities related to data structure,
organization and format is solved and therefore, common algorithms for data
processing may be applied. This allows the interoperability of datasets in
terms of their manipulation, distribution and long-term reuse.

ODYSSEA will adopt an Essential Variables list of terms (aggregated level)
that has been defined and was published in June 2016 on the NERC/BODC
Vocabulary Server  1  .

This new vocabulary is mapped to the standards recommended for ODYSSEA
parameter metadata: P01

(parameter), P07 (CF variable), P06 (units) from SeaDataNet controlled
vocabularies managed by NERC/BODC and the internationally assured AphiaID from
the WOrld Register of Marine Species (WoRMS)  2  .

##  Metadata

Metadata refers to the description of datasets and services in a compliant
form as it has been defined by the Directive 2007/2/EC (INSPIRE) and
Commission Regulation No 1205/2008.

Metadata is the **data about the data** . Metadata describes how, when and by
whom a particular set of data or a service was collected or prepared, and how
the data is formatted, or the service is available. Metadata is essential for
understanding the information stored in and has become increasingly important.

Metadata is structured information that describes, explains, locates, or
otherwise makes it easier to retrieve, use, or manage an information resource.
Metadata is often called as the “data about the data or information about
information”.

Metadata is also data about services. Metadata describes the content, quality,
condition, and other characteristics of a data set or the capabilities of a
service. Creating metadata or data documentation for geospatial datasets is
crucial to the data development process. Metadata is a valuable part of a
dataset and can be used to:

  * **Organize** data holdings (Do you know what you have?). 

  

  * Provide **information about** data holdings (Can you describe to someone else what you have?). 

  

  * Provide information **to data users** (Can they figure out if your data are useful to them?). 

  

  * **Maintain the value** of your data (Can they figure out if your data are useful 20 years from now?). 

  

In the geographical domain we can have a description of spatial data (
**spatial data** metadata), a service ( **service** metadata) or a special
analysis process ( **process** metadata). Most for the standardization work is
done for data metadata, however service and process metadata become
increasingly important. Metadata is used in discovery mechanisms to bring
spatial information providers and users together. The following mechanisms are
recognized:

  * **Discovery** : which data source contains the information that I am looking for? 

  

  * **Exploration (or evaluation)** : do I find within the data sources the right information to suit my information needs? 

  

  * **Exploitation (use and access)** : how can I obtain and use the data sources? 

  

Each mechanism has its own use of metadata. The selected standards should
fulfil the needs to carry out services using these mechanisms. Metadata is
required to provide information about an organisation’s data holdings. Data
resources are a major national asset, and information of what datasets exist
within different organisations, particularly in the public sector, is required
to improve efficiencies and reduce data duplication. Data catalogues and data
discovery services enable potential users to find, evaluate and use that data,
thereby increasing its value. This is also becoming important at the European
level. In addition, metadata received from an external source may require
further information supplied to metadata to allow easy process and
interpretation.

In this context for all types of data the following information is required
(SeaDataNet, 2010):

  * **Where** the data were collected: location (preferably as latitude and longitude) and depth/height; 

  

  * **When** the data were collected (date and time in UTC or clearly specified local time zone); 

  

  * **How** the data were collected (e.g., sampling methods, instrument types, analytical techniques). How do we organize the data (e.g., in terms of station numbers, cast numbers); 

  

  * **Who** collected the data, including name and institution of the data originator(s) and the principal investigator; 

  

  * **What** has been done to the data (e.g., details of processing and calibrations applied, algorithms used to compute derived parameters); 

  

  * **Watch** points for other users of the data (e.g., problems encountered and comments on data quality). 

  

The ICES Working Group on Data and Information Management (WGDIM) has
developed a number of data type guidelines which itemize these elements that
are required for thirteen different data types (see table below). These Data
Type Guidelines have been developed using the expertise of the oceanographic
data centres of ICES Member Countries. They have been designed to describe the
elements of data and metadata considered as important to the ocean research
community. These guidelines are targeted towards most physical-chemical-
biological data types collected on oceanographic research vessel cruises. Each
guideline addresses the data and metadata requirements of a specific data
type. This covers three main areas:

  * What the data collector should provide to the data centre (e.g., collection information, processing, etc.); 

  

  * How the data centre handles data supplied (e.g., value added, quality control, etc.); 

  

  * What the data centre can provide in terms of data, referral services and expertise back to the data collector. A selection of these guidelines, in particular for those data types that are not yet dealt with in detail here, are included in Appendix 1 of this document. 

  

This document summarizes the concept of metadata that is intended to be
adopted by ODYSSEA data platform, following the commonly agreed INSPIRE data
specification template in its relevant parts, i.e., dataset-level, services
metadata and data quality. It also contains detailed technical documentation
on the XML source-code level and therefore provides specific guidelines to
correctly create and maintain metadata in the XML format.

##  Metadata Catalogue Service

A **Metadata Catalogue Service** is a mechanism for storing and accessing
descriptive metadata and allows users to query for data items based on desired
attributes. The catalogue service stores descriptive information (metadata)
about logical data items. The  Open Geospatial Consortium (OGC)  has created
the **Catalogue Service for Web (CSW) standard** to enable the easy data
discovery from a catalogue node. Catalogue services support the ability to
publish and search metadata for data, services, and related information.
Metadata in catalogues can be queried and presented for evaluation and further
processing by both humans and software. Catalogue services (and other
resources such as bibliographic resources, datasets, etc.) are required to
support the discovery and binding to published web map services. The CSW
standard is extremely rich. In addition to supporting a query from a user, it
can support distributed queries (one query that searches many catalogues) and
the harvesting of metadata from node to node.

Catalogue services support the ability to publish and search collections of
descriptive information (metadata) for data, services, and related information
objects. Metadata in catalogues represent resource characteristics that can be
queried and presented for evaluation and further processing by both humans and
software. Catalogue services are required to support the discovery and binding
to registered information resources within an information community.

The  International Organisation for Standardisation (ISO)  includes  ISO/TC
2112,  which is an international, technical Committee for the standardisation
of geographical information. TC 211 has created a strong, globally implemented
set of standards for geospatial metadata: the baseline  ISO 19115;  ISO 19139
for implementation of data metadata and the  ISO 19119  for services metadata.

These open standards define the structure and content of metadata records and
are essential for any catalogue implementation.  ISO 19115  describes all
aspects of geospatial metadata and provides a comprehensive set of metadata
elements. It is designed for electronic metadata services, and the elements
are designed to be searchable wherever possible. It is widely used as the
basis for geospatial metadata services. However, because of the large number
of metadata elements and the complexity of their data model, implementation of
ISO 19115 is difficult.

The  INSPIRE DIRECTIVE  applies these standards and specifications in its
implementation. INSPIRE makes use of three catalogues for unique IDs
management: **(1) SeaDataNet, (2) ICES and (3) CMEMS.** ICES catalogue has a
geospatial component not present in the SeaDataNet catalogue while CMEMS
provides the reference to model results.

###  Catalogue Service for Web (CSW)

This section describes briefly the Open GIS Consortium (OGC) specification for
catalogue services. According to this specification: “ _Catalogue services
support the ability to publish and search collections of descriptive
information (metadata) for data, services, and related information objects;
Metadata in catalogues represent resource characteristics that can be queried
and presented for evaluation and further processing by both humans and
software. Catalogue services are required to support the discovery and binding
to registered information resources within an information community_ ".

The Inspire initiative uses the CSW protocol and the ISO metadata application
profile (AP) for the specification and implementation of the Inspire Discovery
Service. In ODYSSEA, the ODYSSEA ISO metadata profile will be developed and
used as described in this document’s metadata sections.

The diagram presented below illustrates a generic view of the CSW protocol and
architecture.

###  Harvesting

Harvesting is the procedure of collecting metadata records from other
(external) catalogues and synchronize the local catalogue with the collected
information.

In the majority of the cases the harvesting process is scheduled and
automatically executed once or at pre-defined intervals. It is usually also
possible to execute a harvesting procedure on-demand, i.e., executed by human
request.

The diagram below depicts a sample on how the harvesting procedures could be
seen between the ODYSSEA platform catalogue and other external catalogues.

To be noted that the harvesting procedure uses, within Inspire, the CSW
protocol. Within the catalogue responses to the harvesting requests there are
collections of metadata records, using the model described in this document
(i.e., INSPIRE Datasets and Services).

##  Guidelines on using metadata elements

###  Lineage

Following the ISO 19113 Quality principles, if a data provider has a procedure
for quality validation of their spatial datasets then the data quality
elements, listed in Chapter 2, should be used. If not, the Lineage metadata
element (defined in Regulation 1205/2008/EC) should be used to describe the
overall quality of a spatial dataset.

According to Regulation 1205/2008/EC, lineage “is a statement on process
history and/or overall quality of the spatial dataset. Where appropriate it
may include a statement whether the dataset has been validated or quality
assured, whether it is the official version (if multiple versions exist), and
whether it has legal validity. The value domain of this metadata element is
free text”.

Apart from describing the process history, if feasible within a free text, the
overall quality of the dataset (series) should be included in the Lineage
metadata element. This statement should contain any quality information
required for interoperability and/or valuable for use and evaluation of the
dataset (series).

###  Temporal reference

According to Regulation 1205/2008/EC, at least one of the following temporal
reference metadata elements shall be provided: temporal extent, date of
publication, date of last revision, date of creation.

If feasible, the date of the latest revision of a spatial dataset should be
reported using the date of latest revision in a metadata element.

###  Topic category

The topic categories defined in Part D 2 of the INSPIRE Implementing Rules for
metadata are derived directly from the topic categories defined in B.5.27 of
ISO 19115. Regulation 1205/2008/EC defines the INSPIRE data themes to which
each topic category is applicable, i.e., oceanography is the INSPIRE theme for
which the Geoscientific information topic category is applicable.

###  Keyword

Regulation 1205/2008/EC requires that, for a spatial dataset or a spatial
dataset series, “at least one keyword shall be provided from the General
Environmental Multi-lingual Thesaurus (GEMET) describing the relevant spatial
data theme, as defined in Annex I, II or III to Directive 2007/2/EC”. Keywords
should be taken from the GEMET – General Multilingual Environmental Thesaurus
where possible.

#  ODYSSEA datasets

This section describes the structure and the content of the proposed ODYSSEA
metadata profile on the dataset-level and includes general guidelines for the
metadata from two points of view – the first one is the ODYSSEA metadata,
while the second represents ODYSSEA data quality issues.

The structure described in this document is compliant to the existing ISO
standards for metadata – i.e., especially ISO EN 19115 and ISO 19139\. The
full list of used ISO standards can be found in the List of References at the
end of this document. The primary goal of this deliverable is to develop a
metadata profile for ODYSSEA geographic datasets and time-series datasets,
within the framework of these ISO standards, to support the interoperability
between the different metadata and/or GIS platforms.

The metadata model to be adopted in ODYSSEA is described in more detail in
Annex I.

##  Dataset-level metadata

Metadata can be reported for each individual spatial object (spatial object-
level metadata) or once for a complete dataset or dataset series (dataset-
level metadata). If data quality elements are used at spatial object level,
the documentation shall refer to the appropriate definition in the Data
Quality Info section of this document. This section only specifies the
dataset-level metadata elements.

For some dataset-level metadata elements, in particular on data quality and
maintenance, a more specific scope can be specified. This allows the
definition of metadata at sub-dataset level, e.g., separately for each spatial
object type. When using ISO 19115/19139 to encode the metadata, the following
rules should be followed:

  * The scope element (of type DQ_Scope) of the DQ_DataQuality subtype should be used to encode the scope. 

  

  * Only the following values should be used for the level element of DQ_Scope: series, dataset, featureType. 

  

  * If the level is featureType  3  then the levelDescription/MD_ScopeDescription/features element (of type Set <GF_FeatureType>) shall be used to list the feature type names. 

  

Mandatory or conditional metadata elements are specified in the next sub-
section, while optional metadata elements are specified in subsequent sub-
Section. The tables describing the metadata elements contain the following
information:

  * The first column provides a reference to a more detailed description. The second column specifies the name of the metadata element. 

  

  * The third column specifies the multiplicity. 

  

  * The fourth column specifies the condition, under which the given element becomes mandatory (only for the first and second tables). 

  

In **Annex I** a detailed description of the metadata is presented.

##  Service-level metadata

This section describes the structure and the content of the proposed ODYSSEA
metadata profile on the service-level and includes general guidelines for
ODYSSEA metadata from two points of view – the first one is the ODYSSEA-
specific metadata, while the second represents quality issues of the data
published by the services.

The structure described in this document is compliant to the existing ISO
standards for metadata – i.e., especially ISO EN 19115, EN ISO 19119 and ISO
19139 (the full list of used ISO standards can be found in List of References
at the end of this document. The primary goal of this deliverable is to
develop a metadata profile for ODYSSEA geographical data services, within the
framework of these ISO standards, to support interoperability between
instances of discovery services and different metadata and/or GIS platforms as
well.

Metadata can be reported for each individual spatial object (spatial object-
level metadata) or once for a complete dataset or dataset series (dataset-
level metadata). On the other hand, metadata can also be reported for the
services that are publishing ODYSSEA data – i.e., especially INSPIRE view and
download services. This section only specifies service-level metadata
elements.

For some service-level metadata elements, in particular for data quality, a
more specific scope can be specified. This allows the definition of metadata
at sub-dataset level, e.g., separately for each spatial object type. When
using ISO 19115/19139 to encode the metadata, the following rules should be
followed:

  * The scope element (of type DQ_Scope) of the DQ_DataQuality subtype should be used to encode the scope. 

  

  * Only the following value should be used for the level element of DQ_Scope: service. 

  

Mandatory or conditional metadata elements are specified in the ANNEX I.
Optional metadata elements are specified in the subsequent sub-section of this
ANNEX.

##  Data format standards

###  Ocean Data View data model and netCDF Format

As part of the ODYSSEA services, data sets will be accessible via download
services. Delivery of data to users requires common data transport formats,
which interact with other standards (  Vocabularies,  data  quality control).
In SeaDataNet it was decided that Ocean Data View (ODV) and NetCDF format are
mandatory.

ODYSSEA will follow the SeaDataNet (2017) procedures, as main concepts of this
document are reproduced in the following paragraphs. ODYSSEA will also follow
the fundamental data model underlying ODV format which, in practice, is
composed by a collection of rows, each having the same fixed number of
columns. In this model there are three different types of columns:

  * _The metadata columns;_

  

  * _The primary variable data columns (one column for the value plus one for the qualifying flag);_

  

  * _The data columns._

  

The metadata columns are stored at the left-hand end of each row, followed by
the primary variable columns and then the data columns. There are three
different types of rows:

  * _The comment rows;_

  

  * _The column header rows;_

  

  * _The data rows._

  

The CF metadata conventions (http://cf-pcmdi.llnl.gov/) are designed to
promote the processing and sharing of files created with the NetCDF API. The
conventions define metadata that provide a definitive description of what the
data in each variable represents, and the spatial and temporal properties of
the data. This enables users of data from different sources to decide which
quantities are comparable, and facilitates building applications with powerful
extraction, re-gridding, and display capabilities.

The standard is both mature and well-supported by formal governance for its
further development. The standard is fully documented by a PDF manual
accessible from a link from the CF metadata homepage (http://cf-
pcmdi.llnl.gov/). Note that CF is a developing standard and consequently
access via the homepage rather than through a direct URL to the document is
recommended to ensure that the latest version is obtained. The current version
of this document was prepared using version 1.6 of the conventions dated 5
December 2011.

The approach taken with the development of the SeaDataNet profile based on CF
1.6 was to classify data on the basis of feature types and produce a
SeaDataNet specification for storage of each of the following:

  * _**Point time series** , such as current meter or sea level data, have row_groups made up of measurements from a given instrument at different times. The metadata date and time are set to the time when the first measurement was made. The primary variable is time (UT) encoded either as: _

    * _A real number representing the Chronological Julian Date, which is defined as the time elapsed in days from 00:00 on January 1st 4713 BC. If this option is chosen, then the column must have the heading ‘Chronological Julian Date [days]’._

  

    * _A string containing the UT date and time to sub-second precision corresponding to ISO8601 syntax (YYYY-MM-DDThh:mm:ss.sss) for example 2009-02-12T11:21:10.325. If this option is chosen, the column must have the heading ‘time_ISO8601’. If the time is not known to sub-second precision, then use the ISO8601 form appropriate to the known precision. For example, a timestamp to the precision of one hour would be represented by 2009-0212T11:00 and a time stamp to a precision of a day by 2009-02-12._

  

  

_Rows within the row_group are ordered by increasing time. Note that the z co-
ordinate (e.g., instrument depth), essential for many types of time series
data, needs to be stored as a data variable and could have the same value
throughout the row_group._

  * _**Profile data** , such as CTD or bottle data, have row_groups made up of measurements at different depths. The metadata date and time are set to the time when the profile measurement started. The primary variable is the ‘z co-ordinate’, which for SeaDataNet is either depth in metres or pressure in decibars. Rows within the row_group are ordered by increasing depth. _

  

  * _**Trajectories** , such as underway data, have row_groups made up of a single measurement, making the metadata time and positions the spatio-temporal co-ordinate channels. The primary variable is the ‘z co-ordinate’, which for SeaDataNet is standardised as depth in metres. Rows within the row_group are ordered by increasing time; _

  

  * _**TimeSeriesProfile** (x, y, z fixed; t variable) but some variables can be measured at different depths at the same time var=f(t, z). The specification given is for storage of time series profiles such as moored ADCP. _

  

  * _**TrajectoryProfile** (x, y, z, t all variable) but some variables can be measured at different depths at the same time var=f(t, z). The specification given is for storage of trajectory profiles such as shipborne ADCP. _

  

The specification was then developed through discussions on a collaborative
e-mail list involving participants in SeaDataNet, MyOcean, USNODC, NCAR and
AODN. The working objective focussed on producing profiles with the following
properties:

  * _CF 1.6 conformant;_

  

  * _Have maximum interoperability with CF 1.6 implementations in use by MyOcean (OceanSITES conventions), USNODC (USNODC NetCDF templates) and two contributors to AODN (IMOS and METOC);_

  

  * _Include storage for all labels, metadata and standardised semantic mark-up that were included in the SeaDataNet ODV format files for the equivalent feature type._

  

Significant list discussion focussed on the version of NetCDF that should be
used for SeaDataNet. The conclusion was that NetCDF 4 should be used wherever
possible, but that NetCDF 3, although strongly discouraged, should not be
totally forbidden.

On ANNEX II some examples of the structure of these files are presented.

###  Static data (Bathymetric, Chemical, Geologic, Geophysical, Biological,
Biodiversity data)

ODYSSEA will also adopt the SeaDataNet proposed standards for marine chemistry
(to support the EMODNet Chemistry pilot), bathymetry (to support the EMODNet
Hydrography and Seabed Mapping pilots), and geology and geophysics (to support
the Geo-Seas project and the EMODNet Geology pilot). and marine biology.

Based on an analysis of the present situation, and currently existing
biological data standards and initiatives, such as the Ocean Biogeographic
Information System (OBIS), Global Biodiversity Information Facility (GBIF),
Working Group on Biodiversity Standards (TDWG) and World Register of Marine
Species (WoRMS) standards, SeaDataNet proposed a format for data exchange of
biological data.

Key issues that steered the format development were (SeaDataNet III,
publishable summary):

  * _Requirements posed by the intended use and application of the data format (data flows, density calculations, biodiversity index calculations, community analysis, etc…)_

  

  * _Availability of suitable vocabularies (World Register of Marine Species, SeaDataNet Parameter list, SeaDataNet Unit list, etc…)_

  

  * _Requirements for compatibility with existing tools and software (WoRMS taxon match services,_

  

_EurOBIS QC services, Lifewatch workflows, Ocean Data View, etc…)_

The requirements of the extended ODV format for biological data were defined
as follows:

  * _The format should be a general and higher level format without necessarily containing all specifics of each data type, but rather focusing on common information elements for marine biological data._

  

  * _At the same time the format needs to be sufficiently flexible/extendable to be applicable for at least part of the variety of biological data the NODC’s are managing._

  

  * _It should be possible to derive OBIS or Darwin Core compatible datasets from the format._

  

  * _The format should be self-describing, in the sense that all information needed to interpret the data should be included in the file format or be available through links to vocabularies or term lists that are part of the format._

  

A specific ODV extended format for biological data has been defined for
different types of files such as (see for details SeaDataNet deliverable
D8.4):

  * _macrobenthos community with density and biomass values;_

  

  * _zooplankton community with samples from different depths;_

  

  * _demersal fish population with densities for different size classes and individual fish measurements;_

  

  * _pollutant concentrations in biota specimens._

  

###  Open source semantic information

Semantic information may be useful for a myriad of services to the end users.
However, the sources providing semantically rich information are very
heterogeneous. Semantically rich information can be found on Wikipedia and
Wikidata for instance. EMODnet, through the “Human activities” data sets also
provides some semantically rich information.

As one can see the sources of semantically rich information are very
heterogeneous in the availability, reliability and format. Furthermore, they
provide heterogeneous and partially redundant information. No standard model
exists in order for that type of information, as their variability is very
high. However, as one of ODYSSEA platform aim is to integrate and fuse this
kind of information, one must rely on a shared format in order to analyze and
make use of it.

Within the services that will be developed in ODYSSEA, a domain ontology will
be used in order to enable the integration of semantic information sources.
For each ODYSSEA use case, and for each ODYSSEA product relying on semantic
information analysis and integration, end users of the products will have to
develop, together with ODYSSEA technical partners, an ontology defining the
concepts of interest of the use case. This ontology will be the pivot language
and representation format used to integrate heterogeneous open information
sources.

_Figure 1: example of an ontology defining the main concepts used to analyze
the impact of port structures on the quality of bathing waters and fish
production_

#  Data privacy policy

##  General principles

Basic principles regulated by the data protection Act will be observed namely:

  * ODYSSEA only hold the necessary personal data to offer services provided by its platform. 

  

  * Data is only used for the purposes described in the Data Protection Register Form and the Informed Consent Form. 

  

  * Personal data will only be hold for as long as necessary. Once data are no longer needed it will be deleted from ODYSSEA records by the ODYSSEA platform Administrator (namely the CLS Chief Technical Officer (CTO) / IT platform manager). More specifically, in case a certain period (one year) is passed without the entry of an end-user in the platform, CLS will alert him through a standardized electronic message on the destruction of personal data. 

  

  * Personal data storage will be secured to ensure that data are not accessible to unwanted third parties and are protected against disaster and risk. 

  

  * ODYSSEA will regularly email website news and information updates only to those end-users and customers who have specifically subscribed to our email service. All subscription emails sent by the ODYSSEA platform will contain clear information on how to unsubscribe from our email service. 

  

  * In any event, no personal data will be shared with any third party for direct marketing. ODYSSEA will never sell, rent or exchange mailing lists of personal data. 

  

  * All ODYSSEA partners shall comply with the data protection and privacy laws applicable in their country of origin, including their national laws applicable to exporting data into the EU. 

  

  * ODYSSEA partners from non-EU countries have provided signed declarations that they will meet all relevant H2020 ethical standards and regulations. _Exporting personal data from the EU to non-_

  

_EU countries must comply with the applicable EU rules on cross-border
transfer of personal data._

  * In accordance with the Privacy and Electronic Communications (EC Directive) Regulations 2003, 

  

ODYSSEA never send bulk unsolicited emails, (popularly known as Spams) to any
email addresses.

  * ODYSSEA may send emails to existing end-users and customers or prospective end-users and customers having enquired or registered in ODYSSEA platform, regarding products or services directly provided by ODYSSEA platform. 

  

  * All emails sent by ODYSSEA will be clearly marked as originating from this platform. All such emails will also include clear instructions on how to unsubscribe from ODYSSEA email services. Such instructions will either include a link to a page to unsubscribe or a valid email address to which the user should reply, with “unsubscribe” as the email subject heading. 

  

##  Use of Cookies

Cookies are small text files which are placed on your computer by websites
that you visit. They are widely used in order to make websites work, or work
more efficiently, as well as providing information to the owner of the site.

ODYSSEA’s platform may generate cookies in order to work more efficiently.
These will enhance features such as platform search and optimized page
loading.

ODYSSEA may use Google Analytics to collect quantitative information on
platform’s performance and end-users’ interaction with the platform. ODYSSEA
will use this information to improve the service and experience offered by the
platform. The use of Social Media buttons on some of the pages link to third
party websites and services, like Facebook and Twitter also create cookies.
These services use cookies when clicking the button. Privacy policies will be
available for all of these services, and users should be able to read them to
be informed on how their information is being used, and how they can opt-out,
should they wish to.