#  Introduction

##  Scope of D7.4

The purpose of “D7.4 - Data Management plan” is to provide an overview of the
main elements of the data management policy that will be used by the with
regard to all datasets that will be generated by the project. It also
describes the access granted to all parties interested in the data generated
by the RAWFIE system during its development, tests and operations. Finally it
discusses the compliance of the RAWFIE data structure, management and policy
with respect to EU regulations and directives. This deliverable will be
evolved and updated during the lifespan of the project.

Figure 1 presents the steps and actions involved in a typical data management
cycle.

•

Dissemination

•

Preservation

•

Storage

•

Data Processing

•

Models

•

Analytics

•

Data set

•

Data streams

Collect

Process

Share

Archive

This document is structured as follows: Section 2 describes the data types and
metadata that will be collected and processed during the experimentation and
the project lifetime as well as the respective standards and formats. Section
3 contains the data access procedure and the dissemination mechanisms that
will take place to provide reusability and access in the future.

Finally, Section 4 describes the procedures for the archiving and long-term
storage.

##  Abbreviations  
  
<table>  
<tr>  
<th>

**Abbreviation**

</th>  
<th>

**Meaning**

</th> </tr>  
<tr>  
<td>

DMP

</td>  
<td>

Data Management Plan

</td> </tr>  
<tr>  
<td>

SOS

</td>  
<td>

Sensor Observation Service

</td> </tr>  
<tr>  
<td>

OGC

</td>  
<td>

Open Geospatial Consortium

</td> </tr>  
<tr>  
<td>

GML

</td>  
<td>

Geography Markup Language

</td> </tr>  
<tr>  
<td>

KML

</td>  
<td>

KML - formerly Keyhole Markup Language

</td> </tr>  
<tr>  
<td>

WMS

</td>  
<td>

Web Map Service

</td> </tr>  
<tr>  
<td>

WMTS

</td>  
<td>

Web Map Tile Service

</td> </tr>  
<tr>  
<td>

WFS

</td>  
<td>

Web Feature Service

</td> </tr>  
<tr>  
<td>

CSV

</td>  
<td>

Comma-separated values

</td> </tr>  
<tr>  
<td>

JSON

</td>  
<td>

JavaScript Object Notation

</td> </tr>  
<tr>  
<td>

GIS

</td>  
<td>

Geographic Information System

</td> </tr>  
<tr>  
<td>

PMML

</td>  
<td>

Predictive Model Markup Language

</td> </tr>  
<tr>  
<td>

EDL

</td>  
<td>

Experiment Description Language

</td> </tr>  
<tr>  
<td>

XML

</td>  
<td>

Extensible Markup Language

</td> </tr>  
<tr>  
<td>

SensorML

</td>  
<td>

Sensor Model Language

</td> </tr>  
<tr>  
<td>

O&M

</td>  
<td>

Observations and Measurements

</td> </tr>  
<tr>  
<td>

SPS

</td>  
<td>

Sensor Planning Service

</td> </tr>  
<tr>  
<td>

SWE

</td>  
<td>

Sensor Web Enablement

</td> </tr>  
<tr>  
<td>

ML

</td>  
<td>

Machine Learning

</td> </tr>  
<tr>  
<td>

DM

</td>  
<td>

Data Management

</td> </tr>  
<tr>  
<td>

IMC

</td>  
<td>

Inter Module Communication

</td> </tr>  
<tr>  
<td>

ROS

</td>  
<td>

Robot Operating System

</td> </tr>  
<tr>  
<td>

LLF

</td>  
<td>

LSTS Log Format

</td> </tr>  
<tr>  
<td>

ISO

</td>  
<td>

International Organization for Standardization

</td> </tr> </table>

**Table 1: Abbreviations**

#  Data set reference, standards and metadata

This section will describe identifiers for the data set to be produced. It
will include:

  * description of the data that will be generated or collected 

  

  * its origin (in case it is collected) 

  

  * nature and scale, and to whom it could be useful 

  

  * whether it underpins a scientific publication 

  

  * information on the existence (or not) of similar data and the possibilities for integration and reuse. 

  

The following sections include the potential data types to be generated by the
project and reference to existing suitable standards of the discipline.

##  Raw data and sensor observations

Raw sensor data will be collected by on-board mobile sensors located in the
UxV devices. These data has not been subjected to processing or any other
manipulation. The types of raw data will be related to the different sensor
types that will be used in the context of the RAWFIE project. We can classify
the sensors and the relevant data into the following categories:

  * Environmental sensors (temperature, thermal, heat, moisture, humidity) 

  

  * Position, angle, displacement, distance, speed, acceleration 

  

  * Air Pressure 

  

  * Proximity (able to detect the presence of nearby objects) 

  

  * Navigation instruments 

  

All these sensors form the basis for calculating the experiment context. The
context and the metadata that could be retrieved are based on the relevant
format and standard. The basic information that will be generated by the
RAWFIE sensors is specified as follows in Table 2:  
  
<table>  
<tr>  
<th>

**Data**

</th>  
<th>

**Data type**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Identifier (ID)

</td>  
<td>

String

</td>  
<td>

Unique identification of the sensor

</td> </tr>  
<tr>  
<td>

Owner

</td>  
<td>

String

</td>  
<td>

Any text that describes the

owner/manufacturer of the sensor

</td> </tr>  
<tr>  
<td>

Sensor type

</td>  
<td>

String

</td>  
<td>

Description of the sensor type

</td> </tr>  
<tr>  
<td>

Observed area

</td>  
<td>

Coordinates (latitude, longitude, elevation)

</td>  
<td>

The geographical area within which the associated observations were made

</td> </tr>  
<tr>  
<td>

Phenomenon

</td>  
<td>

String

</td>  
<td>

The phenomenon description (e.g., temperature)

</td> </tr>  
<tr>  
<td>

Observed result

</td>  
<td>

Integer or String

</td>  
<td>

The observed value that is related to the phenomenon

</td> </tr>  
<tr>  
<td>

Unit of measurement

</td>  
<td>

String or Character

</td>  
<td>

The unit of measured phenomenon (e.g., degree Celsius °C)

</td> </tr>  
<tr>  
<td>

Date and time

</td>  
<td>

Timestamp

</td>  
<td>

Sequence of characters specifying when the observation took place

</td> </tr>  
<tr>  
<td>

Offering ID

</td>  
<td>

String

</td>  
<td>

Text description that can be used for group purposes

</td> </tr> </table>

**Table 2: Sensor observation data and metadata**

There are already some similar initiatives and EU projects that have generated
and collected sensor observations, but for different purposes and
applications. For instance Fed4FIRE project has collected such data (e.g.,
environmental data) from different experiment executions. These data are
strictly related to the experimentation scenarios that could be a combination
of different data types (e.g., sensor data and geospatial data) therefore
there is no possibility to integrate the existing sensor observation data for
RAWFIE purposes. The project will collect and generate such data from its own
experiment executions.

####  Standards

There are already some standards and formats to encapsulate and manage the
information from sensor observations and raw data. The sensor observation
standard that could be adopted by RAWFIE is yet to be decided. However, some
existing standards could be:

  * Sensor Observation Service (SOS) [1] 

  

The SOS is an interface provided by the OGC consortium in order to allow
access and distribution for sensor description and observation. The
specification leverages the Observations and Measurements (O&M) specification
to encode observations and the Sensor Model Language (SensorML) specification
to encode sensor descriptions. Both of these formats are based on

Extensible Markup Language (XML). This standard defines a Web-based interface
(Web Service) that allows querying observations, sensor metadata or
representations of observed features. Further, it provides means to register
new sensors or remove existing ones. It also defines operations to insert new
sensor observations. The SOS operations follow the general pattern of other
OGC Web Services and inherit or re-use, when needed, elements defined
previously.

  * Sensor Model Language (SensorML) [2] 

  

SensorML provides models and XML encodings for describing any process related
to sensor system. Processes, described in SensorML, define their inputs,
outputs, parameters, method and they also provide relevant metadata. This
standard includes sensors and actuators as well as computational processes
applied pre- and post-measurement. The main objective is to enable
interoperability, first at the syntactic level and later at the semantic level
(by using ontologies and semantic mediation), so that sensors and processes
can be better understood by machines, utilised automatically in complex
workflows, and easily shared between intelligent sensor web nodes. This
standard is one of several implementation standards produced under OGC’s
Sensor Web Enablement (SWE) activity. In RAWFIE, SensorML can be used to
describe different types of sensors (e.g. environmental, air pressure).

  * Observations and Measurements (O&M) [3] 

  

O&M defines a conceptual schema encoding for observations and features
involved in sampling when making observations. O&M provides document models
for the exchange of information describing observation acts and their results,
both within and between different scientific and technical communities. This
encoding is an essential dependency for the OGC Sensor Observation Service
(SOS) Interface Standard. This standard can be leveraged by RAWFIE to describe
the measurements derived by different sensor types.

  * Sensor Planning Service (SPS) [4] 

  

The SPS is an interface standard defining interfaces for queries that provide
information about the capabilities of a sensor and how to task the sensor. The
standard is designed to support queries that have the following purposes: a)
to determine the feasibility of a sensor planning request, b) to submit and
reserve/commit such a request, c) to inquire about the status of such a
request, d) to update or cancel such a request; and e) to request information
about other OGC Web services that provide access to the data collected by the
requested task.

##  Processed data, models and analytics

Processed data refer to the models and the statistics that will be generated
from the stream analytics platform. Typical models include classification and
outlier detection models. The data mining and machine learning community
traditionally relies on the exchange and publication of datasets. This is
achieved by a number of relevant data repositories which will be described in
a later section, and much less through models. Nevertheless there is a
standard, PMML [5] which allows the description and sharing of learned models
between different analytical environments. The publicly available datasets are
used to compare and test different learning algorithms and it is one of the
means that the community has used to ensure the replicability of the
scientific results and the fair comparison of different learning methods.

As mentioned in the previous paragraph the learning and mining community has
focused on the exchange of data and not on that of models. Once the raw data
for a given learning tasks are available, different teams can test their own
approaches and algorithms on them. Probably the most well-known repository for
datasets used by data mining and machine learning teams is the UCI machine
learning repository [6]. We will discuss in more detail the availability and
use of existing repositories in a following section on the dissemination of
the data that will be generated by the project. Within the UCI repository one
may find a number of datasets similar in nature to the data that will be
generated within RAWFIE. These are mainly time-series datasets from different
application domains such as finance, social media, physical activity sensors,
chemical sensors and more. Nevertheless these datasets are not directly
relevant for the RAWFIE project. Some of them might be used to provide
additional testing datasets for the learning and mining

algorithms that will be developed in RAWFIE.

####  Standards

Here we will mention standards that can be used for describing the results of
the data analytical process. We will consider the use of PMML [7], Predictive
Model Markup Language to describe the generated models, provided that the ones
that we will generate are covered by the current PMML version (v 4.2) [5].
Briefly PMML is an industrial standard that is used for the exchange of
machine learning and data mining models between different applications and
data analytical environments. It is based on XML and offers support for models
generated as a result of different data mining tasks such as association rule
discovery, classification, regression and clustering. A description of a data
mining model in PMML contains the following elements:

  * a header which provides general information about the model such as the analytical environment that generated the model, generation timestamps etc 

  

  * a data dictionary describing the dataset from which the model was generated 

  

  * a data transformation component describing transformations that are applied to the data prior to modeling, such as normalization, discretization 

  

  * the model component describing the learned model. 

  

##  Geospatial data

Geospatial data appears in various formats and relations in the RAWFIE system.
Sometimes the data itself has a spatial aspect, sometimes it is just metadata
(i.e. descriptive data belonging to the original data). Basically geometry
data can be distinguished into vector and raster data.

Vector data means that entities consist of one or more coordinates that form a
geometric primitive (geometry type). The commonly supported geometric
primitives are points, multipoints, lines, multi-lines, polygons and multi-
polygons. The difference between the simple and the multi geometries is that
one multi geometry consists of one or many simple geometries of the same type.
There are also geometry collections consisting of geometries of different
types, but they are not very commonly used. Apart from that specific vector
formats may also support ellipses, splines, etc.

Raster data means that an entity is a picture (of one or more sub entities) in
conjunction with a definition of a projection into a specific coordinate
system specifying the exact extent of this picture in terms of the coordinate
reference system. Using this definition, the picture could be rendered on the
right place and shaped in a map using this reference system.

The following list of geospatial information, gives an overview of the types
of data with a spatial reference that will be possibly generated and / or
collected inside RAWFIE.  
  
<table>  
<tr>  
<th>

**Data**

</th>  
<th>

**Data type**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

UxV location

</td>  
<td>

Point

</td>  
<td>

The location of an UxV during an experiment.

Used in the Visualisation Engine

</td> </tr> </table>  
<table>  
<tr>  
<th>

UxV course

</th>  
<th>

Line

</th>  
<th>

The current course an UxV is taking, i.e. an extrapolation of the current
position together and its direction to know where the UxV will probably be in
the next seconds or minutes.

</th> </tr>  
<tr>  
<td>

Waypoints

</td>  
<td>

Point

</td>  
<td>

An ordered list of waypoints for UxV navigation / predefined routes. They can
have absolute coordinates or relative ones in respect to the current position
(e.g. ‘move 30 meters in the direction of 45°’).

Used for experiment authoring and in the resource controller during execution.

</td> </tr>  
<tr>  
<td>

Geo-fence

</td>  
<td>

Polygon

</td>  
<td>

Regions where an event or alarm should be triggered when an UxV enters or
leaves.

Used in experiment authoring (EDL)

</td> </tr>  
<tr>  
<td>

Sensor measurement location

</td>  
<td>

Point

</td>  
<td>

Location where a sensor measurement has been recorded.

It is metadata for sensor data types, see also section **Error! eference
source not found.**

</td> </tr>  
<tr>  
<td>

Detected object

</td>  
<td>

any

</td>  
<td>

An object detected by sensors or evaluation of sensor values.

The type of object highly depends on task which should be performed by UxV,
e.g.:

  * border surveillance: intruders / potential threats 

  

  * firefighting: trees, fire or empty space which would form a natural block to the spreading fire 

  

  * monitoring of water canals: cracks in canal’s wall structure 

  

The position or geo-referenced outline of the object is geospatial meta data
of the experiment results

</td> </tr>  
<tr>  
<td>

Testbed position or area

</td>  
<td>

Point

Polygon

</td>  
<td>

The fixed location of the testbed (meta data). In the simple case it is just a
coordinate, in the more precise case it is the area of the testbed.

Used in experiment authoring (EDL) and resource exploring

</td> </tr>  
<tr>  
<td>

Testbed surroundings

</td>  
<td>

any

</td>  
<td>

The surroundings of a testbed. These could influence the experiments.

Potential objects could be:

  * barriers (buildings, trees etc.) 

  

  * streets 

  

  * water ways 

  

  * water surface 

  

</td> </tr>  
<tr>  
<td>

</td>  
<td>

</td>  
<td>

\- digital elevation model (above and under water)

Used in experiment authoring (EDL) and resource exploring as well as for
validation of experiments in their aftermath.

</td> </tr> </table>

**Table 3: Geospatial data overview**

During the implementation of the project new kinds of geospatial data will
possibly be examined as new services will be integrated that have not been
foreseen in the forefront of the project. When this happens, this list will be
updated to reflect the new data types.

####  Standards

Geospatial data is stored and processed in various, quite diverse, formats.

Internally a common representation of the geospatial data will be used, which
really simplifies the data handling. This representation is yet to be decided.
However, imported data comes in any of the mentioned formats, or even another
different one.

A list of common formats and standard is in the table blow. Many of the
standards are from the OGC [8].  
  
<table>  
<tr>  
<th>

**Format**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Shapefile [9]

</td>  
<td>

  * de factor standard (designed by ESRI) to store vector data 

  

  * supported by almost all GIS systems 

  

  * only one geometry type per Shapefile 

  

  * consists of multiple files 

  

  * attribute data stored in dBASE (version IV) database (.dbf file) [10] 

  

</td> </tr>  
<tr>  
<td>

GeoPackage [11]

</td>  
<td>

  * recently developed OGC standard to store all kinds of geospatial related data (vector features, tile matrix sets of imagery and raster maps at various scales , schema, metadata) 

  

  * database file that can be accessed and updated directly without intermediate format translations 

  

  * can be seen as a modern replacement for shapefiles with the following advantages: o only one file instead of multiple files o smaller file sizes 

    * wider spectrum of attribute types 

  

    * less constraints (e.g. length of attribute names) 

  

  

</td> </tr>  
<tr>  
<td>

GML [12]

</td>  
<td>

  * _Geography Markup Language_

  

  * OGC standard to exchange vector data via XML files 

  

  * very flexible and adaptable to individual needs 

  

  * used in many open source systems 

  

</td> </tr>  
<tr>  
<td>

KML [13]

</td>  
<td>

  * _Keyhole Markup Language_

  

  * OGC standard to exchange vector data via XML files 

  

  * mainly used by Google Earth 

  

</td> </tr>  
<tr>  
<td>

WMS [14]

</td>  
<td>

\- _Web Map Service_

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

OGC standard protocol for serving geo-referenced map images (raster data)

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

images are generally generated by a map server (most using data from a GIS
database)

</td> </tr>  
<tr>  
<td>

WMTS [18]

</td>  
<td>

\-

</td>  
<td>

_Web Map Tile Service_

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

OGC standard protocol for serving geo-referenced raster data

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

very similar to WMS, but with much simpler request interfaces

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

the raster data provided is normally pre-calculated and hence the server-side
computing time is very low, making WMTS a very fast and responsive service

</td> </tr>  
<tr>  
<td>

WFS [15]

</td>  
<td>

\-

</td>  
<td>

_Web Feature Service_

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

OGC standard protocol which provides an interface for geographical feature
requests (vector data)

</td> </tr>  
<tr>  
<td>

World-File [16]

</td>  
<td>

\-

\-

</td>  
<td>

de factor standard (designed by ESRI) to store raster data supported by almost
all GIS systems

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

a text file (in conjunction with an picture file) that describes the
projection of a picture into a specific coordinate system

</td> </tr>  
<tr>  
<td>

GeoTiff [17]

</td>  
<td>

\-

</td>  
<td>

public domain metadata standard

</td> </tr>  
<tr>  
<td>

</td>  
<td>

\-

</td>  
<td>

allows geo-location information to be embedded within a Tiff image file

</td> </tr> </table>

**Table 4: Geospatial data formats and standards**

Also many other formats exist (structured text files, e.g. formatted as CSV,
JSON (GeoJSON) or XML as well as many proprietary binary formats) that are
used to store geospatial data.

##  Image data from UxV platforms

###  Robotnik platforms

Robotnik’s platforms make use of ROS (Robot Operating System) standard
formatting. ROS image format message are  _sensor_msgs_ _and images_ _._

There are three standard data types to get video images in ROS:

o "raw": The default transport o "compressed": JPEG or PNG image compression o
"theora": Streaming video using the Theora codec.

Normally, every camera driver is able to publish all of these formats,
_image_transport_ ROS package is the recommended tool to do it. Robotnik
platforms are usually integrating camera sensors such as Microsoft Kinects or
Asus XTION and AXIS PTZ Cameras. The last ones also provide its own RTSP Web
Server streaming MPEG-4.

The ROS package sensor_msgs defines messages for commonly used sensors,
including cameras and scanning laser rangefinders. A lot of data can be found
under ROS as many other applications and programs make use of such data. For
an example on how to work with specialized ROS messages please refer to
MathWorks [19].

As stated with another critical components/software of these platforms, the
communication with non-ROS systems shall be done by using tools such as the
Rosbridge server (see D4.1) which involve JSON libraries. In addition,
regarding Image data exchange, a better way to get the video stream is by
using the package  _mjpeg_server_ that connects to the ROS topic and publishes
the video stream via MJPEG server. Some other tools like _cv_bridge (_
http://wiki.ros.org/cv_bridge) can be used to interface ROS and OpenCV by
converting ROS images into OpenCV images, and vice versa.

###  MST platforms

Images and related data collected by MST vehicles are provided to the user as
is, which means that the data format is a characteristic of the sensor/camera
being used. As of the date of this writing the MST vehicles use two different
types of cameras. The first type is an industrial camera that records video as
individual JPEG images, the user may configure the desired frame rate which
can be between 4 and 15 frames per second. Each JPEG image contains embedded
navigation data (position and pose) encoded using the Exif format and the
maximum available image resolution is 1376x1032 pixels. This type of camera is
usually used to take georeferenced pictures of the seabed under low-light
conditions occasionally with the aid of an external synchronized illumination
module.

The second type of camera is capable of recording and streaming video encoded
with MPEG-4 (H.264) at a frame rate of 30 frames per second and a resolution
of 1280x720 pixels. This camera is normally used without the aid of artificial
illumination and the video collected is commonly streamed in real time to a
control center for surveillance purposes.

####  Standards

Both types of cameras used by MST comply with the following industry
standards:

  * ISO/IEC 10918-1:1994, Information technology - Digital compression and coding of continuous-tone still images: Requirements and guidelines. 

  

  * CIPA DC-008-2012, Exchangeable image file format for digital still cameras: Exif Version 2.3 

  

  * ISO/IEC 14496 MPEG-4 Standard Parts 1 to 31 

  

##  Simulated data

This section contains the simulated data and formats that Robotnik’s and MST’s
platforms uses during the experiment simulations, which possibly will be
leveraged by the RAWFIE project.

###  Robotnik platforms

Robotnik platforms simulations are normally being carried by Gazebo Simulator
[20]. Although it is not the only simulating software compatible with ROS, it
is the most commonly used tool by ROS users because of its good integration
with the platform. The main reason is that Gazebo is capable of simulating a
wide range of testbeds, from a mapped indoor office, to outdoor environments
with only a map and the model of the robot.

To summarize, this model structures is as follows:

 _Database_

  * _database.config_ : Meta data about the database. This is now populated automatically from CMakeLists.txt 

  

  * _model_1_ : A directory for model_1 

    * _model.config_ : Meta-data about model_1 

  

    * _model.sdf_ : SDF description of the model 

  

    * _meshes_ : A directory for all COLLADA and STL files 

  

    * _materials_ : A directory which should only contain the textures and scripts subdirectories 

  

    * _textures_ : A directory for image files (jpg, png, etc). 

  

    * _scripts_ : A directory for OGRE material scripts 

  

    * _plugins_ : A directory for plugin source and header files 

  

  

Robotnik has developed these models for its platforms and there's no need of
rebuilding them. The key of these simulations lay on the information that
Gazebo is processing. In order to carry these simulations, similar nodes
(processes) to the ones that control the real robot are launched.

The real difference relies on who is subscribing and publishing the data. For
instance, Gazebo generates data such as raw images or joint movements for the
robot to process, while translates movement commands generated by the robot to
move the model in the simulator.

####  Standards

All the simulated data has the standard ROS format [21] that defines messages
for commonly used sensors, including cameras and scanning laser rangefinders.
An example of this format is shown below.

  * Maintainer status: maintained 

  

  * Maintainer: Tully Foote <tfoote AT osrfoundation DOT org>

  

  * Author: 

  

  * License: BSD 

  

In addition Robotnik simulations follows geometry_msgs [22] format that
provides messages for common geometric primitives such as points, vectors, and
poses. These primitives are designed to provide a common data type and
facilitate interoperability throughout the system. An example of geometry_msqs
is shown below.

  * Maintainer status: maintained 

  

  * Maintainer: Tully Foote <tfoote AT osrfoundation DOT org>

  

  * Author: Tully Foote 

  

  * License: BSD 

  

###  MST platforms

MST's simulated and real vehicles use the same data format for inter-module
communication and for data storage to persistent media. This allows for
simulated vehicles to use and replay data from real missions (e.g.,
environmental data, bathymetry). In doing so, while the vehicle's kinematics
are simulated using the vehicle's model and physical characteristics,
environmental sensor data can be simulated or taken from values collected in
the field.

####  Standards

The data format in question is called IMC (Inter Module Communication) and
comprises different logical message groups for networked vehicle and sensor
operations. It defines an infrastructure that is modular and provides
different layers for control and sensing. IMC defines the message entity as
having an associated uniquely identifying number and consisting of a (possibly
empty) sequence of data fields capable of representing fixed-width integers,
floating point numbers, variable length byte sequences and inline messages
(messages within messages). Integers can be signed or unsigned with sizes
ranging from 8 to 64 bits. Floating point numbers have two sizes: 32 and 64
bits. Messages are prefixed with a header and suffixed with a footer to form a
packet. Header and footer entities are defined as non-empty sequences of data
fields and have the same structure for all packets.

In order to transmit a message or save it to persistent storage the message
has to be encapsulated in a packet and serialized. Serialization is performed
by translating the data fields of the packet entities (header, message and
footer) to a binary stream in the same order as they were defined. The first
field of the packet header is the synchronization number, used to mark the
beginning of a packet and to denote its protocol version. By inspecting the
synchronization number the recipient is able to deduce the byte order of the
remaining data fields and perform the necessary conversions for correct
interpretation. Using this approach, communication between nodes with the same
byte order incurs in no byte order conversion overhead and communication
between nodes with different byte orders only introduces the conversion
overhead when deserializing packets. The complete IMC protocol is defined in a
single eXtensible Markup Language (XML) document that, when changed, can be
verified against a XML Schema (XSD). In the XML definition, each message field
must have at least a name, one abbreviation (used for code generation) and a
type. Optionally units and range of permissible values can also be defined.

Having a XML document describing the protocol has proven to be very practical
for continuous development and testing. This happens because just after
agreeing upon a specific version of IMC, two nodes can use the XML document to
understand each other. Python and Java programs are used to automatically
generate the IMC protocol reference documentation and optimized
implementations exist in C++ and Java. Generating native code from the XML
document has provided not only flexibility but also the performance needed for
real-time execution in resource constrained computers.

In addition to the main serialization format described above, there are two
complementary serialization formats with specific intents. One is the LLF
(LSTS Log Format) format, a text format used for logging IMC, amenable to
direct human understanding and easier to parse directly by many standard
applications e.g. Matlab, Microsoft Excel, and custom mission review and
analysis software. In order to be possible to review data from past missions,
the LLF format had to be independent of the originating IMC protocol
description (since the message format can change over time). Our approach was
to define this tab-separated log format where, for each column (message
field), there is a header describing its data type, name and units to be used
when representing the data. Another format is the IMC-XML, which can be used
as a simplified serialization format itself for inter-module interoperability.
The main reason for this additional format is to enable the integration of
web-based components and web-enabled third-party sensors into large-scale data
dissemination applications.

#  Data sharing

This section describes the access procedures, the technical mechanisms for
dissemination and necessary software for enabling re-use.

##  Data access procedures

In RAWFIE, data access will be specified with regards to the project phases,
i.e. the project implementation period and the post-EC-funding phase. During
the project implementation the data access will be set by default as public
providing the possibility to the experimenters to set the access private
according to his/her preferences. In this case, some experiments' data will be
restricted and the results will not be shared to the public. The facilities
will be open to experimenters, beyond participation in the Open Calls for
Experiments, until the end of the project. Interested parties to conduct
experiments on the RAWFIE facilities can contact the project consortium at any
time to explore opportunities. After the project life-cycle, when the
project’s outcomes will be exploited for both commercial and public funded
cases, a different policy will be followed. The data access will be set by
default as private and only in specific cases as public according to the
experimenters' preferences.

The experimenters and other stakeholders will have the opportunity to access
the data through the project portal or the respective repository. The
identification and the type of the repository (i.e., institutional, standard
repository for the discipline) where the research data will be stored will be
defined in an upcoming iteration of this deliverable.

Finally, RAWFIE project results will be fully complied with the EU Regulation
and Directives.

More specifically, the data generated by the project will follow the following
Directives:

  * Data Protection Directive, Directive 95/46/EC 

  

  * Data Retention Directive, Directive 2006/24/EC 

  

  * INSPIRE Directive Infrastructure for Spatial Information in the European Community, , Directive 2007/2/EC 

  

  * Marine Strategy Framework Directive, Directive 2008/56/EC 

  

  * Water Framework Directive, Directive 2000/60/EC 

  

####  Possible integration into EC frameworks

The EC is supporting numerous initiatives, in particular for the consistent
representation of common concepts, such as the building and city modeling,
representation and parameters (IRCABC). RAWFIE may contribute to such
initiatives by using or extending existing models.

##  Mechanisms for dissemination and sharing research data

This section outlines of technical mechanisms for dissemination and necessary
software for enabling re-use.

###  Mechanisms for dissemination

A specific dissemination strategy will be developed in RAWFIE in parallel with
the implementation activities, with the aim to keep potentially interested
stakeholders informed about the availability, and the possibility to access
new research data. Research data include experiments’ results, as well as any
other kind of data described in Section 2, generated within the platform
during the execution of the experiments.

The strategy used for the dissemination of research data will include:

  * identification of the different type of stakeholders (users or groups of users) that will be the intended “recipients” of the dissemination 

  

  * identification of the most suitable tools or mechanisms to be used for the dissemination, according to the type of audience 

  

  * implementation / use of the abovementioned dissemination tools or mechanisms 

  

Possible stakeholders interested in the data generated by the project are:

  * Experimenters 

  

  * Universities and research institutes 

  

  * UxVs and, in general, technology manufactures (e.g. sensor or wireless communication solutions providers) 

  

  * Owners of institutional repositories (if any) 

  

Potential dissemination mechanisms, together with the description on how they
could be used, and the possible mapping with the different stakeholder groups
they could reach, are provided in  
  
<table>  
<tr>  
<th>

**Dissemination mechanism**

</th>  
<th>

**How it will be used**

</th>  
<th>

**Stakeholder type**

</th> </tr>  
<tr>  
<td>

Project website

</td>  
<td>

News will be published with information about executed experiments and data

availability

</td>  
<td>

  * Experimenters 

  

  * Universities and 

  

research institutes

  * UxVs and, in general, technology manufactures (e.g. sensor or wireless communication solutions providers) 

  

  * Owners of institutional repositories (if any) 

  

</td> </tr>  
<tr>  
<td>

Newsletter / email

</td>  
<td>

Periodic news sent to selected stakeholder groups, will be also used to share
information about the availability of relevant research data

</td>  
<td>

  * Experimenters 

  

  * Universities and 

  

research institutes

</td> </tr>  
<tr>  
<td>

Publications

</td>  
<td>

At certain points in time, results and statistics coming from the experiments
will be published in scientific papers, together with the information on how
to access them

</td>  
<td>







</td>  
<td>

Experimenters

Universities and

research institutes

UxVs and, in general, technology manufactures (e.g. sensor or wireless
communication solutions providers)

</td> </tr>  
<tr>  
<td>

</td>  
<td>

</td>  
<td>



</td>  
<td>

Owners of institutional repositories (if any)

</td> </tr>  
<tr>  
<td>

Public Web feeds

</td>  
<td>

Feeds will be created where interested stakeholders can subscribe in order to
receive notification about new

available research data

</td>  
<td>







</td>  
<td>

Experimenters

Universities and

research institutes UxVs and, in general, technology manufactures (e.g. sensor
or wireless communication solutions providers)

</td> </tr>  
<tr>  
<td>

</td>  
<td>

</td>  
<td>



</td>  
<td>

Owners of institutional repositories (if any)

</td> </tr>  
<tr>  
<td>

Social Media (e.g. Twitter, Facebook, Linkedin)

</td>  
<td>

Notification about new available research data will be regularly published
through the social media channels setup by the project

</td>  
<td>







</td>  
<td>

Experimenters

Universities and

research institutes UxVs and, in general, technology manufactures (e.g. sensor
or wireless communication solutions providers)

</td> </tr>  
<tr>  
<td>

</td>  
<td>

</td>  
<td>



</td>  
<td>

Owners of institutional repositories (if any)

</td> </tr> </table>

Table 5: Mapping of mechanisms to stakeholder groups

###  Software tools for sharing research data

Further to the dissemination strategy that will be developed in order to
inform interested stakeholders about availability of new research data, and to
the repository concepts explained in the following section, a number of
technical solutions will be taken into considerations, to allow the
possibility to share some of the generated data in an almost real-time manner.

Software mechanisms or interfaces for sharing some of the data types mentioned
in this document far include:

  * Geospatial Data o WMS and WFS server (e.g. using GIS tools like GeoServer [23] or MapServer 

  

[24]) or WMTS (using tools like e.g. MapProxy [29]) o Custom functionalities
to export them (e.g. download from the Web Portal) as

Shapefile or GeoPackage o Custom functionalities to export them to Google Maps
/ Google Earth [30] (widely used GIS applications)

  * SensorML services and related standard software interfaces (see Section 2.1) to disseminate sensor measurements 

  

###  Repository concept for enabling re-use

The machine learning and data mining community has a long history in sharing
and re-using datasets to test and benchmark algorithms and develop new
learning concepts. We give now a list of the most well-known such
repositories:

  * The oldest machine learning repository is the one maintained by the University of California, Irvine, currently hosting more than 300 datasets [6]. Datasets there usually come in the form of .names and .data files with the first providing the metadata describing the dataset and the second being a simple .csv file containing the actual data. 

  

  * A rather recent repository is mldata.org hosted by the machine learning group at the technical university of Berlin [27], which has been supported by the Pascale network [28]. It contains more than 800 datasets described in a number of different formats such 

  

as  HDF5 (503.6 MB),  XML,  CSV,  ARFF,  LibSVM,  Matlab,  Octave   KDD
nuggets maintain a repository of data repositories [29].

The above repositories can be used for dissemination and re-use of the data
generated by RAWFIE within the Machine Learning (ML) and Data Management (DM)
communities.

Another option for sharing and re-use could be the exploitation of the Linked
Open Data initiative and the reuse if appropriate of the technologies that
have been developed in a number of European initiatives and projects in the
area of open data such as: DaPaas [31], PlanetData [32]. However this might be
less appropriate for the kind of data that will be generated by the RAWFIE
infrastructure since Linked Open Data deal basically with the description of
entities, their properties and their relations to other entities, while in
RAWFIE we will be mainly generating measurements data.

#  Archiving and preservation

RAWFIE will consider all the necessary procedures for archiving and provision
of long term preservation. Suitable file formats and appropriate processes for
organizing files will be followed. In organizing the different data files the
following steps could be considered:

  * File version control 

  

  * File structure 

  

  * Directory structure and file naming conventions. 

  

In addition for the long-term access appropriate data documentation will be
provided. Full understanding and analysis of the metadata that may be needed
will be considered. For instance, for improving documentation process we could
classify the metadata in two levels: project- and data-level. Project-level
metadata describes the “who, what, where, when, how and why” of the dataset,
which provides context for understanding why the data were collected and how
they were used.

Examples of project-level metadata:

  * Name of the project 

  

  * Dataset title 

  

  * Project description 

  

  * Dataset abstract 

  

  * Principal investigator and collaborators 

  

  * Contact information 

  

  * Contact information 

  

Dataset level metadata are more granular. They explain, in much better detail,
the data and dataset.

Examples of data-level metadata:

  * Data origin, experimental, observational, raw or processes, models, images, etc. 

  

  * Data type: integer, boolean, character, floating etc 

  

  * Data acquisition details: sensor deployment methods, experimental design, sensor calibration methods, etc 

  

  * File types: CSV, mat, tiff, xlsx, HDF 

  

  * Data processing methods 

  

  * Dataset parameter list: Variable names, Description of each variable, units 

  

The external repositories that can be used for the purposes of archiving and
long-term storage were described above (see Section 3.2.2). These repositories
are free therefore there will not be expenses for the RAWFIE consortium. In
case of additional procedures are needed for the longterm maintenance the
project consortium will cover the respective costs.