#  EXECUTIVE SUMMARY

The present document is a deliverable of the SARAFun project, funded by the
European

Commission’s Directorate-General for Research and Innovation (DG RTD), under
its Horizon 2020 Research and innovation programme (H2020). It presents the
first version of the project Data Management Plan (DMP). The current document
explains in detail what data will be generated throughout the project’s
lifecycle, the possible means for the sharing of this data in order to become
accessible for verification and reuse, as well as the ways in which it will be
curated and preserved. Additionally, it provides the necessary information in
order for the Data Management Portal to be afterwards created through this
project’s activities.

It is strongly emphasized that this is an ongoing document that is being
evolved along with the project progress and will be regularly updated in order
to reflect up-to-date information.

#  INTRODUCTION

##  PURPOSE

The purpose of this deliverable (D7.5 “Draft Data Management Plan”) is to
deliver an analysis of the main elements of the Data Management Policy that
will be used by the consortium with regard to all the datasets generated by
the SARAFun project. The DMP is not a fixed document, but will evolve
throughout the project’s lifecycle. This first version of the DMP includes an
overview of the datasets to be produced by the project as well as the specific
conditions are attached to them. The next version of the DMP will be published
at M36 through the activities of D7.8 and will describe in more details the
data generated as well as the uses identified by the consortium.

##  GENERAL PRINCIPLES

Through the activities of the SARAFun project [1], pioneer research will be
carried out in order to develop and deliver a next generation bi-manual robot
that can be exploited in the production lines for assisting human workers in a
safety manner through novel human demonstration and teaching algorithms. To
this end, human participants will be involved in the project and data will be
collected regarding their assembly’s movements and assembly forces in a
production line. For the purpose of optimizing the project’s development, a
process of knowledge management will be implemented. This process will provide
the consolidation of the knowledge spiral, enable cooperation and will
additionally allow for the creation of new knowledge. All the participants of
the project have to cooperate in order to reach the most efficient process of
knowledge management. Initially algorithms that have been implemented to
identify objects, grasping and the recognition of the characteristics of a
grip (such as rotation, strength, speed), will be used before their adjustment
on a production line, in order to allow the optimization measures. Therefore,
a database to store data for benchmarking the algorithms developed in the
project lifetime and beyond is required. Several experiments will be made
using the algorithms on a production line and each experiment will derive a
significant data. Developers will refer to these data with a view to obtain
information in order to increase the efficiency of the implemented algorithms.
Moreover, a part of the performed experiment’s data and of the algorithm’s
code will be provided to the scientific community as well as to robotics
researchers in order to support the optimization of their executive power
(e.g. utilizing github repository for open access to code developed in the
project lifetime as well as publication to open access journals).

_**Participation in the Pilot on Open Research Data** _

SARAFun highly supports the Pilot on Open Research Data launched by the
European Commission along with the Horizon2020 programme, and therefore a
significant part of research data generated by the project will be made open
and will be offered to the Open Research Data Pilot, where SARAFun will
participate. To this end, the Data Management Plan provided through the
activities of this deliverable, explains in detail what data the project will
generate, whether and how it will be exploited or made accessible for
verification and reuse, and how it will be curated and preserved.

_**IPR Management & Security ** _

Due to the high innovative nature of the SARAFun project, high level
technologies will be developed during the project’s lifecycle in order to be
afterwards released in the market. Therefore, foreground capable of industrial
or commercial application must be protected taking into account legitimate
interests. All involved partners have Intellectual Property Rights on the
technologies and data developed or collected with their participation. As the
partners’ economic sustainability highly depends on these technologies and
data, SARAFun Consortium will protect all data collected for SARAFun purposes.
Additionally, prior notice of dissemination will be given to other
participants, whereas any dissemination such as publications and patent
applications must indicate the Community financial assistance. Moreover,
appropriate measures will be taken for effectively avoiding a leak of data,
while all data repositories of this project will be adequately protected.

_**Personal Data Protection** _

SARAFun involves the carrying out of data collection in order to assess the
technology and effectiveness of the proposed solution. This will be carried
out in full compliance of any European and national legislation and directives
relevant to the country where the data collections are taking place

(INTERNATIONAL/EUROPEAN):

i) The Convention 108 for the Protection of Individuals with Regard to
Automatic Processing of Personal Data; ii) Directive 95/46/EC & Directive
2002/58/EC of the European parliament regarding issues with privacy and
protection of personal data and the free movement of such data; iii) The
legislation in Sweden _:_ The 1998 Personal Data Act ; iv) The Spanish Organic
Law 15/99 (amendments : 5/02 & 424/05);

v) The Greek Law 2472/1997: Protection of Individuals with regard to the
Processing of Personal Data, and vi) The Greek Law 3471/2006: Protection of
personal data and privacy in the electronic telecommunications sector and
amendment of law 2472/1997.

More detailed information regarding data privacy issues can be found in
Deliverable 1.2 “Preliminary Ethics and Safety Manual for SARAFun technology”.

#  DATASET LIST

For the purposes of SARAFun a number of datasets needs to be created, which
are listed in the following table, together with a short description for each
one of them.

**Table 1: Dataset List Table**  
  
<table>  
<tr>  
<th>

**#**

</th>  
<th>

**Dataset Name**

</th>  
<th>

**Description**

</th>  
<th>

**WPs & Tasks **

</th> </tr>  
<tr>  
<td>

1

</td>  
<td>

DS.01.CERTH.

KeyFrameExtraction

</td>  
<td>

Dataset used for key frame extraction in laboratory and factory environments.

An instructor person will pick up two small objects and, afterwards will
assemble them.

</td>  
<td>

The data are going to be collected within the activities of WP3 and more
specifically Tasks T3.1, T3.2 and T3.3.

</td> </tr>  
<tr>  
<td>

2

</td>  
<td>

DS.02.CERTH.ObjectTracking

</td>  
<td>

Dataset used for object tracking and object pose estimation in laboratory and
factory environments. Three variations will be used:

1) experiments with non-occluded objects, 2) partially occluded objects by
either a) the instructor’s hand or b) another object, and

3) combination of the above.

</td>  
<td>

The data are going to be collected within the activities of WP3 and more
specifically T3.1.

</td> </tr> </table>

##  DATASET “DS.01.CERTH. KEYFRAMEEXTRACTION”

**Table 2: Dataset “DS.01.CERTH. KeyFrameExtraction”**  
  
<table>  
<tr>  
<th>

**1.**

</th>  
<th>

**Data set reference and name**

</th> </tr>  
<tr>  
<td>

**DS.01.CERTH. KeyFrameExtraction**

</td> </tr>  
<tr>  
<td>

**2.**

</td>  
<td>

**Data set description**

</td> </tr>  
<tr>  
<td>

_**General Description** _

Dataset used for key frame extraction in laboratory and factory environments.

An instructor person will pick up two small objects and, afterwards will
assembly them.

_**Origin of Data (e.g. indicative collection procedure, devices used etc.)**
_ Device type: RGBD sensor.

Two aligned streams will be used, extracted from one depth sensor (640X480 or
960X540) and one RGB camera (1920X1080). The two sensors will operate in a low
range area (20cm to 1.5m). Sampling rate: 30 fps.

</td> </tr> </table>  
<table>  
<tr>  
<th>

_**Nature and scale of data** _

The data will be available in video format (e.g. image sequences, video file
format, etc.). Scale will be specified later on.

_**To whom could the dataset be useful** _

This dataset will be useful to key frame extraction algorithms.

_**Related scientific publication(s)** _

This dataset will accompany SARAFun’s publications in the field of key frame
extraction. _**Indicative existing similar data sets (including possibilities
for integration and reuse)** _ To be specified later on.

</th> </tr>  
<tr>  
<td>

**3.**

</td>  
<td>

**Standards and metadata**

</td> </tr>  
<tr>  
<td>

For the metadata RGBD sensors will be used. Two aligned streams will be used,
one depth camera (640X480 or 960X540) and one RGB (1920X1080). Both sensors
will have low range (20cm-1.5m). Sampling rate 30 fps, both nominate and real
sampling rate will be included. Frame sequent. Definition for depth and colour
stream. Horizontal deviation angle (YAW angle). Key frame annotation (ground
truth of the correct key frames). Lighting conditions. Manipulated object type
(primitive shapes, ICT component, etc). FOV: horizontal and vertical.
Annotation will be given based on the outputs of the algorithms produced, and
will be used later on as a basis for evolving other algorithms, in addition to
the manually defined (ground truth) key frames. The metadata will be provided
in xml format with the respective xml schema.

Indicative metadata include a) camera calibration information, b) camera pose
matrix for each viewpoint, c) 3D pose annotation, d) 3D object model in CAD
format. The metadata will be in a format that maybe easily parsed with open
source software."

</td> </tr>  
<tr>  
<td>

**4.**

</td>  
<td>

**Data sharing**

</td> </tr>  
<tr>  
<td>

_**Access type (widely open, restricted to specific groups, private)** _

Open

_**Access Procedures** _

A web page will be created by CERTH on the SARAFun data management portal that
should provide a description of the dataset as well as links to a download
section. _**Embargo periods (if any)** _

None

_**Technical mechanisms for dissemination** _

A link to the dataset will be provided from the SARAFun web page, and in all
relevant SARAFun publications.

_**Necessary S/W and other tools for enabling re-use** _

Commonly available tools and software libraries for enabling reuse of dataset
(e.g.OpenCV).

_**Repository where data will be stored (institutional, etc., if already
existing and identified)** _

A data management portal will be created and maintained by CERTH in order to
accommodate full as well as public versions of the datasets used. Links to the
portal will also exist at the SARAFun website, while the data will be also
stored at CERTH’s servers and other common back-up mechanisms in order to

</td> </tr>  
<tr>  
<td>

avoid loses of data and ensure data reliability.

</td> </tr>  
<tr>  
<td>

**5.**

</td>  
<td>

**Archiving and preservation (including storage and backup)**

</td> </tr>  
<tr>  
<td>

_**Data preservation period** _

Data will be preserved for at least 2 years after the end of the project.

_**Approximated end volume of data** _

The volume of data is estimated at approximately 1,26GB/min for RGB and 1,08
GB/min for depth. 1015 sequences will be captured ranging from 15 to 30
seconds each. Each sequence will hold a volume of approximately 700 MB (400 MB
for colour and 300 MB for depth).

_**Indicative associated costs for data archiving and preservation** _

A hard disk drive (approximately 1 Terabyte) will be probably allocated for
the dataset. There are no costs associated with its preservation.

_**Indicative plan for covering the above costs** _

The initial costs will be covered by SARAFun, while the costs that will come
up after the finalization of the project will be covered by CERTH.

</td> </tr> </table>

##  DATASET “DS.02.CERTH.OBJECTTRACKING”

**Table 3: Dataset “DS.02.CERTH. ObjectTracking”**  
  
<table>  
<tr>  
<th>

**1.**

</th>  
<th>

**Data set reference and name**

</th> </tr>  
<tr>  
<td>

**DS.02.CERTH.ObjectTracking**

</td> </tr>  
<tr>  
<td>

**2.**

</td>  
<td>

**Data set description**

</td> </tr>  
<tr>  
<td>

_**General Description** _

Dataset used for object tracking and object pose estimation in laboratory and
factory environments. Three variations will be used:

  1. experiments with non-occluded objects, 

  

  2. partially occluded objects by either a) the instructor’s hand or b) another object, and 3) combination of the above. 

  

  

_**Origin of Data (e.g. indicative collection procedure, devices used etc.)**
_ Device type: RGBD sensor.

Two aligned streams will be used, extracted from one depth sensor (640X480 or
960X540) and one RGB camera (1920X1080). The two sensors Both will operate in
a low range area (20cm to 1.5m). Sampling rate: 30 fps.

_**Nature and scale of data** _

Video format (e.g. image sequences, video file format, etc.) Scale will be
specified later on.

_**To whom could the dataset be useful** _

This dataset will be useful for object recognition alogirithms.

</td> </tr> </table>  
<table>  
<tr>  
<th>

_**Related scientific publication(s)** _

This dataset will accompany SARAFun’s publications in the field of object
recognition.

_**Indicative existing similar data sets (including possibilities for
integration and reuse)** _

  1. Latent-Class Hough Forests for Object Detection and Pose Estimation 

  

  

( _http://www.iis.ee.ic.ac.uk/rkouskou/Research.html_ )

  2. Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily 

  

  

Cluttered Scenes ( _http://campar.in.tum.de/Main/StefanHinterstoisser_ )

3.The Berkley's B3DO dataset ( _http://kinectdata.com/_ )

4\. The Berkley's BigBird dataset ( _http://rll.berkeley.edu/bigbird/_ ).

</th> </tr>  
<tr>  
<td>

**3.**

</td>  
<td>

**Standards and metadata**

</td> </tr>  
<tr>  
<td>

For the metadata RGBD sensors will be used. Two aligned streams will be used,
one depth camera (640X480 or 960X540) and one RGB (1920X1080). Both sensors
will have low range (20cm-1.5m). Sampling rate 30 fps, both nominate and real
sampling rate will be included. Frame sequent. Definition for depth and colour
stream. Horizontal deviation angle (YAW angle). Key frame annotation (ground
truth of the correct key frames). Lighting conditions. Manipulated object type
(primitive shapes, ICT component, etc). FOV: horizontal and vertical.
Annotation will be given based on the outputs of the algorithms produced, and
will be used later on as a basis for evolving other algorithms. The metadata
will be provided in xml format with the respective xml schema.

Indicative metadata include a) camera calibration information, b) camera pose
matrix for each viewpoint, c) 3D pose annotation, d) 3D object model in CAD
format. The metadata will be in a format that maybe easily parsed with open
source software."

</td> </tr>  
<tr>  
<td>

**4.**

</td>  
<td>

**Data sharing**

</td> </tr>  
<tr>  
<td>

_**Access type (widely open, restricted to specific groups, private)** _

Open

_**Access Procedures** _

A web page will be created by CERTH on the SARAFun data management portal that
should provide a description of the dataset as well as links to a download
section. _**Embargo periods (if any)** _

None

_**Technical mechanisms for dissemination** _

A link to the dataset will be provided from the SARAFun web page, and in all
relevant SARAFun publications.

_**Necessary S/W and other tools for enabling re-use** _

Commonly available tools and software libraries for enabling reuse of dataset
(e.g.OpenCV).

_**Repository where data will be stored (institutional, etc., if already
existing and identified)** _

A data management portal will be created and maintained by CERTH in order to
accommodate full as well as public versions of the datasets used. Links to the
portal will also exist at the SARAFun website, while the data will be also
stored at CERTH’s servers and other common back-up mechanisms in order to
avoid loses of data and ensure data reliability.

</td> </tr>  
<tr>  
<td>

**5.**

</td>  
<td>

**Archiving and preservation (including storage and backup)**

</td> </tr>  
<tr>  
<td>

_**Data preservation period** _

Data will be preserved for at least 2 years after the end of the project.

_**Approximated end volume of data** _

The volume of data is estimated at approximately 1,26GB/min for RGB and 1,08
GB/min for depth. 5-10 sequences will be captured ranging from 15 to 30
seconds each. Each sequence will hold a volume of approximately 700 MB (400 MB
for colour and 300 MB for depth).

_**Indicative associated costs for data archiving and preservation** _

A hard disk drive (approximately 1 Terabyte) will be probably allocated for
the dataset. There are no costs associated with its preservation.

_**Indicative plan for covering the above costs** _

The initial costs will be covered by SARAFun, while the costs that will come
up after the finalization of the project will be covered by CERTH.

</td> </tr> </table>

#  CONCLUSION

This deliverable constitutes a first draft analysis of the procedures and
infrastructures that will be implemented by SARAFun in order to effectively
manage the data produced through the project’s activities. One of the key
elements of the Data Management Plan constitutes the Data Management Portal,
which will handle and manage the large amount of datasets collected from the
devices used for the SARAFun purposes.

Special care will be given in order for the Data Management Portal to allow
specific access to all partners participating in the process of data
production. Additionally, editing and access rights will be managed in an
appropriate way. Moreover, special attention will be given by the SARAFun data
management plan to the appropriate collection and publication of metadata. All
necessary information will be stored in order to facilitate the optimal use as
well as the re-use of these datasets. Each data producer will be responsible
for managing the respective data and metadata, whereas all data and metadata
will be integrated in the Data Management Portal. Specific flexibility levels
are required by the Data Management Portal regarding the public datasets, as
well as attention towards the IPR rights of every partner and the European and
National regulations and directives regarding personal data privacy and
protection.

In conclusion, the current document presents a first overview of the datasets
used and the kind of data gathered for the SARAFun purposes, as well as for
the specific challenges that need to be considered for their effective
management. It is emphasized that this constitutes an ongoing document and
will therefore be updated throughout the project’s lifecycle. The final,
updated version of this document will be delivered in M36 through the
activities of D7.8, and will provide a more detailed Data Management Plan,
whereas the Data Management Portal will be at its final stage by that time.