#  Introduction

Full parallax imaging research requires databases for testing and comparison.
The research conducted in the project will encompass various capture types
(multiview, plenoptic), result images (light fields for different displays)
and compression schemes. For generating ground truth data or interactive light
field visualization the project is going to utilize datasets for real time
rendering as well.

A multiview image set is an image captured with multiple virtual or real
cameras at the same time. A plenotic image is an image captured with a
plenoptic camera. A light field image is an image captured from a scene with
the light rays described as a vector field. A practical example of this is a
light field directly describing the outgoing light rays of an autostereoscopic
3D display.

All of these images can be compressed into video streams. For multiview images
this can be a single stream or per camera streams, for plenoptic camera images
this is a single stream, for light field images, this can be a single stream
or per optical module streams.

For real time rendering of light fields it is also necessary to work with test
3D scenes imported from different 3D file formats. These are usually
containing geometry in the form of meshes or boundary representation surfaces
and materials in various formats. They can also link to various types of
texture files that describe surface properties. They can also contain scene
graph related information and animation.

For realistic environment lighting it is also necessary to use environment map
textures. With HDR imaging the changes of lighting during the day and camera
exposure settings can also be simulated in real time.

The document will cover what kinds of datasets are available for comparison
and testing. It will also introduce new datasets that will likely be created
(captured or rendered) during the project. For each dataset it will describe
data types, relevant standards and tools to interpret them. Plans for storage
and archiving procedures of the datasets will also be laid out.

The datasets will also be of use to future researchers as comparison to
existing methods for capture and compression and (rendered) image quality.

#  Datasets captured or created during the project

##  Captured plenoptic content

_Data set reference and name_

ETN-FPI-MIUN-PLENO

###  Data set description

Plenoptic capture is currently in the planning phase. The document will be
updated once more details become available on the capture setup and the
captured data set. Contact person: Mårten Sjöström e-mail:
Marten.Sjostrom@miun.se

##  Captured multiview content dataset from Bayer camera array

_Data set reference and name_

ETN-FPI-FHG-BAYER

###  Data set description

The dataset contains video and image files captured with the following setup:

The Basler Array consists of 9 cameras in a 3x3 configuration. The cameras are
synchronized via a trigger input and have global shutter sensors. The cameras
output 1 Gigabit Ethernet data which is captured using a PC equipped with a 10
Gigabit Ethernet adapter along with a 10 Gigabit Ethernet switch.

####  (a) (b)

####  (c) (d)

_**Figure 1: Basler Camera Array. (a) Schematic drawing. The distance between
the optical axes of horizontal and vertical neighbors is 65 mm, e.g. the
inter-axial distance is 65 mm. (b) Picture of the camera array. (c) Schematic
drawing of the Basler cameras array inside the rigging box. (d) Perspective
picture of the camera array.** _

###  Standards and metadata

Captured video streams are available as:

  * Individual rectified video streams from 9 cameras 

  

  * Framerate: 30 FPS 

  

  * Resolution: 1920x1080 pixels 

  

  * stream of PNG images 

  

  * 8-16 bits per color channel 

  

Captured images are available as:

  * Individual rectified images from 9 cameras 

  

  * Resolution: 1920x1080 pixels 

  

  * PNG images 

  

  * 8-16 bits per color channel 

  

Metadata:

  * Camera calibration data as text file 

  

###  Data sharing

Several datasets will be captured for the project, especially during WS2.
These will be shared among project partners via a local FTP server. Generally,
download links will be available for a week and will be renewed on request.

When and how datasets will be published to the general public is to be decided
at a later date. Contact person Frederik Zilly e-mail:
frederik.zilly@iis.fraunhofer.de

###  Archiving and preservation

Sequences will be stored on Fraunhofer IIS’s internal FTP server. Local backup
policies will be applied. All data will be mirrored.

##  Captured and compressed multiview content dataset from GoPro camera array

_Data set reference and name_

ETN-FPI-FHG-GOPRO

###  Data set description

The dataset contains video files captured with the following setup:

The GoPro camera array consists of 16 cameras in a 2x8 configuration. The
cameras are synchronized via special back packs and have rolling shutter
sensors. Each camera captures the videos individually on local storage (e.g.
SD-cards). The read-out is performed offline.

####  (a)

####  (b)

####  (c)

####  (d)

_**Figure 2: GoPro camera array. (a) Schematic drawing of the camera’s
centers. The horizontal translation is equidistant, i.e. the distance of two
horizontal neighbors is 60 mm in the drawing. The distance between vertical
neighbors is 70 mm. (b) Picture of the 2x8 camera array. (c) Perspective
drawing of the camera array. (d) Rear view of the cameras showing the red back
packs connected with gray cables to ensure synchronization.** _

###  Standards and metadata

Captured video is available as:

  * Individual rectified video streams from 16 cameras 

  

  * Framerate: 30 FPS 

  

  * Resolution: 1920x1080 pixels 

  

  * Stream of lossless PNG images converted from x264 video streams.  8 bits per color channel 

  

Metadata:

  * Camera calibration data as text file Relevant standards: 

  

  * PNG: ISO/IEC 15948:2003 

  

  * ISO/IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding 

  

###  Data sharing

Several datasets will be captured for the project especially during WS2. These
will be shared among project partners via a local FTP server. Generally
download links will be available for a week and will be renewed on request.

When and how datasets will be published to the general public is to be decided
at a later date. Contact person: Frederik Zilly e-mail:
frederik.zilly@iis.fraunhofer.de

###  Archiving and preservation

Sequences will be stored on Fraunhofer IIS’s internal FTP server. Local backup
policies will be applied. All data are mirrored.

##  Compressed multiview dataset from robot gantry

_Data set reference and name_

ETN-FPI-FHG-ROBOTGANTRY

###  Data set description

The dataset contains image files captured with the following setup:

The Robot gantry is suitable to capture static scenes using a translatable
DSLR camera. The DSLR camera can be translated by 4 meters in horizontal and
50 cm in vertical direction. The images are recorded onto the camera data
storage as RAW or JPG files.

_**(a)** _

_**(b)** _

_**Figure 3: Robot gantry. (a) Picture of the full robot gantry frame. The
camera can be translated by 4 meters in horizontal and 50 cm in vertical
direction (b) Near view of a mountable DSLR camera.** _

###  Standards and metadata

Captured video is available as:

  * Individual rectified video streams from ~20x20 camera positions 

  

  * Framerate: N/A 

  

  * Resolution: 24 MegaPixels 

  

  * Stream of lossless PNG images. 

  

  * 8-16 bits per color channel 

  

Metadata:

  * Camera calibration data as text file Relevant standards: 

  

  * PNG: ISO/IEC 15948:2003 

  

###  Data sharing

Several datasets will be captured for the project especially during WS2. These
will be shared among project partners via a local FTP server. Generally
download links will be available for a week and will be renewed on request.

When and how datasets will be published to the general public is to be decided
at a later date. Contact person: Frederik Zilly e-mail:
frederik.zilly@iis.fraunhofer.de

###  Archiving and preservation

Sequences will be stored on Fraunhofer IIS’s internal FTP server. Local backup
policies will be applied. All data are mirrored.

##  Compressed light field dataset

###  Data set reference and name

ETN-FPI-MIUN-PLENO

Data set descriptionLight field compression is currently in the planning
phase. The document will be updated if more data becomes available on light
field compression. Contact person: Mårten Sjöström e-mail:
Marten.Sjostrom@miun.se

##  Light field datasets rendered for Holografika displays

_Data set reference and name_

ETN-FPI-HOLO

_Data set description_

The dataset contains video and image sequences rendered for one or more
Holovizio light field displays.

###  Standards and metadata

Rendered video is available as

  * Per optical module x264 compressed light field videos 

  

Rendered images are available as:

  * Per optical module png light field images 

  

Metadata:

  * Documentation for the display’s light field 

  

  * Holografika light field description file 

  

  * Holografika SDKs 

  

Relevant standards:

  * PNG: ISO/IEC 15948:2003 

  

  * ISO/IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding 

  

###  Data sharing

Image and video sequences can be requested from Holografika for testing and
evaluation. Inclusion in published scientific work requires the explicit
permission of Holografika. Holografika light field description files and
Holografika SDKs are available only to partners signing an NDA.

Holografika SDKs are currently only available with commercial licensing.

Contact person: Attila Barsi e-mail: a.barsi@holografika.com

###  Archiving and preservation

Sequences are mirrored on Holografika’s internal file server. A backup server
with mirrored data is available. An offline backup of the data is available on
HDD.

#  Datasets used for rendering, testing, compression and comparison

##  Complex 3D scene for light field rendering

_Data set reference and name_

ETN-FPI-COMPLEX3D

###  Data set description

Big Buck Bunny is a Blender community open movie project. The dataset contains
the original Blender project and various 2D, stereoscopic and multiview videos
rendered from the same dataset.

###  Standards and metadata

The project files are available in Blender’s internal .blend file format.
Licensed under Create Commons 3.

Rendered video is available as

  * x264 compressed 4K stereo video 

  

  * x264 compressed Full HD stereo video 

  

  * x264 compressed 4K 2D video 

  

  * x264 compressed Full HD 2D video 

  

  * yuv uncomressed 1280x768 multiview video 

  

Rendered image files are available as

  * .exr 32 bit floating point 1280x768 multiview images 

  

  * .png 1280x768 multiview images 

  

Metadata:

  * documentation for Holografika’s proprietary camera .xml format 

  

  * camera placement in Holografika’s proprietary camera .xml format 

  

  * tone mapping settings for HDR to LDR conversion 

  

Relevant standards:

  * PNG: ISO/IEC 15948:2003 

  

  * ISO/IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding 

  

###  Data sharing

2D, stereo and Multiview are available from the HTTP file servers.

  * Original scene, 2D and stereo videos  _http://bbb3d.renderfarming.net/explore.html_

  

  * Scenes setup for multiview rendering, multiview video and images _http://mpegftv:lf3dvideo@mpeg3dvideo.holografika.com/_

  

###  Archiving and preservation

Scenes setup for multiview rendering, multiview video and images are backed up
to a backup server and an offline backup HDD. Contact person: Attila Barsi
e-mail: a.barsi@holografika.com

##  Simple 3D scene for light field rendering

_Data set reference and name_

ETN-FPI-SIMPLE3D

_Data set description_

This is a 3D datasets for sci-fi vehicles from the movie Star Wars. Licensed
under Create Commons 3.

_Standards and metadata_

The scene files are available in various 3D formats including MAX4, 3DS and
Lightwave .lwo formats.

###  Data sharing

3D scene definition files are available at the following HTTP file server:

  _http://www.scifi3d.com/list.asp?intCatID=8 &intGenreID=10 _

###  Archiving and preservation

Models are mirrored on Holografika’s internal file server. A backup server
with mirrored data is available. An offline backup of the data is available on
HDD. Contact person: Attila Barsi e-mail: a.barsi@holografika.com

##  Environment maps for 3D rendering

_Data set reference and name_

ETN-FPI-ENVMAPS

###  Data set description

Environment maps for 3D scene rendering captured with ball mirrors. The
primary purpose of these maps is to enable environment lighting.

###  Standards and metadata

Images are available as LDR .jpg and HDR .hdr images.

Licensing allows the free samples to be used for commercial work.

_Data sharing_ _https://www.hdri-hub.com/hdrishop/freesamples/freehdri_

###  Archiving and preservation

Maps are mirrored on Holografika’s internal file server. A backup server with
mirrored data is available. An offline backup of the data is available on HDD.
Contact person: Attila Barsi e-mail: a.barsi@holografika.com