Deliverable
Deliverable nameD5.2 – Report on Immersive Content Production –Part 2 and Interactive Media Authoring

Project Acronym:
Grant Agreement number:
Project Title:

IMMERSIFY
762079
Audiovisual Technologies for Next Generation
Immersive Media

Revision:
Authors:
Delivery date:Dissemination level (Public / Confidential)

2.0
Maciej Glowiak (PSNC), Anna Kuthan (AEF)
M30
Public

Abstract
The subject of deliverable 5.2 is to deliver video clips for project needs (type is “DEC: Websites,
patents filing, press & media actions, videos, etc.”). This report focuses on Immersive Content
Production and summarizes work done in “Task 5.1 Content Production” and in “Task 5.2 Interactive
Media Authoring”. The document describes how T5.1 and T5.2 addressed objectives defined for“WP5 Creative Development and Demonstrations” and describes a set of immersive content
material created using different production methods for the Immersify project.
This report is the second iteration of immersive content after D5.1 (M12). It includes additional
information on content produced in the second reporting period as well as describes the technicalaspects and methodology used for creating interactive media content.

This project has received funding from the European Union’s Horizon 2020 research and innovation
programme under grant agreement 762079.

REVISION HISTORY
Revision

Date

Authors (Entity)

Description of changes

0.1

1 Sep 2018

D5.1: Initial content of the document, integration of
PSNC content descriptions

0.2

17 Sep 20180.3

18 Sep 2018

0.4

21 Sep 2018

1.0
1.1

28 Sep 2018
24 Sept 2019

Maciej Glowiak
Maciej Strozyk
Eryk Skotarczak
Christian Södergren
Erik Sunden
Anna Kuthan
Sergio Sanz-Rodriguez
Maciej Glowiak
Maciej Strozyk
Maciej Glowiak
Anna Kuthan (AEF)

1.2

6 Mar 2020

1.3

16 Mar 2020

1.4
2.0

30 Mar 2020
31 Mar 2020

Clemens Scharfen, Roland
Haring, Anna Kuthan
Maciej Strozyk, Maciej
Glowiak, Maciej Jaskiewicz
Maciej Glowiak
Anna Kuthan

D5.1: Content/immersive production
D5.1: Content + review of document
D5.1: Adding missing sections, first complete draft forreview
D5.1: Final review, final version
D5.2: Initial content of the document basing on
information from D5.1
D5.2: Content produced in T5.2
D5.2: Content produced in T5.1
D5.2: Review
D5.2: Final version

CONTRIBUTORSContributor’s nameEntityContact e-mailClemens Scharfen
Maciej Strozyk
Roland Haring
Anna Kuthan
Maciej Glowiak
Maciej Jaskiewicz
Eryk Skotarczak
Sergio Sanz-Rodriguez
Christian Södergren
Erik Sundén

AEF
PSNC
AEF
AEF
PSNC
PSNC
PSNC
SD
NVAB
NVAB

Clemens.Scharfen@ars.electronica.art
mackostr@man.poznan.plrolandha@aec.at
annaku@aec.at
mac@man.poznan.plmjaskiewicz@man.poznan.pleryk@man.poznan.pl
sergio@spin-digital.com
christian.sodergren@visualiseringscenter.se
erik.sunden@liu.se

Deliverable 5.2

Revision 1.0

2/41

TABLE OF CONTENTS
1

Introduction

5

2

Results52.1

Addressing Objectives

5

2.2

Video Clips

6

2.2.1

Summary6

2.2.2

Clips produced by Immersify project in (Part I: M1-M12)

92.2.2.1

Kornik Castle Garden 3D92.2.2.2

Slowmotion 4K 3D

10

2.2.2.3

HighFrameRate 120p

112.2.2.4

Anomalia

13

2.2.2.5

The Noise and The Structure (rooftop)

14

2.2.2.6

LuxFest 360

162.2.2.7

Poznan Drive 3D 8K

17

2.2.2.8

Hel Peninsula

18

2.2.2.9

Always dead and alive (Immersive Minimalism)20

2.2.2.10

Supercomputers (Theresa Schubert)

21

2.2.2.11

Fallout Shelter

22

2.2.2.12

Poznan Ice Festival 8K 3D

24

2.2.2.13

Krambabula 360

25

2.2.2.14

InterBee 2019 8K

26

2.2.2.15

Ars Electronica Festival 201928

Clips produced with third parties

30

2.2.3
2.2.3.1

Island in the Sky II

30

2.2.3.2

Singing Sand 2.0

31

2.2.3.3

tx-reverse 360°32

2.2.3.4

The translucent St. Stephen’s Cathedral

34

Preparing the content for final showcase

35

2.2.4
3

Interactive Media Authoring
3.1

36

Deep Space 8K DevKit

36

3.1.1

Interaction

36

3.1.2

Features37

3.1.2.1

Project Settings37

3.1.2.2

Command Line Configuration

37

Deliverable 5.2

Revision 1.0

3/41

3.1.2.3

Wall-Floor-Camera

37

3.1.2.4

Pharus Laser Tracking

37

3.1.2.5

Xbox Controller

38

3.1.2.6

Mobile Control

38

3.1.2.7

VRPN

38

3.1.2.8

Networking

38

3.2

Dome and Interactive Content

3.2.1

39

AdventureDome

39

3.2.1.1

Features39

3.2.1.2

Interaction

39

3.2.1.3

Production

39

3.2.2

OpenSpace

40

4Conclusions

40

5

References

41

LIST OF ACRONYMS
AcronymDescription

HMD
CGI
360°

Head Mounted Display
Computer Generated Imagery
360° video

Deliverable 5.2

Revision 1.0

4/41

1 Introduction
This report focuses on Immersive Content Production and involves tasks “Task 5.1 Content Production”
and “Task 5.2 Interactive Media Authoring”.
Task 5.1 is dedicated to the creation of new immersive content that can be used to demonstrate thecreative and immersive capabilities of the new formats. It also determines how to improve and assess
the tools to be developed in the project. We assumed that the content for the project could beacquired and produced in several ways: by using existing content; by producing new immersive video
content using the infrastructure and equipment from the project partners or by partnerships withexternal partners.
Content for the project was to be produced using different techniques in order to explore and compare
different production methods: time-lapse photography, 3D laser scan based renderings, CGI andanimation, 360° panoramic high-resolution video, and real-life 8K footage. It was also assumed that apart of the content creation and production will be subcontracted to leading content producers. The
new test content was also needed to validate the codec technologies developed in the project andoptimize the media creation workflows with regard to different content genres in collaboration withthe content producers.
Task 5.2 focuses on the creation process of interactive media for VR environments and VR applications.
Using the playback infrastructures created in WP3 (mostly T3.5 Unity3D integration) and the content
from T5.1 several prototypes for interactive non-linear narratives were created. The prototypes target
different VR systems (dome, deep space, VR glasses), audiences (children - adult, single user - multiuser groups), interaction forms (gesture based, positional tracking, head tracking, direct and indirect
manipulation), presentation forms (guided, unguided), media forms (video, audio, real-time CGI) and
rendering techniques (monoscopic - stereoscopic, augmented virtuality). The results are evaluated inT5.3 and integrated in T5.4.

2 Results
2.1 Addressing Objectives
WP5 defines 7 major objectives, but only some of them are related to T5.1 and T5.2
Objective 1: Create new 360° content that is able to demonstrate the capabilities of the project’s
outcomes on different environments: Head-mounted-display, multi-screen, multi-projection.
We created several video clips and gained some from other sources, as part of subcontracting. The
video clips created during the first reporting period were reported in D5.1 Report on Immersive
Content Production Part 1. The list of video clips created after the first reporting period (from M13M30) is enlisted in Section 2.2.
Objective 2: Explore different methods for producing and presenting beyond UHD-4K content
including time-lapse photography, 3D laser scanning, CGI and animation, panoramic video, and reallife 8K footage.
Deliverable 5.2

Revision 1.0

5/41

A number of video clips were produced in various technologies and different conditions: indoor andoutdoor, fast movement, negative temperatures, stereoscopic, timelapse, laser scanning, CGI etc.
Objective 3: Experiment and prototype on new forms of interactive, non-linear storytelling buildingon the technologies created in WP3 and WP4 and unique high-resolution content.
Using the playback infrastructures created in WP3 (mostly T3.5 Unity3D integration) and the content
from T5.1 several prototypes for interactive non-linear narratives were created.
Objective 4: Encode the immersive content using the HEVC encoder developed in WP4, and find theappropriate quality-rate points for target applications and demonstrations. and Objective 5: Find
optimal configurations of the encoder for high-quality 8K delivery and playback. and Objective 6:
Perform an informal subjective test on target display environments to validate the quality
enhancements developed in the encoder, and give feedback for potential encoder improvements.
All video contents are accessible for all the partners using common repository. Each movie is encodedby Spin Digital HEVC encoder in several ways and optimal parameters are set. Parameters are selectedbased on experience gained so far, as well as after screening on different devices and subjective quality
assessment (e.g. 8K video wall, 8K monitors, Deep Space and CAVE projection).
Objective 7: Establish and document best practice guidelines for media production workflows
During production of each movie, we documented the whole process incl. equipment used. The
making-of photos and video clips are provided on www.immersify.eu webpage and were acontribution to best practice guidelines that are published on webpage.

2.2 Video Clips
2.2.1 Summary
In the project proposal, we planned production (or gaining from external sources) of several video clips
in various technologies, as follows:

Count

Resolution

Video

Technique

Duration

2 clips

8K or higher

Immersive 360°,2D

360° Time lapse or 360°
camera1 minute or
longerResults: Done
Total number of clips: 6
PSNC produced several 360° video clips in ultra-high resolution (8K up to 16K) using Orah 4i (4K),
Insta360 Pro camera (8K) and custom 360° composed of 8 Blackmagic cameras (16K). Partners of
the project tried different methods of obtaining high resolution and high quality omnidirectional
content with using various techniques and obtained some content from external companies such as
Arri (which provided 12K content for the project).
Content finished:
Part I● Cathedral 8K 360° (PSNC)● Poznan 8K 360° (PSNC)● Arri demonstration reel 12K (SD) - Pandarama
Part II
Deliverable 5.2

Revision 1.0

6/41

●
●
●
●

Anomalia Jazz Band 8K (PSNC)
Noise and Structure 14K (PSNC)
LuxFest 8K (PSNC)
Krambabula 16K (PSNC)

2 clips

8K or higher

Stereo 3D

360° camera
3D scan CGI

1 minute or
longerResults: Done
Total number of clips: 10
PSNC has made 3D laser scans of Poznan Cathedral church and generated a number of highresolution clips, ranging from 8K to beyond 16K. AEF released several clips in 12K-30K (point cloud
renderings of the Great Pyramid and St. Stephen’s Cathedral, body scans and CGI in Virtual Anatomy)
and NVAB produced 2 interactive clips.
Content finished:
Part I● From the inside - Poznan Cathedral (PSNC)● Adventure Dome (NVAB)● Open Space (NVAB)● Great Pyramid (AEF) - 3 clips● Virtual Anatomy (AEF)
Part II● tx-reverse 360°(AEF)● Fallout shelter 16K (PSNC)● The translucent St. Stephen’s Cathedral (AEF)
2 clips

8k or higher

Stereo 3D

3D animated rendered
content

1 minute or
longerResults: Done
Total number of clips: 8
PSNC produced several video clips in various formats for 3D displays available in the project (PSNC
8K wall, Deep Space at Ars Electronica, Dome Theater in NVAB). Two productions in this category(however only in 2D) were created by Theresa Schubert during her STARTS Vertigo residency at
PSNC: Always dead or alive, Supercomputers. The stereoscopic piece Prima Materia from Istanbul
based studio Nohlab was rendered in 8K x 8K in 3D. Singing Sand 2.0, a 3D stereoscopic audiovisual
work in 8K resolution was developed especially for Deep Space 8K.
Content finished:
Part I● From the Inside in 16K, 12K and 8K (PSNC) - 3D version, 2 clips● Prima Materia 8K (AEF)
Part II● Singing Sand 2.0 (AEF)● Fallout shelter 8K 3D (PSNC)● Always dead or alive (PSNC, Theresa Schubert) - 2D, 2 clips● Supercomputers (PSNC, Theresa Schubert) - 2D (in postproduction)

Deliverable 5.2

Revision 1.0

7/41

2 clips

8K

2D
60-120 fps

SONY F65

1 minute or
longerResults: Done
Total number of clips: 18
Since PSNC has SONY F65 cameras recording 8K images, this category is represented by the largest
number of video clips. In total, 18 of them have been recorded. Additionally, PSNC provided several
8K videos recorded before official start of the project:
Content finished:
Pre-project content:● Clock 8K [link]● Fish 8K [link]● Magda 8K (PSNC) [link]● Palm House 8K (PSNC) [link]
Part I● Dziekanowice Ethnographic Park (PSNC) - 3 clips● Poznan Race Track (Followcar) (PSNC)● Karkonosze - Szklarka Waterfall (PSNC)● Karkonosze - Kamieńczyk Waterfall (PSNC)● Karkonosze - Szrenica Peak (PSNC)● Karkonosze - Wang Temple in Karpacz (PSNC)● Karkonosze - Glassworks in Szklarska Poręba (PSNC)● Cathedral (PSNC)● Baltic Legend Opera (PSNC)● 60th TVP Anniversary Gala
Part II● HighFrameRate 120p (PSNC), 2 clips● Anomalia 8K (PSNC)● Hel Peninsula 8K (PSNC)● InterBee 8K streaming (PSNC, SD)● ArsElectronica 8K streaming (PSNC, AEF)
2 clips

8K

3D
60-120 fps

SONY F65 + 3D rig

1 minute or
longerResults: Done
Total number of clips: 5
During the project 5 3D clips were made with different techniques (rig with two SONY F65 cameras,
rig with two Phantom Flex 4K slowmo cameras, 3D timelapse).
Content finished:
Part I● one in postproduction (Kornik Castle Garden 8K 3D)
Part II● Kornik Castle Garden 8K 3D (PSNC)● Slowmotion 4K 3D (PSNC)● Poznan Drive 8K 3D (PSNC)● Ice festival 8K 3D (PSNC)
Deliverable 5.2

Revision 1.0

8/41

2 clips

8K or more

2D

Timelapse

1 minute or
longerResults: Done
Total number of clips: 5
AEF has subcontracted three timelapse videos in 8K and 10K and PSNC recorded 2 in 8K and 16K.
Content finished:
Part I● Atacama Desert (8K) - Martin Heck / Timestorm Films (AE)● Pano LA 10K - Joe Capra / Scientifantastic (AE)Part II● Poznan Timelapse (PSNC)● Island in the Sky II (8K) - Timestorm Films (AE)● Hel Peninsula 16K (PSNC)

2.2.2 Clips produced by Immersify project in (Part I: M1-M12)
Clips produced in reporting period 1 (until M12) have been already described in D5.1 and are as
following:

2.2.2.1

Kornik Castle Garden 3D

General description
Title

Kornik Castle Garden 3D

DateMay, 2018

Place

Kornik, Poland

Number of clips

1 (8K 3D)

Content Summary
8K 3D photos taken in the arboretum of the castle in Kórnik near Poznań
Technology
Two 8K SONY F65 cameras placed on Stereotec 3D rig

Deliverable 5.2

Revision 1.0

9/41

Introduction
Kórnik Castle is a castle in Poland, which was constructed in the 14th century. The current neogothic
design and remodeling was done in 1855. The current look of the castle resembles Gothic Revival
architecture, one of popular historicising styles in the 19th century.
The castle is surrounded by Kórnik Arboretum founded by Count Tytus Działyński in the first half ofthe 19th century - the oldest and largest one in Poland as well as the fourth largest arboretum inEurope covering around 40 hectares and containing more than 3300 taxa of trees and shrubs.
In this idyllic scenery, in May 2018, the PSNC team recorded footages of flowering plants in thearboretum using two 8K cameras and a 3D rig, due to lack of power in the park, battery supply of
the cameras and all equipment was used. The editing process took quite a long time because of thelarge volume of 8K 3D raw data and time-consuming post-production but thanks to the use of 8K
technology it was possible to capture the richness of colours and details of the filmed plants, andthe stereoscopic technology further expanded and strengthened the final effect.
Reference to the content

https://www.youtube.com/watch?v=rhjYfemleo

2.2.2.2

Slowmotion 4K 3D

General description
Title

Slowmotion 4K 3D

Date

2019

Place

Poznan, Poland

Number of clips

1 (4K 3D 60p)

Content Summary
Deliverable 5.2

Revision 1.0

10/41

3D recordings in PSNC studio made using slowmotion technology with two Phantom Flex 4K cameras
and 3D rig
Technology
Ultra High-speed cameras Phantom Flex 4K, 3D rig from Stereotec
Liquid Nitrogen for freezing objects
Introduction
The PSNC used two ultra high-speed cameras capable of recording 4K video at about 1000 frames
per second to record the falling objects frozen in the Liquid Nitrogen. This speed allows us to recordand see phenomena that are invisible at normal speed. Using 60 fps recording with 1000fps camera,
we can achieve slow motion effect and can see every detail of the processes. Additionally, aninnovative approach was to synchronize two independent HFR cameras and make a 3D recording
using a special 3D Stereotec rig.
On the recordings you can see, among other things, watering of light bulbs, frozen jelly and tulips
crashing on the sheet of plexiglass and a flying mini-drone.
Reference to the content

https://www.youtube.com/watch?v=Y461szKsgK4

2.2.2.3

HighFrameRate 120p

General description
Title

HighFrameRate 120p

Date

2019

Place

Poznan, Poland

Number of clips

2 (4K 120fps)

Deliverable 5.2

Revision 1.0

11/41

Content Summary
4K recordings from Poznan recorded at 120 frames per second
Technology
RED EPIC camera
Introduction
In order to validate 120fps playback feature implemented in the the Spin Player, the PSNC teamrecorded several shots from Poznan using a RED EPIC camera. These shots were specially selected
to take advantage of the HFR effect, so the shots show objects that move very fast. The recordings
were made in two totally different places, the first place was the surroundings of the park and the
Poznań opera house, and the second was the workshop and it documents the work of the welder.
Reference to the content

https://www.youtube.com/watch?v=KwYDfvGtbRc

https://www.youtube.com/watch?v=WMMLFSJlhrE

Deliverable 5.2

Revision 1.0

12/41

2.2.2.4

Anomalia

General description
Title

Anomalia

Date

2019

Place

Poznan, Poland

Number of clips

2 (360° 8K with ambisonics sound and 8K music video clip)

Content Summary
Recording of ambisonic sound from the studio concert of Anomalia jazz band and video in 360° and
8K technologies
Technology
DPA 4099 microphones (wind instruments), condenser SE2200a microphone (tenor saxophone),dynamic Shure SM57 and condenser Audio Technica AT 2050 (guitar), Ambeo VR Mic from
Sennheiser (ambient),
Insta360 Pro camera, SONY F65 cameras, audio equipment including Midas Pro’s microphonepreamps and KlarkTeknik DN9650, Reaper DAW.
Introduction
In 2019 PSNC started experiments with ambisonic recordings and multi-channel sound projection. A
jazz band called Anomalia (members of which are students of the Ignacy Jan Paderewski Academyof Music in Poznań) was invited to take part in the project. The audio content recorded by the band
was mixed in a spherical ambisonic space created by 24 loudspeakers in the New Media Laboratoryof PSNC. In addition to the sound recordings, 8K and 360° videos were produced to show Anomalia’s
performance. The audio material was eventually encoded as an ambisonic sound of the third andfifth order, then decoded binaurally and finally combined with a 360° video. All of that required aproper workflow for the production and mixing of surround sound for the Immersify project.
Anomalia band is jazz septet with musicians playing: drums, double bass, guitar, tenor saxophone,soprano saxophone, trombone and trumpet. All recordings were taken in a sound-adapted eventroom located in the PSNC. The place was illuminated in an atmospheric way using some historic
stage lamps. We placed the members of the band in a circle, although we did not try to separate
each of the instruments. We wanted to achieve the most spatial and natural recording possible, so
different types of microphones were used.
Two video clips were produced - one traditional using SONY F65 camera, and second one as anaddition to ambisonics sound was recorded by Insta360 Pro camera.

Deliverable 5.2

Revision 1.0

13/41

Reference to the content

https://www.youtube.com/watch?v=B75AKthg2mY

https://www.youtube.com/watch?v=sYALEBPa01E

2.2.2.5

The Noise and The Structure (rooftop)

General description
Title

Deliverable 5.2

The Noise and The Structure (rooftop)

Revision 1.0

14/41

Date

Summer, 2019

Place

Poznan, Poland

Number of clips

1 (360° 16K + ambisonics)

Content Summary
Outdoor recording of ambisonics sound and the 360° video on the roof of the PSNC building using
own 360° rig
Technology
Custom 360 rig composed of 8 Blackmagic cameras
Various types of microphones
Introduction
The idea of this footage was to record ambisonic sound and 360°/VR video content in outdoor
conditions and to obtain both interesting background sound and 360° image in higher resolution
with the help of a custom constructed 360° rig consisting of 8 BlackMagic cameras. Due to the 360°
video recording, we wanted to choose a place to be visually attractive. For this purpose, we decided
to use the roof of the PSNC building , which allowed us to get amazing scenery and a lot of details.
From the musical point of view, we recorded Jan Skorupa's original contemporary composition
entitled: „The Noise and The Structure". This composition is written for string quintet (double bass,cello, viola, violin, violin), five crystal bowls and an electronic layer performed live in an eight-channel
octagonal system. All the musicians, as before, were placed in a circle, in the middle of which an
ambisonics microphone and a camera rig allowing for 360° recording at 16K resolution was set up.
Reference to the content

https://www.youtube.com/watch?v=jIFuI3zb9Xc
Deliverable 5.2

Revision 1.0

15/41

2.2.2.6

LuxFest 360

General description
Title

LuxFest 360

Date

August, 2019

Place

Poznan, Poland

Number of clips

1 (360° 8K + ambisonics)

Content Summary
360 and ambisonics recording during Luxfest rock festival in Poznan
Technology
Insta360 Pro video camera
Ambisonic microphone - Ambeo VR Mic from Sennheiser
Introduction
Luxtorpeda band was founded in Poznan in 2010 by a charismatic leader Robert 'Litza' Friedrich andnow is one of the most popular Polish rock bands. In 2011 they released their debut album; sincethat time they've released another 3 albums and the band has been awarded many times in Poland
and abroad. Luxtorpeda is also interested in new means of expression through technology.
The opportunity for collaboration was Luxfest concert in July 2019, where VR and ambisonics audio
session was recorded by PSNC. In this case, due to the lack of possibility to build our own microphone
setup, we had to base the recording only on the ambisonics microphone and sums from the mixerof the sound engineer of the concert. In this case we wanted to recreate the full stage atmosphere
where, combined with a 360° video shootings, the recipient would have the impression of finding
themselves live between the band and the audience. We placed the ambience microphone together
with the 360° camera at the edge of the stage. This setting made it possible to avoid placing the
microphone within the array of the main speaker system. The most interesting effects of spatial
sound were achieved during the singing of songs by the audience.
Due to the commercial limitations of the rights, the content may be shown only for the Immersify
project dissemination and can’t be published in Internet or other public broadcasting channels.
Reference to the content

Deliverable 5.2

Revision 1.0

16/41

2.2.2.7

Poznan Drive 3D 8K

General description
Title

Poznan Drive 3D 8K

Date

Summer, 2019

Place

Poznan, Poland

Number of clips

1 (timelapse 8K 3D)

Content Summary
Timelapse in 3D from the car drive through streets of Poznan
Technology

Deliverable 5.2

Revision 1.0

17/41

Two photo cameras Canon 5DSr + rig + car mountIntroduction
PSNC produced a short 8K 3D time-lapse by mounting two photo cameras on the car roof. The
content was recorded during the driving through the most interesting places in the city of Poznan. It
is certainly a different view of the city, where even traffic jams are not so terrible. The final clip has
been produced in 8K 3D.
Reference to the content

https://www.youtube.com/watch?v=nuM6JDElj9M

2.2.2.8

Hel Peninsula

General description
Title

Hel Peninsula

Date

September, 2019

Place

Poznan, Poland

Number of clips

3 (6K underwater, 8K 3D timelapse, 16K panoramic)

Content Summary
Underwater and aerial video clips from Hel Peninsula.

Deliverable 5.2

Revision 1.0

18/41

Technology
RED EPIC-M DRAGON (6K camera) with underwater case
SONY F65 (8K)
2 photo cameras Canon 5DSr with Canon CN-E30-300mm T2.95-3.7 L SP zoom lens + special rig
Introduction
In September 2019 the PSNC film crew took advantage to dive on the Hel Peninsula in the north of
Poland. Hel is the most sea-going part of the Polish coast, hence the surrounding waters were astrategic crossing between the Baltic Sea and the Gulf of Gdansk. Many boats, ships and otherequipment are sunk there, so these waters are the target of diving expeditions not only from Poland
but also from the world. When it turned out that one of the PSNC employees is a professional diverand organizes diving expedition, it was a great opportunity to shoot the underwater world of theBaltic Sea. For this purpose, the underwater housing had to be adapted to the camera, for which theelectronics team constructed dedicated additional focus control. The housing was also equippedwith lighting lamps. The film team together with the diver first made tests of the immersion andoperation of the camera on a 5m deep pool in Poznan. The tests were successful and the parameters
for underwater shooting were selected and tuned. After such training and preparations, it waspossible to rent a boat and go underwater near Hel town, at the very end of the peninsula. The 6K
camera from RED was inserted into the housing due to its compact dimensions. The diver managed
to descend to a depth of about 40 meters, but the best shots were taken at a depth of 25 meters. In
the underwater view recorded by camera there are sunken shipwrecks, fish and jellyfish, and even...
a bicycle.
By the way, it was possible to record 8K clips from the boat deck and from the port. The port has awonderful view of the Puck Bay and the Gulf of Gdansk, behind which you can see the ports of Gdynia
and Gdansk, about 25 km away. A drone with a 4K camera was also used in the port, thanks to whichfilms and making-of could be enriched with aerial shots. In the port, two Canon 4DSr timelapse 3D
cameras were also recorded, and on the beach, using the same setup, also 12K panoramic shots
were taken. In the latter case, in post-production, the images from both cameras were combined
into panoramas using PTGui software.
Reference to the content

https://www.youtube.com/watch?v=d-kTux84xco

Deliverable 5.2

Revision 1.0

19/41

https://www.youtube.com/watch?v=tvwzvDaMsxg

https://www.youtube.com/watch?v=-5JJs_X9iXI

2.2.2.9

Always dead and alive (Immersive Minimalism)

General description
Title

Always dead and alive (Immersive Minimalism) by Theresa Schubert

Date

Summer, 2019

Place

Poznan, Poland

Number of clips

2 (full time clip and excerpt)

Content Summary
The CGI graphics in 8K based on Cellular Automata produced as a part of STARTS residency
Technology
Deliverable 5.2

Revision 1.0

20/41

Computer generated graphics based on Cellular Automata
Introduction
Theresa Schubert's Always Dead and Alive (2019) is an 8K video environment working with cellular
automata (CA). In a coded architecture of abstract geometries, a sequence of self-generative
patterns plays out, completed and enhanced by an abstract soundscape. It is an immersive andsensorial work intended to make a digital space experienceable in an organic audiovisual flow oflight, color and sound, based on the mathematics of nature. In Always Dead and Alive, organic
processes unfold within a digital structure. Aspects of living matter (the cell) are applied to hitherto
lifeless code (the pixel). The work is an artistic exploration of the interface between life andtechnology. It applies processual powers of biologic self-organization to intangible mathematicalcode, describable in zeros and ones, demonstrating the similarities and the possible
interconnectedness of the organic and the digital. The work was created as part of a VERTIGO STARTS
residency called “Immersive Minimalism“ at PSNC by German Artist Theresa Schubert. Dedicatedsound was composed by Jan Skorupa from PSNC.
Reference to the content

Source: http://theresaschubert.com/artworks/art/always-dead-and-alive/#images-videos

2.2.2.10 Supercomputers (Theresa Schubert)
General description

Deliverable 5.2

Revision 1.0

21/41

TitleSupercomputers (Theresa Schubert)

Date

2020

Place

Poznan, Poland

Number of clips

1 (in progress)

Content Summary
The CGI graphics in 8K
Technology
Laser scanner, 8K camera, CGI
Introduction
After several visits to PSNC and also after having seen their infrastructure, Theresa Schubert started
to be fascinated by large server rooms. The huge concrete halls with server racks, blinking LEDs anddeafening noise create a very specific atmosphere. The concept for 25-min. film is to juxtapose
images of real Server Spaces, that hold the Supercomputer, but also obsolete server technology. The
story unfolds based on visual metaphors of computer networks discussed via a real supercomputer
server space and the forest as a superorganism. The film is told from a female first-person
perspective via different text-to-speech programs. Produced by the PSNC-team in native 8K and 4K
respectively it uses newest film technologies in combination with 3D laser scans and computergenerated visuals.
This film is currently (Mar 2020) in the final stage of postproduction and will be an additional
outcome of the residency.
Reference to the content

2.2.2.11 Fallout Shelter
General description
Deliverable 5.2

Revision 1.0

22/41

TitleFallout Shelter

Date

2019

Place

Poznan, Poland

Number of clips

2 (8K 3D + voiceover, 360 14K ambisonics)

Content Summary
Point cloud rendering of the fallout shelter from 1950s.
Technology
FARO Focus 3D X330
Introduction
Under a small house on the outskirts of Poznań there is an fallout shelter built in the 1950s prepared
for the Mayor of the city. During the communist times, only few people knew about the shelter andits location was a state secret. After the fall of communism, in 1989, the shelter remained unused
and, preserving its original equipment, it was transformed into a museum. The PSNC team had theopportunity to go down to the shelter and go back in time to a moment when a nuclear war seemed
inevitable. Using a laser scanner, several dozen scans were made and a complex model of the shelter
was obtained in the form of a point cloud. Using Blender software, two versions of video clips wereprepared - 8K 3D and 360 in 16K.
Reference to the content

Deliverable 5.2

Revision 1.0

23/41

2.2.2.12 Poznan Ice Festival 8K 3D
General description
Title

Poznan Ice Festival 8K 3D

Date

December, 2019

Place

Poznan, Poland

Number of clips

1

Content Summary
Timelapse in 3D showing the long process of ice carving during Poznan Ice Festival
Technology
2 Canon 5DSr + rig
Introduction
Since 2006, every year the Poznan Ice Festival has been attracting crowds becoming one of thegreatest points of wintertime in Poznań. It’s an international competition hosting guests from allover the world. Some of them come from places familiar with ice like Canada, USA or Poland but
also from countries you wouldn’t even suspect: Philippines, Malaysia, Spain.
For the PSNC film crew the festival was an opportunity to test the 3D timelapse. A few hours of ice
carving process was a perfect scenery for this. Two Canon 5DSr photo cameras placed on a special
ring and situated on a pedestal were taking a 8K 3D timelapse for about 8 hours at a temperature
close to 0 degrees Celsius. By the way, we managed to get a record of the amazing ice spider
sculpture that won the competition.
Reference to the content

Deliverable 5.2

Revision 1.0

24/41

https://www.youtube.com/watch?v=0oPjSFPxqCo

2.2.2.13 Krambabula 360
General description
Title

Krambabula 360

Date

2020

Place

Poznan, Poland

Number of clips

1 (16K 360 25fps + ambisonics)

Content Summary
Recording of ambition sound and video 360 in the modernist building Okraglak in Poznań
Technology
8xBlackmagic
MicroStudioAmbisonic microphone - Ambeo VR Mic from Sennheiser

Camera4K+

rig

Introduction
Krambabula is a folk-rock band from Poznań, which plays an energetic mixture of Slavic folk musicwith a touch of rock. The lyrics of the performed songs are in Lemko, Ukrainian and Polish. Many ofthem are songs from traditional music, sung in a so-called white voice and filtered by various musicalpassions of the group members. The PSNC crew invited the members of the band to record an

Deliverable 5.2

Revision 1.0

25/41

ambisonic sound of several songs. The whole recording was made in Okrąglak building in Poznan. In
order to record 360 in 16K resolution, the 360 rig built by PSNC was used.
This content was specially prepared for new ambisonics and immersive installation for Cannes XR
event.
Reference to the content

2.2.2.14 InterBee 2019 8K
General description
Deliverable 5.2

Revision 1.0

26/41

TitleInterBee 2019 8K

Date

November 18-20 2019

Place

Poznan-Tokyo

Number of clips

1 (8Kp60)

Content Summary
Recording from the realization of the 8Kp60 live streaming demonstration during InterBee
Technology
4 x 8K Sony cameras
3 x IP Live devices
Advantech’s VEGA real-time HEVC encoder
Spin Live Streaming software (HLS, AWS)
Spin Player as HEVC decoder and media player
Introduction
For the InterBEE 2019 (Tokyo) Immersify partners – PSNC and Spin Digital – have been collaboratingwith NHK Technologies to present a special live streaming event from Poznan to Tokyo. At the New
Media Laboratory in Poznan, PSNC hosted a unique artistic performance (live painting) that wascaptured and transmitted in 8Kp60 via public Internet. At the NHK Technologies booth in Tokyo thecontent was presented on a large 8K screen and was open for all visitors. The audience was able towatch the process of painting in an unusual technological scenery consisting of four 8K cameras anda robot. In addition, an 8K live video signal produced by NHK Technologies in Tokyo was transmitted
to Poznan and presented inside the scene of the artistic performance, creating a special sense of
remote participation.
Reference to the content

Deliverable 5.2

Revision 1.0

27/41

https://www.youtube.com/watch?v=KCqtynuu4B8

2.2.2.15 Ars Electronica Festival 2019

General description
Title

Ars Electronica Festival 2019

Date

September 5-9, 2020

Place

Poznan-Linz

Number of clips

1 (8Kp60)

Content Summary
Recording from the realization of the 8Kp60 live streaming demonstration during AEF 2019Technology

Deliverable 5.2

Revision 1.0

28/41

4 x 8K Sony cameras
3 x IP Live devices
PC server equipped with:- Blackmagic DeckLink 8K Pro- two nVidia Quadro P5000 cards
MYO Armband
Spin Player as HEVC decoder and media player
Introduction
During this festival, the Immersify consortium was involved in several different activities and one ofthem was 8K Live Streaming Demo. The demonstration used a number of new interesting solutions– from the artists’ performance, through the use of a PC and graphics cards to encode HEVC 8K,
transmitting network streams via the traditional Internet instead of a dedicated link, and finally –thanks to a low delay of about 2 seconds – remote interaction with the artists.
The performance was created by the Movement Sonification Lab, a group of artists oriented aroundartistic, research and affective aspects of feedback loops between the body in motion and soundenvironment. They apply the technologies that are able to sonificate and visualise kinetic data andshape the relationship between movement and the sound in real time. The “Receive” is aninteractive performance based on the sound improvisation between the dancer – Anna Kamińska
and the musician – accordionist Przemysław Degórski. It is the short study of the generating soundby the accordion which is mediated by the dance and movement sensors (Myo armband). The soundis processed in various ways with the motion – from becoming another disembodied musician toenhancing the analogue boundaries of the instrument timbre.
Reference to the content

https://youtu.be/STKz_HJYzlc
Deliverable 5.2

Revision 1.0

29/41

2.2.3 Clips produced with third parties
A part of the content creation and production was subcontracted to leading content producers. Clips
produced in reporting period 1 (until M12) have been described in D5.1. Following is a description ofclips produced by/with subcontractors after M12 (M13-M30).

2.2.3.1

Island in the Sky II

General description
Title

Island in the Sky II

Subcontractor

Timestorm Films

Date

May 2018

Number of clips

1

Content Summary
Timelapse video of La Palma, Spain
Technology
Timelapse with sound
8K@60fps, 4:30 min
Camera: Nikon D850, Sony A7RIII, Sony A7RII, RED Epic-W
Introduction
Almost seven years after the release of the original “Island in the Sky” timelapse film by Christoph
Malin – back then showing timelapse techniques that were revolutionary to say the least –Timestorm Films is back with a state of the art sequel. Shot in native 8K 60fps, this film tries toshowcase future display and video technologies.
Reference to the content
https://immersify.eu/content-demos/island-in-the-sky-ii/
https://youtu.be/CI8P7gMJbVg

Deliverable 5.2

Revision 1.0

30/41

2.2.3.2

Singing Sand 2.0

General description
Title

Singing Sand 2.0

Subcontractor

Tadej Droljc

Date

Feb 2019

Number of clips

1

Content Summary
Singing Sand 2.0 is a 3D stereoscopic audiovisual work in 8K resolution that was developed especiallyfor Deep Space 8K in the frame of Immersify.
Technology
8K stereoscopic CGIIntroduction
Singing Sand was inspired by the sonic potential of abstract 3D computer graphics. The central
element of the composition is a physics-based visual material whose movement is sonified in realtime. The piece explores what various particle fluctuations, tensions or shape-morphings soundlike, what internal rhythms they create and how that affects our perception of the visuals in
return.
Grains of sand resembling visual particles are sonified by mapping their individual velocities tovarious parameters of individual grains inside a custom-made granular synthesizer. At the same
Deliverable 5.2

Revision 1.0

31/41

time the velocities determine the colours of the particles which together creates a spectrum of
colours and sounds. This results in perceptually tight audiovisual bonds, which is a crucial elementof this piece as the composition focuses on the gestural qualities of the audiovisual material.
The aim of the composition was to explore how such audiovisual material could function in a freeaudiovisual paradigm and also how to meaningfully force it onto the grid of harmony and metricrhythm. In the latter case the material created liquid grooves on top of dub-tech influenced fixedrhythmic elements, as well as ever changing spectral swirls that emerged from cross-breeding an
originally noisy and non-stable sound source with pitch-based material.
The piece represents an attempt to compose from within an audiovisual paradigm where most of
the compositional decisions came as a consequence of cross-fertilisation between both modalities.
Reference to the content
https://immersify.eu/content-demos/singing-sand-2-0/
https://youtu.be/n3gSq3nQD-k

2.2.3.3

tx-reverse 360°

General description
Title

tx-reverse 360°

Subcontractor

Martin Reinhart, Virgil Widrich

Date

2019

Number of clips

1

Deliverable 5.2

Revision 1.0

32/41

Content Summary
Virgil Widrich Film- und Multimediaproduktions G.m.b.H. provided us with tx-reverse 360°, a spacetime cut through cinema. 20 years after Martin Reinhart and Virgil Widrich used this film technique
for the first time in a short film (“tx-transform”, 1998), they again deal with the question of which
previously unseen world arises when space and time are interchanged, aptly in a cinema and at full
360°.
Technology
360° stereoscopic 10k
Introduction
What is behind the cinema screen? It is not surprising that cinema-in-the-cinema scenes are often
used in horror films. For they irritate and unsettle by reminding us – the immobile viewers hiddenin the cosy darkness – of our own questionable position. What if the forces of unlimited
imagination penetrate through the canvas into our reality? What if the auditorium dissolves andwith it the familiar laws of cinema itself? In a way never before seen, "tx-reverse" shows thiscollision of reality and cinema and draws its viewers into a vortex in which the familiar order of
space and time seems to be suspended.
Back in the 1990s, Martin Reinhart invented a film technique called "tx-transform", which
exchanges the time (t) and space axis (x) in a film. Normally, each individual film frame represents
the entire space, but only a brief moment of time (1/24 second). In the case of tx-transformed
films, however, the opposite is true: each film frame shows the entire time, but only a tiny part ofthe space – in cuts along the horizontal spatial axis, the left part of the image thus becomes the"before", the right part the "after".
20 years after Martin Reinhart and Virgil Widrich used this film technique for the first time in a shortfilm ("tx-transform", 1998), they again deal with the question of which previously unseen worldarises when space and time are interchanged, aptly in a cinema and at full 360°: at the Babylon Kino
in Berlin they filmed with the OmniCam-360 about 135 actors and calculated the installation "txreverse 360°" for the ZKM from this material.
Reference to the content
https://immersify.eu/content-demos/tx-reverse-360/

Deliverable 5.2

Revision 1.0

33/41

2.2.3.4

The translucent St. Stephen’s Cathedral

General description
Title

The translucent St. Stephen’s Cathedral

Subcontractor

ScanLAB Projects

Date

March 2020

Number of clips-

Content SummaryHigh resolution interactive tour of the St. Stephen’s Cathedral in Vienna
Technology
Interactive 360° stereoscopic 8K from Point Cloud data
Introduction
With more than 20 billion laser points the cathedral church of St. Stephen in Vienna was spatiallymeasured. In this unique level of detail they correspond to a data volume of almost 400 gigabytes
and form the basis of the transparent St. Stephen's Cathedral. This innovative approach offers
spectators an interactive journey through the transparent sacred building in stereoscopic 8K. The
novel insights through the layers of the Cathedral Church were created as a cooperative effort
Deliverable 5.2

Revision 1.0

34/41

between the Ars Electronica Futurelab, ScanLAB Projects, RIEGL Laser Measurement Systems andthe Dombauhütte St. Stephan zu Wien.
Reference to the content
https://immersify.eu/content-demos/the-translucent-st-stephens-cathedral/

2.2.4 Preparing the content for final showcase
All the content produced or acquired in this task are master files in very high quality (4:2:2/RGB format,
10/12-bit). In order to show the content at the envisioned events or installations (Trade Fairs, DeepSpace 8K, large screen displays, etc.), they must be encoded and played using the Spin Digital HEVC
codec. The encoding parameters also have to be carefully selected according to the application or use
case. In other words, not all the target applications demand the same level of quality or resolution. In
D5.3 Report on QA and Content Preparation Guidelines we specify the encoding options we proposed
Deliverable 5.2
Revision 1.0
35/41

for each use case. These options are the result of subjective and objective quality analysis performedby the project partners.

3 Interactive Media Authoring
Following describes best practice examples and findings on interactive media authoring for highresolution VR infrastructures.

3.1 Deep Space 8K DevKit
The Ars Electronica Deep Space 8K Unity Development Kit (short: Deep Space Dev Kit) shall make itpossible for developers all over the world to easily create Unity applications for the Deep Space 8K, alarge-scale multiuser VR environment.
This is just a very brief overview of the possibilities and features, that are provided by the Deep Space
Dev Kit. Please have a look at the documentation for more detailed information:
https://docs.google.com/document/d/1DFtmojlEZNq_6QAa1wZN3GKHvegcunNurQoUmE6cAvs/edit

3.1.1 Interaction
The Deep Space introduces several interaction possibilities to control applications or play games. The
Dev Kit supports the following interfaces:●

VR Controller (Guides only)
The VRController is an Android application developed at the Ars Electronica Futurelab to start,
stop and control Deep Space applications. It shows all available applications in Deep Space asa list where a specific application can be started. Furthermore, it can send predefined control
messages (such as button triggers or position and rotation data) to the currently runningapplication. It implements the VRPN protocol and uses the build-in VRPN server at the DeepSpace.

●

Mobile Control (Guides only)
Each Highlight Guide has access to a mobile device to have basic control over the Deep Space
8K. The Mobile Control is an Android app consisting of generic views to interact with a DeepSpace 8K application. Therefore, a Unity Application can tell the Mobile Control, which views
and content shall be presented on the screen and the Mobile Control can tell the Unity
Application, what option has been chosen on the screen (e.g. button presses or list item
selections).

●

Pharus Laser Tracking
Six Laser Rangers track movement on the floor. Our own Tracking Software (Pharus) is
processing the measured data and provides the results (people moving in the Deep Space 8K
in real-time) as UDP Stream. A Unity application can get access to this Tracking Stream and use
the Tracks as interaction possibility.

●

Xbox Controller

Deliverable 5.2

Revision 1.0

36/41

A standard controller can be used to control an application.

3.1.2 Features
The Deep Space Dev Kit provides a range of functionalities and features that support to create
applications for the Deep Space 8K.
3.1.2.1

Project Settings

Depending on what you want to do in your Unity application, there need to setup some settings toachieve these aims. All suggested settings can be done via an integrated editor window, which can befound in the menu: DeepSpace -> Settings…
You can do all recommended settings by pressing the “Do the magic” button on the very top, or only
do selected settings by clicking the available buttons. The descriptions should be self-explaining.
Once a button was pressed and the setting was changed, the button vanishes. If there are no buttons
left to press, everything is set up as recommended.
3.1.2.2

Command Line Configuration

In general, we are using command line arguments to configure dynamic parameters. For example,
where the application runs (on the wall or on the floor), IP addresses and ports (to know how toconnect to each other or to any other hardware), or if there shall be shown a debug UI.
You can use your own command line arguments by deriving from the CmdConfigManager class andoverriding the ParseArgument Method. If you need a callback, after all parameters have been parsed,
you can implement the virtual method FinishedParsingArguments. Have a look at the
UdpCmdConfigMgr to learn more about this.
3.1.2.3

Wall-Floor-Camera

An often wanted camera view is a connected Wall-Floor-Camera, which means that a user who
stands in the so called sweet spot can look at Wall and Floor without seeing the edge between thesetwo projection planes. This effect is created by an off-axis projection.
This setup can be taken directly from the WallFloorCombinedScene which can be found in
Assets/DeepSpace/Demo/Scenes. Alternatively you can use the prepared prefabs and scripts to build
the setup yourself. The camera's sweet spot can be adopted to your needs.
3.1.2.4

Pharus Laser Tracking

One of the nicest ways to interact with an application in the Deep Space 8K is by using the Laser
Tracking System, because it is a multi-user-input-system that can handle up to 30 users at the sametime.
Our in-house developed software "Pharus" receives the data from our laser rangers, calculates current
positions of people in the room and sends this information to Unity. Thanks to the Dev Kit, a user does
not need to care about the networking that happens in the background, but can just use the positions
Deliverable 5.2

Revision 1.0

37/41

as input. To get a first impression of how these recorded data can be used in a Unity application, justopen the PharusTUIOScene or PharusTracklinkScene.
3.1.2.5

Xbox Controller

The Deep Space SDK provides a controller input manager that wraps the Unity input system forcontrollers. All functionality is packed into the ControllerInputManager class. It is a singleton and canbe accessed easily from all other script components. To make the wrapper work, the Unity Input
Settings have been already set to support all controller keys.
3.1.2.6

Mobile Control

The Mobile Control is our own proprietary Android App (client) that can be connected with a server(the Unity application on the wall) via TCP. The basic communication concept is that the server tells
the client what view shall be shown and the client tells the server what buttons have been pressed in
the currently visible view. You do not need to care about the specific protocol between these two
because all messages that can be sent and received are wrapped in the MobileControlMessages
namespace classes.
To make the beginning of using the MobileControl easier, the DevKit provides the
MobileControlManager. This class handles the communication between server and client. The public
interface for this class handles Registering and Unregistering of active and passive views, getting thecurrent or a specified view, and activating a specific or the next or last in order view.
3.1.2.7

VRPN

In addition to the aforementioned ways of communication, it is also possible to use the built-in VRPN
server in the Deep Space. The VRPN server is mainly used by VRController. As mentioned before, it is
an Android application to start, stop and control Deep Space applications. Using VRController brings
additional control possibilities such as interacting with buttons or using its virtual touchpad that
sends touch positions to Unity3d applications through VRPN protocol.
3.1.2.8

Networking

The reason why we decided to implement some networking messages ourselves was because of
performance and control. Using UDP directly is slightly faster than using UNET. You buy this little
performance by a big loss of features and comfort.
Nevertheless, you can extend and use this feature if you prefer a simple byte or JSON packaged
based network communication. Furthermore, if you are using an external server application, you caneasily communicate between your server and the unity applications via JSON packages.
The UdpManager owns the references to two senders (one sending to the Wall and the other one
sending to the Floor) and one receiver. The manager activates the right sender and the receiver
based on the configuration (UdpCmdConfigMgr) and disables the not needed sender based on theapplication type (e.g. the wall does not need to send data to the wall).

Deliverable 5.2

Revision 1.0

38/41

3.2 Dome and Interactive Content
The dome and planetarium industry are seeking new forms of immersive content and particularly nonlinear interactive content is in great demand.

3.2.1 AdventureDome
AdventureDome is an experimental immersive and interactive full dome prototype created to explore
the possibility of conducting interactive shows for a group of people where the audience can interact
simultaneously.
Guidelines:
https://docs.google.com/document/d/1gYCm99dplswd5R6lltji8RwE1OgvpTzHftl2vCu5BMg/edit
3.2.1.1

Features

The prototype is intended for children aged 5-8 years andcontains a full dome video consisting of multiple scenes, mostly
straight forward cutscenes but also scenes where the audiencecan answer trivia questions and to some extent control thecharacters and make choices to create alternative paths in realtime.
3.2.1.2

Interaction

The interaction is analogue which encourages the audience tomove and consists of holding arms up and swaying to steer thecharacters and clapping to increase characters speed.
When answering cutscenes questions, like trivia, choose paths ordecide the fate of the characters, the audience holds up cards
over their heads marked by a triangle or circle. The guide who
controls the show decides which answer the audience chooses
and then controls the video according to the decision.
The audience's movements have been greatly appreciated by thechildren and have led to collaborations and many laughs.
3.2.1.3

Production

AdventureDome was created in Unity, C# and Autodesk Maya and is primarily designed to be displayedin real time. A shorter and simplified offline rendered version is also available for playback in an HMD
and as it is developed in Unity it is flexible and can be exported to variable formats like high resolution
flat
screens.
The real-time version consists of a 360 degrees world in Unity projected to the dome’s half a spheresurface. The real-time video is a 4K 30-60 fps fisheye stream divided into 6 slices and sent out to 6
projectors. As the back-end engine does not support real-time synchronization between multiple
machines the resolution and frame-rate depends on the capabilities of the single machine running theapplication, as such, an offline version could potentially have higher resolution then the real-timeversion.

Deliverable 5.2

Revision 1.0

39/41

3.2.2 OpenSpace
Immersive environments can be constructed in various ways, such as a dome, cave, powerwall etc. The
environments usually want to achieve a very high resolution, and even today the 8K setups consideredstate-of-the-art is still considered to not have the desired pixel density in a large display environment.
It is then very demanding to produce content for these high-resolution environments, which can havevarious display setups. A lot of the content produced then is based on more interactive applications incomparison to traditional modelling and offline ray-tracing output. But then to achieve a desired
resolution in a multi display/projector scenario such softwares needs to be very flexible and highperformance. OpenSpace was built from the ground up to be a visualization software for showing thefindings and research mission conducted in space, both on desktop environments but with clear
support for any multi-display/projector with 3D/stereoscopic capabilities and running on a multimachine/cluster support. As built with these requirements from the beginning, this software has,within this project, been tailored to record and render out very high resolution clips (basically any
resolution for any display setup, with proper eye/user positions) in a reasonable amount of time.
Currently result from rendering out very high resolution fly-by of the Mars surface in fisheye view,
8Kx8K 3D, where performed on a high performance gaming machine with roughly 1 TB of frames perhour, where the bottleneck where considered to be storage/disk write speed and not CPU/GPU power.
These offline rendering capabilities were introduced as there are many immersive environments thatstill struggle with having the compute power necessary for running OpenSpace within sufficientinteractive framerate in the desired resolution, and such only support high-resolution video playback.
Close-fly-by of the Surface of Mars with high-resolution imagery was rendered with OpenSpace forvarious immersive environments in their native resolution within Immersify. Any resolution (tests withup to 16K) is possible.

4 Conclusions
In the second reporting period (M13-M30) we focused on two main areas related to the content
production in various technologies. First set of the created content was dedicated mainly for theverification of the new functionalities provided by the software developed in scope of the project,
especially SD encoder and player. PSNC and AE prepared a new set of high resolution laser scans (e.g.
Deliverable 5.2

Revision 1.0

40/41

Fallout Shelter or St. Stephen’s Cathedral) to check efficiency and quality of the encoder in case of highresolution (16K) CGI images consisting of huge amounts of not connected dots. The next set of clips
dedicated for the functional verification of the software processing modules were HFR movies (120fps
and above), 8K timelapses and 360° stereoscopic panoramas in high 16K resolution.
Some parts of those materials were also used in the second area which was promotion of the projectat several fairs and festivals (the full list of dissemination events is collected in D2.3 Dissemination and
Exploitation Report and D2.4 Final Plan for Use and Dissemination of Foreground). Additionally wecreated a set of 8K (3D) videos using typical 8K cameras and combined it with ambisonic sound toextend immersive user experience. Finally, we invited artists and individual media creators (e.g. Tadej
Droljc, Timestorm Films, Theresa Schubert) for cooperation which allowed us to produce or get licenses
for several other interesting video clips including a timelapse in 8K and several stereoscopic works in
8K and 10K (e.g Singing Sand 2.0 or The translucent St. Stephen’s Cathedral).
Using the playback infrastructures created in WP3 (mostly T3.5 Unity3D integration) and the content
from T5.1 several prototypes for interactive non-linear narratives were created. Best practice
examples and findings on interactive media authoring for high resolution VR infrastructures were
documented. These include a guideline for developers to easily create Unity applications for the DeepSpace 8K, a large-scale multiuser VR environment and also a guideline on creating interactive content
for a multi-user dome or planetarium environment using real-time game engines.

5 References

Declaration
The information, documentation and figures available in this deliverable, is written by the members of
Immersify (Audiovisual Technologies for Next Generation Immersive Media) project consortium anddoes not necessarily reflect the views of the European Commission. The European Commission is notliable for any use that may be made of the information contained herein.
This document contains original unpublished work except where clearly indicated otherwise.
Acknowledgement of previously published material and of the work of others has been made throughappropriate citation, quotation or both.

Deliverable 5.2

Revision 1.0

41/41