Project Introduction
Mauricio Alvarez-Mesa (Spin Digital) & Project partners

Immersify
Audiovisual Technologies for Next Generation Immersive Media❯ Duration: 30 months: Oct 2017 – March 2020
❯ Funding: European Commission, Horizon 2020, Innovation Action❯ Budget: 2.5 million €Immersive Media

Source: Cambridge English Dictionary

Next Generation Video
VR 360°Video
SDR

HDR

FHD

4K

8K

Immersive & InteractiveExperience

DynamicRange

Resolution
Next
GenerationVideoColor
Gamut
UHD

Frame rate
Bit Depth

HD8 bit

30 fps
60 fps

10 bit

120 fpsVR & 360° Video❯ Virtual Reality: immersive andinteractive experience❯ Total 360° video requires 3X-

4X the resolution of the Field ofView❯ Stereoscopic 360 ° video

requires 2X the data rate

Quality Requirements for Next Gen VR
16K and 240 fps are needed for “true VR”
Raja Koduri, head of GPU at AMD
8K per eye is the ideal resolution for fullimmersive virtual reality
Palmer Lucky, co-founder of Oculus Rift

4K

8K

12K

16K

Per eye resolution

1Kx1K

2Kx2K

3Kx3K

4Kx4K

360° total resolution

4Kx2K

8Kx4K

12Kx6K

16Kx8K

360° 3D total resol.

4Kx4K

8Kx8K

12Kx12K

16Kx16K

Data Requirements: Uncompressed Video
FormatHD

UHD-4K

UHD-8K

16K (“True VR”)

Resolution

1920x1080

3840x2160

7680x4320

15360x15360

Framerate

3060

120

240

Video type

Planar 2D

Planar 2D

Planar 2D

Spherical + 3D

Color format

4:2:0

4:2:0

4:2:0

4:2:0

Bit depth

8-bit

10-bit10-bit10-bit

HDR

No

Yes

Yes

Yes

Raw bitrate

0.747 Gbps

7.5 Gbps

59.7 Gbps

849 Gbps

Content / 1 TB

2.97 h17.8 min

2.2 min

9.4 secProject Partners

Immersify Partners

PSNC❯ Poznan Supercomputing andNetworking Center❯ Poznan, Poland❯ 8K media lab❯ Immersive caveSpin Digital❯ Berlin, Germany❯ SME❯ Video Codecs for 8K andbeyond

Ars Electronica❯ Linz, Austria❯ Deep Space 8K❯ FutureLab R&D Center❯ Ars Electronica Festival

Marché du Film
❯ Cannes/Paris, France❯ NEXT: innovation hub at theMarché du Film

❯ VR Content and Immersive
CinemasNorrköping
Visualization Center❯ Norrköping, Sweden❯ Research & Science Center❯ 8K DomeObjectives

General Objectives
1.

To improve the quality of immersive media using advanced compression technology

2.

To enable immersive media for multiple display devices and environments

3.

To support new ways in interactive experiences by providing the required tools for

personalized and interactive non-linear storytelling4.

To promote immersive media content and tools in the creative & media industries in
Europe and worldwide

Video Codecs❯ To improve the quality of
immersive media using
advanced compression
technology❯ High quality HEVC/H.265
codec❯ Optimizations for VR & 360°
videoImmersive
Display❯ To enable immersive media for
multiple display devices andenvironments❯ Ultra-high-res: 8K & 16K❯ Player for multi-projection
systems❯ Player for Deep Space & DomeInteractive &non-linear❯ Provide the required tools forpersonalized and interactive
non-linear storytelling❯ 6 Degrees of freedom VR video❯ Ultra-HD (4K & 8K) video
textures in interactiveapplications

ImmersiveStreaming❯ Real-Time Media Server andStreaming❯ 8K VR live events

Content & Tools❯ To promote immersive media
content and tools in thecreative & media industries inEurope and worldwide

Creators andTools developers❯ Create immersive content
using new tools❯ Time-lapse photography❯ 3D laser scanning❯ CGI and animation❯ Panoramic video❯ Real-life 8K footage❯ Interactive VR

Document❯ Document best practiceguidelines for media
production workflows

Showcases❯ NEXT at Festival de Cannes❯ Ars Electronica Festival❯ IBC, NAB and InterBEE

Collaboration

Advisory Board❯ Participate in project meetings: 2018, 2019, 2020

❯ Give Feedback❯ Technical developments: objectives, milestones, and risks❯ Validate the dissemination and exploitation plan

❯ Current members❯ Kenji Sugawara. NHK Media Technology. Japan❯ Akihide Nagara. Technology Joint Corporation. Japan

Immersive content showcases❯ Innovative VR content will be produced❯ Project partners❯ External content creators❯ The content will be presented at major international events❯ Festival de Cannes - NEXT❯ Ars Electronica Festival❯ Content will be used reference for❯ Codec development and assessment❯ Best practice show cases and guidelines❯ Open for collaboration with content creators❯ Small sub-contracting possible❯ Other instruments (STARTS initiative) available

Project Advances
Oct 2017 – March 2018

WP3: Immersive Displays❯ Optimizations for next generation formats and platforms❯ Up to 16K and higher fps (120 fps)
❯ New generation Intel Xeon CPUs (Skylake-X/W/SP based platforms)❯ Multiple GPU setups❯ Optimizations of HEVC decoder and video render❯ Efficient Pixel Formats: compaction and compression❯ SIMD instructions: Intel Advanced Vector Extensions AVX 512 for Skylake CPUs
❯ Multi-GPU rendering: scaling video rendering with multiple GPUs
❯ NUMA Optimizations: highest performance in new multi-socket platforms

World First 16Kp60 Video Player❯ 16Kp60 HEVC video player❯ Resolution: 16K (15360 x 8640 px)❯ Frame rate: 60 fps❯ Platform❯ CPU: 2x Intel Xeon Scalable Platinum 8168
(2×24 cores)❯ Pixel format: HEVC 4:2:0 – 10-bit❯ GPU: 4x Nvidia Quadro M4000❯ Codec: HEVC at 400 Mbps

❯ OS: Windows 10❯ Display: 16x 4K Monitor – DisplayPort 1.2

16K Video Player Demonstrations

NAB 2018 – Las Vegas – April 20188K/16K Video Powered by Intel Xeon Scalable Processor
Joint demo with Intel Corporation

ISE 2018 – Amsterdam – January 201816K 60P Playback
Joint demo with Azlab

Thanks for your attention!
Questions?

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 762079

PSNC
Poznan Supercomputing andNetworking Center

Spin Digital Video
Technologies GmbHArs Electronica
Futurelab

info@immersify.eu

Marché du Film –Festival de Cannes

Visualization Center C