How to Make 3D-360 Live-Action Videoby using multi action cameras

Takashi Sekitani / STEREOeYe

Different btw VR and 3D-360 video• VR – Virtual Reality
Almost CGI. VR space is made on a computer.
You can see around you 360-degree in 2D/3D,
You can move your location freely in the VRspace.
• 3D-360 video
Live-action or CGI. Almost just play a video.
You can see around you 360-degree view in 3D,
Less interaction, most 360 video can not moveyour location freely. But it can show a real view.

OUTPUTHMD (Head Mount Display)Oculus Rift

Google Cardboard

・ Field of View : Less than 120deg
・ 2D or 3D

OUTPUT
Dome theater・ Field of View : 360deg half sphere・ 2D or 3D (difficult on the Top)OUTPUT
Circle theater・ Field of View : 360deg circular
・ 2D or 3D

OUTPUTOn flat screen (such as PC , TV)

On demand view by mouse operation
Little planet view・ Field of View : various
・ 2D or 3D (Limited)INPUT
2D-360 Camera

BUBLCAM
THETA

Freedom360

・ Simple to use, Easy to use
・ Already available to consumers

INPUT
3D-360 CameraiZugar
Z6X3D

SAMSUNG
Project Beyond

GoogleJUMP・ Not easy to use yet. Not perfect yet.
・ Not general yetDifficulty in showing 3D-360 video• Difficult to keep the parallax horizontally inall direction of 360 degree, especially at Nadir
the top & bottom of view.
ex. Showing 3D-360 at dome

ex. Recording by 3D cameras

Difficulty in recording 3D-360 video
Nodal Point for perfect 360

・ Nodal point is the axis that parallax doesn't
generate even if a camera rotates.
・ In order to shoot panorama for perfect stitch,
the camera should rotate on the nodal point.

You can stitch perfectly!

Dilemma between Panorama & 3D

Ｎｏ Ｐａｒａｌｌａｘ
Ｒｕｌｅ ｆｏｒ Ｐａｎｏｒａｍａ

Ｎｅｅｄ Ｐａｒａｌｌａｘ
Ｒｕｌｅ ｆｏｒ Ｓｔｅｒｅｏ３Ｄ

for Stereo 3D - 360
This is a solution to mediate between
panorama and stereo3D

Nodal point is slightly misaligned.
But You can get 3D-360 photo with some stitch errors

Nodal point is slightly misaligned.
But You can get 3D-360 photo with some stitch errors

Nodal point is slightly misaligned.
But You can get 3D-360 photo with some stitch errors

Nodal point is slightly misaligned.
But You can get 3D-360 photo with some stitch errors

This rig is good for still photo like a chacha,
But not for video because multi cameras can not
be placed on the plate at the same time.

for 3D–360 video

4 pairs of stereo camera.
Stitch 4 left images and 4 right images separately

for 3D–360 video, other setting

8 cameras are on a plate radially
Make a stitch profile using from all images

Stitch left image by using right half of each image,Stitch right image by using left half of each image

My recent rig for 3D-360 video8 GoPro cameras are on a plate radiallyStitch left image by using right half of each image,Stitch right image by using left half of each image

Stitch left image by using right half of each image,Stitch right image by using left half of each image

8 GoPro 3D-360 rig on a tripod

Circle LED lamp for sync

A Stitching Workflow of mine
Copy all videos into computer
↓
PtGui
↓
Mask the left half on each images,
Adobe PremierCount numbers of slipped frames then save as PtGui file for right↓
eyeVideo Stitch
Import videos into ‘Video Stitch’↓PtGui
and Set the slipped frame numbers Mask the right half on each images,
to the gaps of sync
then save as PtGui file for left eye↓Video Stitch↓
Video Stitch
Extract still images for ‘PtGui’
Apply the PtGui file for right eye↓
PtGui
Then stitch out right video movie
Import the still images into ‘PtGui’
Video Stitch ↓↓PtGui
Apply the PtGui file for left eye
Stitch 360 still image by using all Then stitch out left video movie
imagesPtGui

Import the still images into ‘PtGui’ stitch softwarePtGui

Stitch 360 still image by using all images

PtGui

Mask the left half on each images

And then, save as PtGui file for right eye

PtGui

Mask the right half on each images

And then, save as PtGui file for left eyeVideo Stitch

Apply the PtGui file for right eye to VideoStitch,
Then stitch out right video movie

Video Stitch

Also, Apply the PtGui file for left eye to VideoStitch,
Then stitch out left video movieAdobe Premier

Construct final video in Over/Under or Side-by-side
format for 3D viewers

Example of equirectangular Over/Under format
for 3D-360 HMD viewers

Tips
How many cameras is the best for 3D-360
・About 865 degree of total camera FOV is needed.
865 means 360 x2(for 3D) x1.2(for stitch loss)
GoPro3 has 120degree FOV → 865/120 = 7.2
Fisheye camera 220degree FOV → 865/220 = 3.9
・Shooting with more camera with narrower lenscontributes to increase image resolution.
・Shooting with fewer camera with wider lenscontributes to reduce stitch lines.

How to make 3D-360 is still not perfect becauseof the dilemma between 'No Parallax' forpanorama and 'Need Parallax' for 3D.
However the technique and technology are grown
up time after time in this field.
So I may change my workflow in the near future.
But I hope this workshop will help you who wantto know about 3D-360 filming.
Thank you for your attention.

END