Package: lkg-quilt
Quilt Generator from Lightfield CapturesOverviewLooking Glass Factoryhas created a file format for exchanginglight fieldimages and video calledquilts.Lightfield images, unlike traditional 2D images, contain multiple viewpoints for display on a glasses-free 3D displays
they make. The file format uses existing image formats like PNG, JPEG, MP4, and webm, but the format describes how to
store multiple viewpoints in the same file. The trick is the store them in a grid, akaquilt.This utility script works off the principle oflinear light field captureand turns a set of images or a video, captured on alinear rail(dollying left to right)
into a quilt. The script is built on ffmpeg filtersLimitationsTheoretically this script could stitch multiple videos into a 4D (video) quilt, but there currently isn't any tech
commercially available to capture 48 or more views simultaneously. So I've only tested this with an array of images
or a single video, where the script extracts multiple frames from the video and assumes it was recorded on anautomated dolly rail.You could generate the multiple views with computer graphics, butthere are already tools to create quiltsdirectly in major game engines and web technology today, so you won't need this script then.UsageThis is a command-line tool written in Python. So if you have Python 3.12 or higher installed, you can install it with...pipinstalllkg-quiltIf you prefer to use conda as your package and environment manager...condainstalllkg-quiltGenerating Quiltsusage: lkg-quilt [-h] [-r ROWS] [-c COLUMNS] [-a ASPECT] [-W WIDTH] [-H HEIGHT] [--output OUTPUT] [--invert] 
                [--focus FOCUS] [--rail] [-v] [input ...]

Create an lightfield quilt.

positional arguments:
  input                 A list of light field capture views. File name patterns are accepted. ffmpeg supports C printf 
                        syntax for referencing multiple files. (default: frame_%04d.png)

options:
  -h, --help            show this help message and exit
  -r ROWS, --rows ROWS  Number of rows in the grid. (default: 6)
  -c COLUMNS, --columns COLUMNS
                        Number of columns in the grid. (default: 8)
  -a ASPECT, --aspect ASPECT
                        Aspect ratio for cropping the input images (width/height). (default: 0.75)
  -W WIDTH, --width WIDTH
                        Pixel width of the quilt. (default: 3360)
  -H HEIGHT, --height HEIGHT
                        Pixel height of the quilt. (default: 3360)
  --output OUTPUT       Output file name. (default: output_qs{columns}x{rows}a{aspect}.png)
  --invert              Reverse the order of the light field images. (default: False)
  --focus FOCUS         A shift of the focal plane. Larger values push into the image. (default: 0.0)
  --rail                Treat the input as a video filmed on a linear photo rail to extract views from. (default: False)
  -v, --verbose         Output more information. (default: False)Generating a quilt from sequentially named framesAssuming you've got your light field capture frames exported to JPEG files in a directory, and they are namedframe_001.jpg,frame_002.jpg, etc. This argument is passed directly to ffmpeg which supports C printf syntax.
This example will match all files prefixed withframe_followed by 3 digits before the file extension.lkg-quiltexample/frame_%03.jpgGenerating a quilt from randomly named framesAssuming you've got your light field capture frames exported but you've got clear
naming strategy, or they are out of order, or they don't sort alphanumerically
(ex.frame1.jpg,frame10.jpg,frame2.jpg, etc..)lkg-quiltframe1.jpgframeTwo.jpg003_view.jpgfourth.jpg...Generating a quilt from a videoIf you have a video that you captured, you can point to the single video file and the script will
extract the views it needs.lkg-quilt--railcapture.mp4Note for linear video capture:The script assumes the video is a capture of a dolly/track movement of constant speed,
so it's recommended that you trim out the beginning and end in a video editor so that it only includes the frames of the video that are in motion on
the camera rail.Reversing a quiltHere is how to generate a quilt if the capture was in the wrong direction (right to left instead of left to right)lkg-quilt--invertpath/to/my/frames_%02.pngGenerating a quilt for a non-portrait deviceI have a Looking Glass Portrait, so the defaults create a quilt that displays well on it's vertical 3:4 aspect ratio. You might
want to generate quilts for other aspect ratios. For example, if you have a horizontal 16:9 device you'll want to override the
aspect ratio. You might also want to consult the best combinations ofquilt settings for your device.# Looking Glass 16"lkg-quilt-a1.7778-c5-r9-W4096-H4096# Looking Glass 32"lkg-quilt-a1.7778-c5-r9-W8192-H8192# Looking Glass 65"lkg-quilt-a1.7778-c8-r9-W8192-H8192Adjusting the focal planeYour light field capture may appear blurry when displayed. This is because in a raw capture everything moves in the
capture, except for things that are infinitely far away. To fix this you can manually adjust the focal plane, which
practically means shifting the set of captured views toward the center causing a new plane of the real world to not move
from view to view, the focal plane.lkg-quilt--focus0.5Note about focus:I'm not sure what the "units" of focus mean in this context, but 0.0 keeps the focal plane infinitely far away and 1.0
moves the focal plane to where the leftmost and rightmost fields of view intersect.Tip:I don't have an interactive way to tune this focus value, so you'll need to experiment.
I can eyeball the focal plane in a quilt at a glance by looking up and down the columns to see what part of the image
isn't moving. For example, in a headshot I adjust the focus until my eyes don't change position.Values larger than 1.0 are allowed but the left and right sides of the capture will be greatly cutoff, narrowing the usable view.