Motion Grading Comes of Age
By Chris Chinnock, Insight Media
chris@insightmedia.info

Contents
Introduction – Why is TrueCut Needed? ...................................................................................................... 2
TrueCut Building Blocks ................................................................................................................................ 4
TrueCut Overview ......................................................................................................................................... 7
TrueCut for Distribution and Playback.......................................................................................................... 9The Plan....................................................................................................................................................... 101

Pixelworks, a well respected Silicon Valley pioneer in video and display processing, has been perfecting anew technology that allows visual storytellers to finely tune the motion “look” of their titles, and then to
consistently and accurately deliver that look across different screens. This “motion grading” capability islayered onto the company’s TrueCut® content creation tool and delivery platform.
The “TrueCut Motion” tool works with any source frame rate (including 24 fps) allowing filmmakers tocreate a finely tuned judder, motion blur and frame rate appearance in post-production, on a scene-byscene basis. The “TrueCut Delivery” platform then ensures that this referenced look from the grading
suite is correctly displayed when streamed to TV sets, or played on the latest cinema screens.
The roll out is coming in mid-2020, so here is a sneak peek about what the TrueCut Motion tools andDelivery platform will offer.

Introduction – Why is TrueCut Needed?
With the explosion of more and more advanced displays, it has become increasingly difficult to delivercreative intent. Over the last 20 years we have seen a rapid expansion in the capabilities of display
solutions across a range of platforms. Not only are screens getting larger, but significant improvements
in resolution, dynamic range, color capabilities, and frame rates have become evident. Display
advancements are outpacing mainstream content formats, creating a quality gap (see Figure 1).

Figure 1: Display Capabilities are Outpacing Content Formats Creating a Quality Gap

Today, as many as nine different, separate deliverables are created for each movie release. But even thisis not enough to ensure a consistent experience across these varied platforms. TVs and mobile devices
utilize advanced display processing to adjust the image to better suit the capabilities of that display.
Pixelworks actually pioneered many of these algorithms and is well known as the picture-quality leader
in display processors for mobile and 3LCD projection devices. But TrueCut addresses the problem at thesource, providing creative controls that can take advantage of these advanced displays, and a moreadvanced delivery format that ensures consistent presentation of a new generation of titles that are
authored with the TrueCut tools.
2This is important, since the capabilities of the display and picture processing algorithms can vary widely,and, without a new solution, the format gap is being bridged, and decisions on the motion “look” arebeing made, downstream by the display device makers, not by the creatives.
Attempts to take back creative control of presentation have already started. For example, Netflix
calibrated mode and the new Filmmaker mode disable enhancements and motion processing at thedevice. Both solutions are designed to ensure that motion smoothing is turned off in particular. This
concerted effort by the DGA, TV makers and studios represents an important step forward in delivering
creative intent for existing content formats. The format gap is at least partially bridged by driving thedisplay closer to the original format for dramatic content, yet allowing the technology to still be fully
utilized for other content, video games being one example.
So how do we allow filmmakers to take advantage of these continuing improvements in display
capabilities? Creatives certainly have different opinions regarding the benefits of new technologies, butmany are interested in pushing the envelope and delivering new looks for their stories. Camera anddisplay technologies continue to advance, and studios continue to push for new “looks” for the latest
shows.
High Dynamic Range or “HDR” provides a larger palette of contrast and color options and is a great
example of how a new technology can benefit storytelling. HDR is increasingly supported on the latest
TVs, mobile devices and even in cinemas. It is fairly well accepted and even required by many studios
and distributors – it isn’t going away any time soon, even though filmmakers may use it in different
ways.
However, HDR changes the way motion appears. For example, judder and motion blur (or shutter speed)
vary a great deal between SDR and HDR, and across different display technologies and screen sizes. As aresult, HDR deliverables are often compromised in order to avoid unintended changes in motion
appearance.
TV makers added motion smoothing in an attempt to improve the user experience and reduce judder.
However, without any information regarding creative intent, the results are random and widely derided
by filmmakers. Furthermore the motion smoothing technologies used in most TVs are limited and cancause halos, pixel breakup and other artifacts.
Moving to higher frame rates will reduce judder but also results in a smoothness that most filmmakers
feel is no longer cinematic. Higher frame rates also require a faster shutter speed and have a slightly
choppy look that is often undesirable.
Some filmmakers have experimented with higher frame rates, but the results have had mixed reactions
from audiences. For example, one of the Hobbit films was captured and displayed at 48 fps but wasprocessed in such a way that many felt it looked too much like video – i.e. the “soap opera effect” –rather than like conventional cinematic content. More recently, Ang Lee shot two films at 120 fps in HDR(and 4K stereo 3D) with similar criticism but also acknowledgement that the motion look was improving.

3

Lee’s post-production process used a virtual shutter developed by RealD but required a capture at 120fps.
HFR capture involves trade-offs. Compared to 24-frame, 180-degree capture, the sensors may be
exposed for longer or shorter times depending upon the frame rate and shutter angle chosen. This canthen impact other camera settings, lens choices and lighting requirements. This is not necessarily a badthing if you have a post-production process that can create the motion look you want. Fortunately, thatsolution is now here: TrueCut Motion. TrueCut can be used with any capture rate – a significant factor
for creatives – and can be delivered to any display, preserving a referenced look for motion that isselected at the source by the filmmakers.

TrueCut Building Blocks
The TrueCut platform from Pixelworks has been developed to address the market needs discussedabove. Key to designing such a system is the input and support of filmmakers, cinematographers, postproduction professionals and more. This was the approach taken by Pixelworks to develop TrueCut.
The requirements were clear. If a motion grading solution were to be adopted, it would have to:Incorporate creative intent in the deliverable
Fit into the workflow process without major disruption
Work across a broad range of displays and playback devices
Be cost and time efficient in production and distribution

A key element in the development of the TrueCut solution was a first-of-its-kind Motion Appearance
Model, which is similar in concept to a Color Appearance Model (CAM). A Color Appearance Modelseeks to determine how a specific color in a pre-defined lighting environment will be perceived by ahuman observer when the lighting environment changes or the luminance of the color changes. The goal
is to maintain the perception of the reference hue in any environment.
The Motion Appearance Model does the same for movement. When combined with a Color Appearance
Model, a new Motion Picture Appearance Model is created. This is the first time such a model has beendeveloped.
To build a Motion Appearance Model, Pixelworks collected a dataset using expert viewers. Theseexperts looked at a variety of content to document “just noticeable differences” for motion acrossdifferent displays and viewing distances. This content featured a variety of shots with different cameraand subject motion, different contrast and peak-white levels, and different shutter speeds.
Figure 2 illustrates the judder equivalence data collected by Pixelworks. Note that judder that is
acceptable for the theatrical release for an SDR movie (graded for 48 nits or 14 foot-lamberts anddisplayed at a contrast ratio of around 1500:1 to 2000:1) can become unacceptable when played back in
the EDR (extended dynamic range) theatrical version (at 31 foot-lamberts or 108 nits) and even more soin high dynamic range on a TV with 1000 nits of peak luminance.

4One way to mitigate this is by significantly slowing down the camera pan or subject motion to speedsthat are much slower than used for SDR. Alternatively, judder can sometimes (but not always) be
reduced via laborious windowing and re-grading with lower contrast for any troublesome shots. TrueCut
allows filmmakers to avoid restrictions for motion, contrast or detail and allows adjustments for motion
to be made in post-production in much the same way as they are today with color grading in the digital
finishing process.

Figure 2; Judder Equivalence vs. Luminance for Various Camera Pan Speeds

Figure 3 illustrates the Motion/Blur data collected by Pixelworks. The idea was to compare the motion
blur of a 24 fps, 180-degree shutter angle shot, graded and played back at different peak white levels,with the motion blur of the same shot at different shutter angles played back at 48 nits. This illustrates
how a change in display brightness alters the perception of motion blur in terms of an equivalent shutter
angle.
For example, increasing the peak luminance of the screen from 48 to 100 nits reduces the perceived
motion blur (or conversely, increases motion clarity) and would require shooting with a 230-degree
shutter angle to match the perceived blur of 180 degrees at 48 nits. At 800 nits, the required shutter
angle would be 310 degrees.
As a result, it is now clear that as the luminance of the display increases, our perception of judder
increases (Figure 2) while our perception of motion blur decreases (Figure 3). If creatives want tomaintain the same perception of judder and motion blur of an SDR, 48 nits grade on displays with higher
luminance, then higher shutter angles and slower pans are required. This means a change in the originalacquisition choices if you know the content will go directly to higher luminance devices, or a robust
post-production process like TrueCut.

5Figure 3: Motion/Blur Model

But we need more for our model: how our perception of motion changes when we vary the display
refresh and frame rate, response time, dynamic range and peak white, screen size and viewing distance.
Figure 4 shows how all these factors were brought together by Pixelworks. By collecting a dataset andcharacterizing each type of target display, a new Picture Appearance Model was developed to cover awide range of display variables and incorporated into the TrueCut platform. The idea is to be able topredict what judder and motion blur will look like on displays of varying performance and size in orderto mitigate them.

6Figure 4: Adding a Motion Appearance Model to create a holistic Motion Picture Appearance Model

TrueCut Motion Tool - Overview
TrueCut Motion is a software package that has been designed to work throughout the capture-todistribution chain. The platform allows content to be captured at 24 fps or higher with the ability todeliver any motion look creatives desire.
On-set, much like a Color Decision List is created today, cinematographers can use the TrueCut tools topreview and create an initial motion look which is captured as a Motion Decision List. TrueCut Motion
can be used to deliver this motion look for dailies as well. This is particularly useful if a higher frame rate
is being used for capture, by adding a filmic look very similar to what will be used for delivery.
In post-production, TrueCut Motion is generally used after color grading. The user interface is available
integrated into Black Magic Design’s DaVinci Resolve, as well as in other forms. Motion grading isaccomplished using a reference display by adjusting the “motion-look” controls:Frame rate – defines the frame rate appearance regardless of the capture frame rate
Judder - a scale from 0 to 360, where 0 is the judder typically seen in 24 fps footage and 360 isno perceptible judder.
Motion Blur – a scale from 0 to 720, where 0 is no motion blur added to the original, and 720 is
720 degrees of effective shutter added.
An optional fourth control, Crank, can be used to adjust the speed.
7If one was created, the on-set Motion Decision List can be used as a guide in the final motion gradingsession.
As an example of what TrueCut motion grading can do, consider the composite illustrations in Figure 5.
When shot at 24fps, this scene has significant strobing in the window, and loss of clarity on the boy’sface. The objects in the foreground and background have a lot of judder. The result distracts the viewer,potentially taking the viewer out of the story and destroys any “suspension of disbelief” that may havebeen built by this point.
The HFR “motion smoothed” shot in the center frame, removes all the strobing and judder completely,
and looks very clean due to the fast shutter. However, this look is not considered “filmic” and issometimes referred to as a “soap opera effect.”
The motion graded illustration below is intended to convey one of many possible motion looks that canbe achieved with TrueCut. The strobing has been removed and the judder has been reduced to a levelthat is no longer distracting. But the shot still maintains a 24 fps film feel, and a natural level of motion
blur, even though it is being played back at a higher frame rate.

Figure 5: Illustration of TrueCut Motion Grading

To get to this point, Pixelworks has worked closely with members of the American Society ofCinematographers (ASC) as it was critical to get their input and acceptance of these new tools. Thefeedback has been very positive, and most cinematographers who have seen the tools haveacknowledged that there are many benefits to TrueCut motion grading. For example, cinematographers,
working closely with colorists, can now achieve the same motion look for both the SDR and HDR
deliverables. Furthermore, the HDR grade need not be compromised in order to avoid strobing andjudder, and this allows the colorist to use the full HDR palette of contrast choices. Without the TrueCutsolution, productions currently require tradeoffs and sometimes even expensive and time-consuming
visual effects work in order to mitigate these problems. TrueCut allows motion grading to be done close
to production, enabling a broader range of camera options and lighting choices as well.
8Since the TrueCut tools can always add motion blur and deliver a 24 frame-per-second filmic look,cinematographers have the ability to shoot at higher frame rates and with faster shutter speeds. As aresult, high frame rate allows for the broadest range of motion looks in post, even though the tools work
very well with standard, 24 fps, 180-degree shutter angle content.
For example, since TrueCut has a unique virtual shutter angle feature, motion blur for higher shutter
angles can be adjusted in post. Capturing with a fast shutter (1/120th second) allows the recreation of astandard film look (1/48th second- or 180-degree shutter) in post for one shot, or a 90-degree shutter foran adjacent shot, if desired. Often, the HDR grade can benefit from a more natural look by adding
shutter, so TrueCut allows the SDR and HDR grade to use different shutter choices. The same benefits
are available for judder.
In post-production, creatives can customize the motion look they want on a shot-by-shot basis,
providing a consistent look for each deliverable. These are tools content creators have never had.

TrueCut for Distribution and Playback
As discussed above, TrueCut can be used to deliver a consistent motion look across different theatricaland home displays. On the theatrical side, the DCPs are mastered at 48 fps for SDR, EDR and HDR (directview LED) theaters.
Pioneering efforts and early adoption have been underway in China. “Pegasus” was the first TrueCut 48
fps cinema title released in February of 2019 to over 3200 theaters in China. This was followed by therelease of “The Bravest” in broad distribution in China at 48 fps.
TrueCut has also been used to master at 300 nits/48 fps for the HDR Samsung Onyx LED screen; 108
nit/48 fps version for EDR screens, and up to 800 nits/48 fps for the Sony CLED display.
Post-production facilities using TrueCut in Beijing, China, have mastered six full length features forpresentation on these HDR, EDR and SDR cinema screens, with over 3,200 theatres certified for TrueCutin China alone.
On the home side, TrueCut is aimed squarely at streaming service providers who wish to maintaincreative intent for motion and deliver the best experience to their customers. Pixelworks has developed
a high frame-rate delivery format for TrueCut motion graded content to play back across a growing setof TVs and mobile devices.
When selecting a title, the TV or STB streaming app will play the TrueCut motion graded version if aTrueCut version of the title exists and the device is certified compatible. This is similar to the whitelisting
approach used already to determine whether to play the SDR, HDR or other formatted version of thetitle. Over 40 models of TV and over 150 models of mobile devices have been certified for TrueCutcompatibility to date.
As a side note, TrueCut’s high frame-rate format responds very well to compression, achieving the same
perceptual quality at a similar bit-rate to the 24 fps version. For example, TrueCut UHD content ladders
typically start at 18Mbps, the same bitrate used for UHD content today. One reason for the good9compression efficiency is that codecs take good advantage of the additional frames in the TrueCut
stream to improve the quality of the motion vectors.

The Plan
Pixelworks continues to work with the creative community to demonstrate the benefits that TrueCutoffers to filmmakers. At the same time, the company is working with streaming service providers andmajor TV brands to create TrueCut motion graded content and ensure that TVs are ready to accept andprocess TrueCut content. Everything should be ready in the second half of 2020 for the roll out of 4K
TrueCut content. And 8K content could follow later as the platform is also 8K capable.

10