マット (映像制作)
単一の画像からデジタルでマットを抽出する

Often, it is desirable to extract two or more mattes from a single image. This process, dubbed "matting" or "pulling a matte," is most commonly used to separate the foreground and background elements of an image, and these images are often individual frames of a video file. In case of video, matting methods can use temporal relations as additional information. Compositing techniques are a relatively simple way of pulling a matte - the foreground from a greensceen scene could be imposed on an arbitrary background scene, for instance. Attempting to matte an image that doesn't use this technique is significantly more difficult. Several algorithms have been designed in an effort to address this challenge.
Ideally, this matting algorithm would separate an input video stream Irgb into three output streams: a full-color, foreground-only stream αFrgb with a pre-multiplied alpha (alpha compositing), a full-color background stream Brgb, and a single-channel stream of partial coverage of the pixels in the foreground stream. This ideal algorithm can take any arbitrary video as input, including video where the foreground and background are dynamic, there are multiple depths in the background, there exist overlaps between background and foreground share the same color and no texture, and other various features that such algorithms traditionally have some difficulty in dealing with. Unfortunately, achieving this algorithm is impossible due to the loss of information that occurs when translating a real-world scene into a two-dimensional video. Smith and Blinn formally proved this in 1996.
Matting also has some other fundamental limitations. The process cannot reconstruct parts of the background that are occluded by the foreground, and any sort of approximation will be limited. Additionally, the foreground and background of an image still have an effect on each other due to shadows being cast and light being reflected between them. When compositing an image or video from mattes of different origin, missing or extra shadows and other details of light can ruin the impact of the new image.[要出典]
The process of matting itself is a difficult problem to solve. It has been under research since the 1950s, and yet its most popular use - filmmaking - resorts to the classic but constrained compositing method. Specifically, they use a kind of global color model. This technique is based on a global color assumption; for instance, that the entire background is green. (Incidentally, this is why weather forecasters sometimes appear to have invisible ties - the color of the tie is similar to that of the background, leading the algorithm to classify the tie as part of the background stream.) Any color in theory could be used, but the most common are green and blue. Luminance matting (also called black-screen matting) is another variation of the global color model. As opposed to color, it assumes that the background is darker than a user-defined value.[要出典]
Another approach is using a local color model. This model assumes the background to be a static, previously-known image, so in this case the background stream is given. A simple matte can be pulled by comparing the actual video stream with the known background stream. Lighting and camera angle requirements are very strict unlike in global color models, but there is no restriction for possible colors in the foreground stream.[要出典]
There also exist machine learning tools that can pull mattes with the assistance of a user. Often, these tools require iteration on the part of the user - an algorithm provides a result based on a training set, and the user adjusts the set until the algorithm provides the desired result. An example of this is using a manually-created coarse matte with a trimap segmentation, so called because it separates the image into three regions: known background, known foreground, and an unknown region. In this case, the algorithm attempts to label the unknown region based on the user's input, and the user can iterate through multiple trimaps for better results. Knockout, a plug-in tool for Adobe Photoshop, is an implementation of this process.[要出典]
Another digital matting approach was proposed by McGuire et al. It makes use of two imaging sensors along the same optical axis, and uses data from both of them. (There are various ways to achieve this, such as using a beam-splitter or per-pixel polarization filters.) The system simultaneously captures two frames that differ by about half the dynamic range at background pixels but are identical at foreground pixels. Using the differences between the backgrounds of the two images, McGuire et al. are able to extract a high-resolution foreground matte from the scene. This method still retains some of the shortcomings of compositing techniques - namely, the background must be relatively neutral and uniform - but it introduces several benefits, such as precise sub-pixel results, better support for natural illumination, and allowing the foreground to be the color that a compositing technique would identify as part of the background matte. However, this means that intentionally masking something in the foreground by coating it in the same color as the background is impossible.
A third approach to digital matting is using three video streams with different focusing distances and depths of field. As with the previous method, all three image sensors share a common optical axis, though now the algorithm uses information about what part of the image is in focus in which video feed to generate its foreground matte. With this technique, both the foreground and background can have dynamic content, and there are no restrictions on what colors or complexity the background has.
All of these approaches share one notable weakness: they cannot take arbitrary videos as inputs. In video, distinct from film, Chroma key requires the background of the original video to be a single color. The other two techniques require more information in the form of synchronized but slightly different videos.