Source: http://www.google.com/patents/US7835570?dq=7557380
Timestamp: 2017-08-19 19:51:18
Document Index: 28137497

Matched Legal Cases: ['Application No. 60', 'Application No. 2003', 'art1', 'art1', 'art 2', 'art2', 'art 2', 'art2', 'art1', 'art1', 'art1', 'art1']

Patent US7835570 - Reducing differential resolution of separations - Google Patents
Certain disclosed implementations use digital image processing to reduce the differential resolution among separations or images in film frames, such as, for example, red flare. A location in the red image may be selected using information from another image. The selected location may be modified using...http://www.google.com/patents/US7835570?utm_source=gb-gplus-sharePatent US7835570 - Reducing differential resolution of separations
Publication number US7835570 B1
Application number US 11/608,556
Also published as US6947607, US7218793, US20040109611, US20060034541
Publication number 11608556, 608556, US 7835570 B1, US 7835570B1, US-B1-7835570, US7835570 B1, US7835570B1
Inventors Keren O. Perlmutter, Sharon M. Perlmutter, Eric Wang, Paul R. Klamer
Patent Citations (76), Non-Patent Citations (29), Classifications (27), Legal Events (2)
US 7835570 B1
This application claims priority from and is a continuation of U.S. application Ser. No. 11/206,182, filed Aug. 18, 2005, now U.S. Pat. No. 7,218,793, and titled “Reducing Differential Resolution of Separations,” which is a continuation of (1) U.S. application Ser. No. 10/657,243, filed Sep. 9, 2003, now U.S. Pat. No. 6,947,607, and titled “Reduction of Differential Resolution of Separations,” and (2) U.S. application Ser. No. 10/657,138, filed Sep. 9, 2003, now U.S. Pat. No. 6,956,976, and titled “Reduction of Differential Resolution of Separations,” both of which claim priority from and are continuations-in-part of U.S. application Ser. No. 10/035,337, filed Jan. 4, 2002, now U.S. Pat. No. 7,092,584, and titled “Registration of Separations,” and claim priority from U.S. Provisional Application No. 60/434,650, filed Dec. 20, 2002, and titled “Reduction of Differential Resolution of Separations.” All of the above-identified applications are incorporated herein by reference in their entirety.
Film studios may recombine the three color separations onto a single reel of color film using a photographic process that is performed in a film laboratory. In the case of three color separations that are each located on a separate reel, an optical film printer is employed to resize and reposition each source reel, one at a time. In particular, three passes are made. First, the magenta source reel is projected through an appropriate color filter onto the destination reel. Thereafter, the destination reel is rewound, the next source reel is loaded and resized, and the color filter is changed. The process is repeated until all three color separations have been printed on the single destination reel using the optical film printer. The resulting destination reel is called an interpositive (“IP”), and the colors are now represented as red, green, and blue (as opposed to cyan, magenta, and yellow).
Many implementations may be characterized as including a “where” operation and a “how” operation. The “where” operation determines where to modify an image, and may do so, for example, by determining the portion(s) at which one or more properties of an image are to be modified. The “how” operation determines how to modify the portion(s) of the image identified in the “where” operation. Either or both of the “where” and “how” operations may use, for example, frequency-based information, time-based information, or both, and the information may be, for example, intra-frame or inter-frame.
FIG. 11 is a flow chart of a process for pruning “modify” pixels.
FIG. 13 is a classification map showing “modify” pixels.
System 300 includes a digitization unit 310 that receives the three separation images C (cyan), M (magenta), and Y (yellow), and provides three digital color component images (“digital images”) RD (red), GD (green), and BD (blue). The subscript D indicates that the image is digitized. Digitization unit 310 thus performs the steps of inverting the photographic negatives (C, M, Y) into photographic positives and digitizing the resulting positives into three digital images (RD, GD, BD).
System 300 also includes a classification unit 330 that receives digital images R, G, and B from pre-processing unit 320 and provides a classification map for one of the images. For example, a classification map CR identifies one or more locations (pixels) within the red digital image for which the resolution is to be increased. CR can be represented by the same array as the red digital image, except that each pixel contains either “modify” (M), “potentially modify” (PM), or “non-modify” (NM) labels rather than an intensity value. “Modify” indicates that the pixel intensity value of the corresponding pixel location in the red digital image is to be modified; “potentially modify” indicates that the pixel intensity value might be modified, depending on, for example, further testing; and “non-modify” indicates that the pixel intensity value is not to be modified.
Process 400 includes determining descriptive criteria for the edge map for the red digital image (420) and for the edge map for the reference digital image (430). The descriptive criteria may be determined for each edge pixel in the edge or may be determined jointly for multiple edge pixels within an edge (that is, up to and including all edge pixels within the edge). Hereafter the term edge may be used to describe either one edge pixel within an edge or multiple edge pixels within an edge (that is, up to and including all edge pixels within the edge). Descriptive criteria include, for example, (i) whether the edge is a horizontal and/or a vertical edge, (ii) whether the edge transitions, in a given direction, from high to low intensity or low to high intensity—an intensity-change direction, (iii) the location of the edge, (iv) the extent of the edge, (v) the range of intensities that is traversed in the edge, and (vi) various other functions of the pixel intensities and pixel locations of the edges of the red digital image and the reference digital image.
The “edge extent” refers to the set of pixels that define the edge transition. The edge extent can be determined from a given edge using a variety of factors, such as, for example, intensity values. The edge extent of a particular edge also may be influenced by whether there are other edges in the neighborhood of the particular edge. The edge extent may be determined in one or more directions; for example, the edge extent may be determined for either one or two dimensions depending on whether the edge map contains one or two dimensional edges. An edge extent also may be defined for a single edge pixel.
Process 400 includes determining, for each edge in the edge map for the red digital image, if the edge matches an edge in the edge map for the reference digital image (440). The term “match” is used not only to indicate that the edges correspond to one another with respect to spatial location, but also to indicate that differential resolution exists between the edge of the red digital image and the edge of the reference digital image and that the edge in the red digital image is considered a candidate for modification. Other implementations may determine a “match” using other criteria, such as, for example, by considering only whether edges correspond spatially. The factors used to evaluate edges may provide information relating, for example, to a spatial relationship, to the existence of differential resolution, or to both. The determination of whether edges are matched may be made in a variety of ways, including, for example, using information about each edge individually as well as information comparing edges. In one implementation, descriptive criteria for the edges is compared by considering, for example, whether the edges have a similar direction, whether the edge extent intensity values transition in the same direction (for example, low to high, or high to low), whether the edge extent intensity values have similar intensity ranges, and whether the edges satisfy a particular distance metric (for example, whether the edges, or some designated part of the edges such as their beginnings, ends, or middles, are within a particular distance of each other).
FIGS. 5 and 6 can be used to provide an illustration of operation 440. As discussed above, the twelve edge pixels define four edges in edge map 500. In this example, each edge pixel (and its associated edge extent—in this example, the edge extents run vertically) in edge map 500 is considered in turn to determine if the edge pixel (and its associated extent) matches an edge pixel (and its associated extent) in edge map 600. In other implementations, multiple edge pixels of an edge (up to and including the entire edge) can be jointly considered to determine if the edge matches an edge in edge map 600.
FIG. 10 shows an edge map 1000 obtained after applying operation 460 to edge map 900 using a neighborhood having an extent of five in both the horizontal and vertical directions. With an extent of five in both the horizontal and vertical directions, the neighborhood around pixel (3,3) is shown by a bold outline. Continuity operation 460 changes pixel (4,2) to an M pixel, as shown, because pixel (4,2) lies between M pixel (5,1) and the M pixel under consideration—pixel (3,3). Edge map 1000 also shows with a bold outline a neighborhood having an extent of five in both the horizontal and vertical directions around pixel (7,6). However, continuity operation 460 does not result in changing any pixels in the neighborhood around pixel (7,6). Pixels (3,8) and (4,8) are not explicitly labeled NM pixels because all unlabeled pixels are understood to be NM pixels.
FIG. 15 is a table 1500 showing the results of a simplified example involving one row. Row “x” refers to the classification map and shows that row x has sixteen pixels, seven of which are M pixels (the intensity values of row x of the digital image are not shown). The seven M pixels are in positions 1-6 and 9, and the remaining pixels are NM pixels, although the NM pixels are not labeled. In general, a one-level, one-dimensional row-wise wavelet transformation of the digital image produces two subbands having a total of sixteen coefficients (not shown) that are correlated or associated with row x of the digital image. One subband represents low resolution information and the other subband represents high resolution information. A four-level, one-dimensional wavelet transformation of the digital image produces five subbands, where there is one coefficient for subband 0 (baseband), one coefficient for subband 1, two coefficients for subband 2, four coefficients for subband 3, and eight coefficients for subband 4. The positions in row x to which each of these sixteen coefficients are correlated are indicated by the numbers shown in table 1500.
In general, the wavelet transformation performed by modification unit 340 on the entire digital image produces, for each row of the digital image, “z” coefficients for the highest subband, where “z” is equal to half the number of pixels in a row, “z/2” coefficients for the second highest subband, “z/4” coefficients for the third highest subband, and so forth. The wavelet coefficients are said to be correlated or associated with a spatial region (a particular row or portion of a row in this case) of the digital image, although the resolution component identified by the wavelet coefficient will not generally be spatially limited to that region.
Process 1400 includes determining, for a specific row in the red digital image, which wavelet coefficients in the highest subband to modify (1420). In one implementation, the coefficient located at position “j” in the highest subband of the wavelet transformation is modified if position 2*j or 2*j+1 in the specific row of the classification map CR is an M pixel, where “j” is greater than or equal to zero and is measured from the beginning location of the highest wavelet subband.
Process 1400 includes determining, for the same specific row in the red digital image, which wavelet coefficients in the remaining subbands to modify (1430). In one implementation, the remaining subbands are processed from highest to lowest. The coefficient located at position “j” in the next highest subband of the wavelet transformation is modified if position 2*j or 2*j+1 in the previous (higher) subband was modified.
Third, post-processing unit 350 attempts to reduce the perceptual effects of one or more discontinuities that may exist by “feathering” (or, equivalently, smoothing) at the boundaries between M pixels and NM pixels, or between two M pixels that use different reference digital images (Ref1 and Ref2, respectively). Feathering may be applied, for example, and as explained below for several implementations, to locations containing one type of pixel (for example, to pixels on a single side of a transition boundary), or to locations containing multiple types of pixels (for example, to pixels within a neighborhood extending across a transition boundary between different types of pixels). Feathering is performed, in one implementation, along the vertical boundaries between two horizontally neighboring pixel areas and along the horizontal boundaries between two vertically neighboring pixel areas.
FIG. 16 provides a simplified example involving a portion of a single column, i. In FIG. 16, pixel (i,j) is an NM pixel and (i,j+1) is an M pixel. M1 R (i,j)=130, M1 R (i,j+1)=90, and diff=40. Also, assume diff2 is 1. Assuming “constant” has a value of 0.1, then E1=4 which includes M1 R (i,j+1) through M1 R (i,j+4).
FIG. 17 provides a two-dimensional example 1700 identifying several pixels near the NM/M border transformation that are affected by the feathering scheme described above. For ease of viewing, the NM/M pixels are not labeled. Instead, the transition between the NM/M pixels is indicated by solid bold lines. The pixels affected by the feathering scheme in both the vertical and horizontal directions are labeled as ‘S.’
As discussed briefly above, a one-dimensional wavelet transform may have been applied in only one direction by the modification unit 340 using a classification map that described edges in only one direction. In system 300, the unmodified red digital image R is used to obtain a new classification map in an orthogonal direction. The new classification map and the “old” M2 R are received by the modification unit 340 which uses the new classification map to apply a one-dimensional wavelet transform in the orthogonal direction to the old M2 R. M2 R is then modified to produce a new M1 R. The new M1 R is sent to post-processing unit 350 where a final M2 R is generated. Other implementations may combine the results of multiple passes, orthogonal or otherwise, in various ways.
The implementations and techniques described herein also may be applied to composite color images, that is, to images that have more than one color component. For example, a video image may have red, green, and blue components combined into one “composite” image. These components may be separated to form separations, and one or more of the implementations and techniques described herein may be applied to the separations.
The digital images described above include information that generally spans the same object. For example, the red, blue, and green digital images each contain information (red information, blue information, or green information) that spans the entire frame. Such information (red, blue, and green) can be termed “complementary” information because the information relates to the same object. Other implementations may use digital images that span a different object, such as, for example, an area in a scene being filmed, a portion of a body, or a portion of the sky.
US4171871 Jun 30, 1977 Oct 23, 1979 International Business Machines Corporation Achromatic unit magnification optical system
US4541688 Jun 11, 1984 Sep 17, 1985 Eastman Kodak Company Optical beam splitters
US4783591 Nov 9, 1987 Nov 8, 1988 Honeywell Inc. Color mark sensor
US4924521 Dec 18, 1987 May 8, 1990 International Business Machines Corporation Image processing system and method employing combined black and white and gray scale image data
US5414782 Dec 16, 1992 May 9, 1995 The United States Of Amercia As Represented By The Secretary Of Commerce Procedure for digital image restoration
US5777799 Jul 19, 1996 Jul 7, 1998 Olympus Optical Co., Ltd. Three-lens-group zoom lens system having a zoom ratio exceeding 3.5
US6002434 Jan 14, 1997 Dec 14, 1999 Matsushita Electric Industrial Co., Ltd. Registration correction waveform determination method and system for a television camera
US6017688 Jan 27, 1998 Jan 25, 2000 Applied Science Fiction, Inc. System and method for latent film recovery in electronic film development
US6169562 Sep 3, 1998 Jan 2, 2001 Fuji Photo Film Co., Ltd. Image exposure apparatus
US6449414 Oct 27, 1997 Sep 10, 2002 Mitsubishi Rayon Co., Ltd. Low chromatic aberration rod lens array, its unit and image scanner employing them
US20010030697 Mar 5, 2001 Oct 18, 2001 Dischert Lee Robert Imager registration error and chromatic aberration measurement system for a video camera
US20020106133 Jan 18, 2002 Aug 8, 2002 Applied Science Fiction, A Delaware Corporation Method and system for altering defects in a digital image
US20020146171 Sep 28, 2001 Oct 10, 2002 Applied Science Fiction, Inc. Method, apparatus and system for black segment detection
1 "Aberrations," reprinted from http://hyperphysics.phy-astr.gsu.edu/hbase/geoopt/aberrcon.html on Nov. 21, 2002, 2 pages.
2 "Announcing the End of Dirty Images," Digital ICE Applied Science Fiction, reprinted from http://www.asf.com/products/ice/FilmICESpecs.shtml on Apr. 15, 2003, 3 pages.
3 "Automatically Eliminates Surface Defects," Digital ICE Applied Science Fiction, reprinted from http://www.asf.com/gallery/ice/film/ on Apr. 15, 2003, 1 page.
4 "Chromatic Abberation," reprinted from http://hyperphysics.phy-astr.gsu.edu/hbse/geoopt/aber2.html on Nov. 21, 2002, 3 pages.
5 "Chromatic Aberration" reprinted from http://www.yorku.ca/eye/chraber.htm on Nov. 21, 2002, 1 page.
6 "Digital ICE for Film Scanners: Frequently Asked Questions," Digital ICE Applied Science Fiction, reprinted from http://www.asf.com/support/scanners/FilmICEFAQs.shtml on Apr. 15, 2003, 2 pages.
7 "Digital ICE-Film Scanner Technology," Digital ICE Applied Science Fiction, reprinted from http://www.asf.com/products/ice/FilmICEExample.shtml on Apr. 15, 2003, 2 pages.
8 "Digital ICE—Film Scanner Technology," Digital ICE Applied Science Fiction, reprinted from http://www.asf.com/products/ice/FilmICEExample.shtml on Apr. 15, 2003, 2 pages.
9 "Increased Demand for Digital Images," Digital ICE Applied Science Fiction, reprinted from http://www.asf.com/products/ice/FilmICEOverview.shtml on Apr. 15, 2003, 2 pages.
10 "Technicolor," reprinted from http://www.everything2.com/index.pl?node=Technicolor, on Dec. 18, 2002, 3 pages.
11 "The Wide Screen Time Machine: Mechanix Illustrated Mar. 1939, How the Technicolor Camera Works," reprinted from http://www.widescreenmuseum.com/widescreen/mi-techcamera.htm, on Dec. 18, 2002, 3 pages.
12 Askey, P., "Chromatic Aberrations," DPReview.Com, © 1998-2001, reprinted from http://www.dpreview.com/learn/Glossary/Optical/Chromatic-Aberrations-01.html on Nov. 21, 2002, 3 pages.
13 Askey, P., "Chromatic Aberrations," DPReview.Com, © 1998-2001, reprinted from http://www.dpreview.com/learn/Glossary/Optical/Chromatic—Aberrations—01.html on Nov. 21, 2002, 3 pages.
14 Ball, J.A., "The Technicolor Process of Three-Color Cinematography," Journal of Motion Picture Engineers, vol. XXV, No. 2, Aug. 1935, pp. 127-138.
15 Boult, T.E., and Wolberg, G., "Correcting Chromatic Aberrations Using Image Warping," Proceedings of DARPA92, pp. 363-377, 1992.
16 Brown, Lisa G., "A Survey of Image Registration Techniques," ACM Computing Surveys, vol. 24, No. 4, pp. 325-376, 1992.
17 Canny, John, "A Computational Approach to Edge Detection," IEEE Tranactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, No. 6, Nov. 1986, pp. 679-698.
18 Japanese Office Action issued in Japanese Application No. 2003-559159, mailed Jun. 9, 2009, 4 pages.
19 Li, H., et al., "A Contour-Based Approach to Multisensor Image Registration," IEEE Transactions on Image Processing, vol. 4, No. 3, Mar. 1995, pp. 320-334.
20 Polikar, Robi, "Fundamental Concepts & an Overview of the Wavelet Theory," The Wavelet Tutorial Part I, Rowan University, College of Engineering Web Servers, Jun. 5, 1996, 15 pages, reprinted from http://engineering.rowan.edu/~polikar/WAVELETS/Wtpart1.html on Nov. 21, 2002.
21 Polikar, Robi, "Fundamental Concepts & an Overview of the Wavelet Theory," The Wavelet Tutorial Part I, Rowan University, College of Engineering Web Servers, Jun. 5, 1996, 15 pages, reprinted from http://engineering.rowan.edu/˜polikar/WAVELETS/Wtpart1.html on Nov. 21, 2002.
22 Polikar, Robi, "Fundamentals: The Fourier Transform and the Short Term Fourier Transform," The Wavelet Tutorial Part 2, Rowan University, College of Engineering Web Servers, 17 pages, reprinted from http://engineering.rowan.edu/~polikar/WAVELETS/Wtpart2.html on Nov. 21, 2002.
23 Polikar, Robi, "Fundamentals: The Fourier Transform and the Short Term Fourier Transform," The Wavelet Tutorial Part 2, Rowan University, College of Engineering Web Servers, 17 pages, reprinted from http://engineering.rowan.edu/˜polikar/WAVELETS/Wtpart2.html on Nov. 21, 2002.
24 Polikar, Robi, "Multiresolution Analysis & the Continuous Wavelet Transform," The Wavelet Tutorial Part III, Rowan University, College of Engineering Web Servers, 28 pages, reprinted from http://engineering.rowan.edu/~polikar/WAVELETS/Wtpart1.html on Nov. 21, 2002.
25 Polikar, Robi, "Multiresolution Analysis & the Continuous Wavelet Transform," The Wavelet Tutorial Part III, Rowan University, College of Engineering Web Servers, 28 pages, reprinted from http://engineering.rowan.edu/˜polikar/WAVELETS/Wtpart1.html on Nov. 21, 2002.
26 Polikar, Robi, "Multiresolution Analysis: The Discrete Wavelet Transform," The Wavelet Tutorial Part IV, Rowan University, College of Engineering Web Servers, 10 pages, reprinted from http://engineering.rowan.edu/~polikar/WAVELETS/Wtpart1.html on Nov. 21, 2002.
27 Polikar, Robi, "Multiresolution Analysis: The Discrete Wavelet Transform," The Wavelet Tutorial Part IV, Rowan University, College of Engineering Web Servers, 10 pages, reprinted from http://engineering.rowan.edu/˜polikar/WAVELETS/Wtpart1.html on Nov. 21, 2002.
28 Spencer, G., et al., "Physically-Based Glare Effects for Digital Images," SIGGRAPH 95 Conference Proceedings, Annual Conference Series, ACM SIGGRAPH, Aug. 1995, pp. 325-334.
29 Srinivasan, S., et al., "mist: Multispectral Image Similarity Transformation," LNK Home Page, pp. 1-20, printed Dec. 12, 2001.
U.S. Classification 382/162, 382/266, 382/275, 348/625
International Classification G06T5/00, G06T3/40, H04N5/262, H04N1/58
Cooperative Classification G06T7/13, G06T7/12, G06T3/403, H04N1/58, G06T3/4084, G06T5/50, G06T2207/20064, G06T5/10, G06T2207/20192, G06T5/003, G06T2207/10024, H04N5/262, G06T2207/10016
European Classification H04N1/58, G06T3/40T, G06T3/40E, G06T7/00S2, G06T5/00D, H04N5/262
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AOL LLC;REEL/FRAME:024549/0335
Free format text: CHANGE OF NAME;ASSIGNOR:AMERICA ONLINE, INC.;REEL/FRAME:024549/0255
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KLAMER, PAUL R.;PERLMUTTER, KEREN O.;PERLMUTTER, SHARON M.;AND OTHERS;SIGNING DATES FROM 20040115 TO 20040129;REEL/FRAME:024549/0239