Abstract:
A waveform monitor for generating a modified image from an original image includes a measuring system to select luminance frequency values of the original image that are above a first and a second threshold. A modifier changes selected pixels of the original image that fall above the first and second thresholds. In some embodiments these changed pixels are given a false color as color markers to indicate DOF and areas of sharpest focus of the original image. Methods of modifying images in this manner are also described.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims benefit of U.S. Provisional application 61/975,624, entitled MULTIPLE FOCUS-PEAKING PICTURE MARKERS ALLOWING DEPTH OF FIELD INDICATION FOR ADJUSTING CAMERA FOCUS, filed on Apr. 4, 2014, the teachings of which are incorporated herein by reference. 
     
    
     TECHNICAL FIELD 
       [0002]    This disclosure is directed to a method of displaying information on a monitor, and, more particularly, to an apparatus and method of creating a focus peaking false-color overlay that simultaneously indicates both the point of sharpest focus and the range of focus or depth-of-field (DOF). 
       BACKGROUND 
       [0003]    “On-set” or “on-site” video and film production often requires precise camera lens focus/aperture adjustment to set the focused area of the image frame and the depth-of-field (DOF) to determine how much of the image content should be focused. Often there are artistic reasons to minimize the DOF to target the viewer&#39;s attention to a particular section of the image which may not be in the center of the image. Sometimes it is possible to zoom-in on the area of attention and set the focus but this is not always possible without re-framing or when tracking focus on subjects moving to or from the camera. Also, not all lenses hold focus over the entire zoom range. 
         [0004]    Presently, both film and video production use high quality cameras with high resolution imagers of 4K pixels or more. Therefore, it has become more difficult to set focus precisely without using a large reference monitor. Unfortunately, small viewfinder-size monitors are typically all that is available on the camera or on set. These viewfinder-monitors often have a means of indicating the focused region of the image by adding a false colored marker over the displayed camera image. These are often called “focus peaking” or just “peaking” overlays. Oftentimes, even illustrating the focus peaking overlays do not provide the user with as much focus information as desired. 
         [0005]    Embodiments of the invention address these and other limitations of the prior art. 
       SUMMARY OF THE DISCLOSURE 
       [0006]    Embodiments of the invention include a real-time method of creating a focus peaking false-color overlay that simultaneously indicates both the point of sharpest focus and the range of focus or depth-of-field (DOF). 
         [0007]    Some embodiments of the invention include a waveform monitor having an image input that includes a measuring system to measure frequency information for pixels from the input image. A converter selects high-frequency values from the frequency information and a selector selects first portions of the high-frequency values as first control information and second portions of the high-frequency values as second control information. A modifier changes selected pixels of the original image that fall within the first portions and the second portions of the high-frequency values. In some embodiments these changed pixels are given a false color. 
         [0008]    Other embodiments include methods of generating a display for output from a measurement instrument. Example methods include receiving an original image formed of pixels, extracting frequency information from the pixels, and selecting a first and second range of frequencies that fall above a first and second frequency thresholds, respectively. Then, the original image is modified by changing original pixels that fall within the first range of frequencies and changing original pixels that fall within the second rage of frequencies range of frequencies to produce a modified image. In some embodiments the modifying includes falsely coloring the selected pixels. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]    The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. 
           [0010]      FIG. 1  is a block diagram of an example Video Waveform Monitor including picture markers according to embodiments of the invention. 
           [0011]      FIG. 2  is a graph showing a sample luminance spectrum output, as well as a spectrum of two low-pass-filter outputs overlaid with the luminance spectrum output, as used by embodiments of the invention. 
           [0012]      FIG. 3  is a block diagram of central components of another example Video Waveform Monitor including picture markers according to embodiments of the invention. 
           [0013]      FIG. 4A  is a base image capture that may be used by embodiments of the invention. 
           [0014]      FIG. 4B  illustrates the based image of  FIG. 4A  that has been marked with picture markers to indicate areas of focus according to embodiments of the invention. 
           [0015]      FIG. 5A  is a base image capture that may be used by embodiments of the invention. 
           [0016]      FIG. 5B  illustrates the based image of  FIG. 5A  that has been marked with picture markers to indicate areas of focus according to embodiments of the invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0017]      FIG. 1  is a block diagram showing material portions of an example Video Waveform Monitor according to embodiments of the invention. As illustrated in  FIG. 1 , a Waveform Monitor  20  is coupled to and receives input from a camera  12  that is pointed at a subject  14 . The camera  12  may produce still or moving images. The camera  12  typically includes adjustments for focus and the depth of the field of focus (DOF) that may be controlled by a camera operator, or the adjustments may be automatically controlled. The camera  12  output is input to the Waveform Monitor  20 . The output from the camera  12  may be gamma corrected, or log corrected, Y′Cr′Cb or R′G′B′ components. This correction may occur in the camera  12 , modifying signals from a linear function of the camera&#39;s light input, to a non-linear transfer function to obtain adequate dynamic range or bit-size of the output signals. Commonly used gamma transfer functions have been used in television and computer images for many years and coincidently approximate the non-linear human vision lightness function. The human vision lightness function is often approximated as a cube-root (exponent of ⅓) power-law function and many of these common gamma-correction functions (sRGB, REC-601, REC709, etc.) also follow power-laws with exponents in the range of 1/2.2 to 1/2.6. Therefore, when gamma correction is applied to the signals, the prime notation is used. As mentioned above, this correction is usually performed in the camera  12  and the camera output signal is typically R′,G′,B′ or Y′,Cb′,Cr′. Newer cameras may have both a gamma corrected proxy output as well as a high dynamic range output with log processing for recording to further compress the newer imagers high dynamic range light sensitivity. Embodiments of the invention work with any of these outputs. 
         [0018]    In the Wave Form Monitor  20 , the camera  12  output is first processed by an input processor  30  to isolate a digital luminance, Y′, of the image. The digital luminance Y′ is then passed to a fine mixer  40  as well as a coarse mixer  50 , before being passed to a first low-pass-filter  60  and a second low-pass-filter  70 , respectively. 
         [0019]    The fine mixer  40  includes a multiplexer  42  controlled by, for example, a square-wave input having a frequency of Fs/2, where Fs is the Y′ sample-rate. The coarse mixer  50  includes a multiplexer  52  controlled by, for example, a square-wave input having a frequency of Fs/4. The path through the fine mixer  40  and low-pass-filter  60  is the fine or sharp focus path, while the path through the coarse mixer  50  and low-pass-filter  70  is the coarse focus or Depth-of-Field (DOF) path. 
         [0020]    The signals driving the multiplexers  42 ,  52  are preferably square wave signals so the mixing processes are implemented as digital multiplexers selecting between Y′ and minus Y′ at the local oscillation rate (i.e., Fs/2 or Fs/4, depending on the focus path) thereby multiplying Y′ by plus and minus one by multipliers  44 ,  54 , respectively. With digital processing, the function of the multipliers  44 ,  54  to multiply unsigned input data by −1 is merely a negative sign representation of the unsigned input data, which simplifies processing. The spectrum of each mixer output with the Low Pass Filter (LPF) filter response overlaid is illustrated in  FIG. 2 . 
         [0021]    The signed mixer output from the fine mixer  40  is filtered with a low pass filter  60 , and then an absolute value of the filter output is taken in Abs element  62 . Similarly, the signed mixer output from the coarse mixer  50  is filtered with a low pass filter  70 , and then an absolute value of the filter output is taken in Abs element  72 . Each unsigned absolute value signal is then compared with separate thresholds in comparators  64 ,  74 , respectively, to develop a first binary gate to mark the image area of fine focus as well as a second binary gate signal to mark the image area of coarse or wide focus (DOF). 
         [0022]    In the example of  FIG. 1 , the original output signal from the camera  12  is optionally converted to monochrome (black and white) in a luma converter  80 , and then delayed in a delay element  82  to match the processing delay of the mixer+LPF paths. This monochrome signal is then applied to switches or multiplexers  90 ,  96  which allows a gate signal output from the respective comparator  64  or  74  to select a different false color for each gate signal. For example, as illustrated in  FIG. 1 , a red color is used for the fine focus indication on the image and yellow for the coarse focus. Alternatively, other colors or visible patterns such as crosshatch or zebra stripes could also be selected by the gate signals. 
         [0023]    The yellow switch  90  and/or red switch  96  then insert either a respective yellow or red color overlying the original portions of the monochrome image that indicate on the image a depth of focus (yellow), and areas of highest frequency of the image, which indicates the areas of tightest focus of the image. The remaining pixels, i.e., those pixels having focus levels outside of the specified focus threshold values from the original image, may be shown on the display  98  without change. If the luma conversion block  80  is present and active, then the un-modified portions of the original image will be presented as black and white. Otherwise, the un-modified portions of the original image will be in the same form as received from the camera  12 . The combined highlighted image is then sent to the display  98  for viewing. In some embodiments the display  98  may be a monitor or display that is separate from the Wave Form Monitor  20 . 
         [0024]    For example, with reference to  FIGS. 4A and 4B , the original, non-modified monochrome image is illustrated in  FIG. 4A . This is the image created from the output of the luma converter  80  in  FIG. 1 . Inspection of  FIG. 4B  shows that the DOF is overlaid or highlighted on the monochrome image in yellow, while the areas of sharpest focus are overlaid or highlighted on the monochrome image in red. The red areas (finest focus) are typically surrounded by yellow areas (coarse focus), but may have a common edge. Of course, the colors or patterns substituted for the DOF and the areas of sharpest focus are user selectable. With reference back to  FIG. 1 , the area in  FIG. 4B  illustrated in yellow originates from the yellow value input to the yellow switch  90 , while the area in  FIG. 4B  illustrated in red originates with the red value input to the red switch  96 . Different colors or patterns could be implemented by providing desired colors or patterns at the color inputs to the yellow and red switches  90 ,  96 . 
         [0025]    Assuming that the indicated red and yellow values are used, the red and yellow marker colors can either have a fixed saturation and brightness, or may instead be proportional to the input signal luminance, or could be a combination of both, before being applied to the gated switches or multiplexers  90 ,  96 . 
         [0026]    In other embodiments, the Wave Form Monitor  20  may include more than two focus zones for highlighting. For example, by including additional focus paths such as those set forth in  FIG. 1  in a Wave Form Monitor, a user may control three, four, or any desired number of separate focus zones, each producing an individual color marker. 
         [0027]    In operation, a user may use a user interface  22  to control several parameters of the Wave Form Monitor  20 . For instance filter parameters of the low pass filters  60 ,  70  may be user selectable. Additionally, a fine focus threshold input to the comparator  64  allows the user to select which areas of the image are indicated by the fine focus indicator. In some embodiments users may set the fine focus threshold so high that only the DOF is indicated on the resulting color-marked image. Users may also use the user interface  22  to select color inputs for the fine and course markers. As described above,  FIG. 1  shows these marker colors as red and yellow, but any color or pattern may be used to indicate the areas of focus. 
         [0028]      FIG. 3  is a block diagram of central components of another example Video Waveform Monitor including picture markers according to embodiments of the invention. In general, the components of  FIG. 3  add another set of filters to each of the course and fine paths to allow for both horizontal and vertical focus detection, as described below. The components of  FIG. 3  may operate with the remaining components in  FIG. 1 . For example, the Y′ input in  FIG. 3  may be the output from the input processor  30  of  FIG. 1 . The gate outputs from the OR gates  140  and  180  may be used to respectively control the red switch  96  and yellow switch  90  of  FIG. 1 , for example. 
         [0029]    The components of  FIG. 3  scan the image in both the horizontal and vertical directions, while the components of  FIG. 1  scan the image in only the horizontal direction. Scanning in both the horizontal and vertical directions allows for increased fidelity in the focus markings. 
         [0030]    Instead of including a single low pass filter  60  for the fine focus detection of  FIG. 1 , the components of  FIG. 3  include a fine horizontal low pass filter  120  as well as a vertical low pass filter  130 . The horizontal and vertical low pass filters  120 ,  130  may be coupled in parallel, or the vertical low pass filter  130  may optionally be coupled to the output of the horizontal low pass filter  120 , which makes the system more sensitive to horizontal edges rather than vertical edges. A multiplexer  110  is driven by the output of an XOR gate  112 , which preferably has square-wave inputs having frequencies of Fs/2 and Fh/2. As above, the Fs/2 is one-half the frequency of the frequency of the Y′ signal, while Fh/2 is one-half the line rate for the particular video format being analyzed. Driving the XOR gate  112  with the two signals Fs/2 and Fh/2 creates a similar output to having two separate multiplexers. In the illustrated embodiment, each of the low pass filters  120 ,  130  is passed to a respective absolute value processor  122 ,  132 , which functions as described above. 
         [0031]    In addition to the fine focus path including two low pass filters, the coarse focus path also includes a horizontal low pass filter  160  and a vertical low pass filter  170  respectively coupled to absolute value processors  162 ,  172 . The differences between the fine and coarse focus paths include the signals driving the XOR gates  112 ,  152  and the threshold settings to the comparators  124 ,  134 ,  164 , and  174 . 
         [0032]    An OR gate  140  generates the red gate signal, i.e., the control signal for adding red color, when either the frequency information of the horizontal edge or vertical edge of the image being analyzed triggered the fine focus threshold. An OR gate  180  generates the yellow gate signal when either the horizontal edge or vertical edge exceeded the coarse focus threshold. In each case the vertical and horizontal thresholds for both the fine and course focus paths are user settable, for example by using the user interface  22  ( FIG. 1 ). 
         [0033]    The Waveform Monitor  20 , or any parts of it, may be embodied in firmware, such as an FPGA, specifically designed circuitry such as an ASIC, or may be implemented by one or more software process running on one or more processors. In other embodiments the Waveform Monitor  20  may include may include a combination of components or operations running on firmware, ASICs, FPGAs, and software, for example. 
         [0034]    Although specific embodiments of the invention have been illustrated and described for purposes if illustration, it will be understood that various modifications may be made without departing from the spirit and scope of the invention.