Source: http://www.google.com/patents/US8086067?dq=6,192,476
Timestamp: 2016-08-30 08:24:30
Document Index: 672056264

Matched Legal Cases: ['Application No. 60', 'art 600', 'art 600', 'art 600', 'art 600', 'art 700', 'art 700', 'art 700', 'art 700', 'art 800', 'art 800', 'art 800', 'art 800', 'art 800', 'art 900', 'art 900', 'Application No. 2', 'Application No. 06814256', 'Application No. 06814256']

Patent US8086067 - Noise cancellation - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA technique for reducing noise in a digital video signal is disclosed. In one embodiment, the technique involves receiving a digital signal. The digital signal can be filtered thereby generating a filtered signal. The digital signal and the filtered signal can be mixed according to a composite blend...http://www.google.com/patents/US8086067?utm_source=gb-gplus-sharePatent US8086067 - Noise cancellationAdvanced Patent SearchPublication numberUS8086067 B2Publication typeGrantApplication numberUS 11/941,050Publication dateDec 27, 2011Priority dateDec 20, 2006Fee statusPaidAlso published asUS20080152253, WO2008076566A1Publication number11941050, 941050, US 8086067 B2, US 8086067B2, US-B2-8086067, US8086067 B2, US8086067B2InventorsLaurence A. ThompsonOriginal AssigneeSilicon Image, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (86), Non-Patent Citations (39), Referenced by (1), Classifications (15), Legal Events (6) External Links: USPTO, USPTO Assignment, EspacenetNoise cancellation
US 8086067 B2Abstract
A technique for reducing noise in a digital video signal is disclosed. In one embodiment, the technique involves receiving a digital signal. The digital signal can be filtered thereby generating a filtered signal. The digital signal and the filtered signal can be mixed according to a composite blend map thereby generating an optimized signal. The optimized signal can be provided as an output.
1. A method for reducing noise in a digital video signal comprising:
receiving a digital signal, filtering the digital signal thereby generating a filtered signal, mixing the digital signal and the filtered signal according to a composite blend map thereby generating an optimized signal, outputting the optimized signal, wherein a mixing includes retaining one or more non-noisy regions of the digital signal that do not contain noise, and replacing one or more noisy regions that contain noise with the filtered signal, wherein mixing is performed based a composite blend map, wherein the composite blend map is generated by subtracting an edge detected signal from a proximity map, and wherein the composite blend map illustrates the one or more noisy regions that contain noise, wherein the one or more noisy regions are replaced with the filtered signal to generated an optimized signal.
applying an edge detector to the filtered signal thereby generating the edge detected signal, applying a predictor to the edge detected signal thereby generating the proximity map.
3. A method as recited in claim 2, wherein the edge detector generates the edge detected signal by calculating a gradient of the filtered signal by taking a first derivative of change in luminance in the filtered signal.
4. A method as recited in claim 2, wherein the predictor includes a 17.times.17 block and applying the predictor involves passing the 17.times.17 block over each location corresponding to a pixel location of the edge detected signal.
5. A method as recited in claim 4, wherein the 17.times.17 block has a center capable of detecting the value of first derivatives in the edge detected signal, the 17.times.17 block being further capable of selecting the largest edge detection value within the pixel locations covered by the 17.times.17 block and assigning that value to the location corresponding to the center of the 17.times.17 block in the proximity map.
modifying the composite proximity map in order to further reduce noise thereby generating a modified composite proximity map.
modifying the edge detected signal in order to improve selectivity of filtered areas and disqualify areas where filtering is inappropriate.
8. A method as recited in claim 1, wherein the digital signal includes compression artifacts and the optimized signal has a reduced amount of compression artifacts.
9. A method as recited in claim 1, wherein the filtering step involves applying a low pass filter to the digital video signal.
10. A method as recited in claim 1, wherein the filtering step involves applying a Gaussian blur to the digital video signal.
11. A system for removing noise from a digital signal, the system comprising:
a filter capable of receiving and filtering a digital signal, a composite blend map generator capable of receiving the digital signal and generating a composite blend map based on the digital signal, a mixing module capable of mixing the digital signal and the filtered signal according to the composite blend map, wherein mixing includes retaining one or more non-noisy regions of the digital signal that do not contain noise, and replacing one or more noisy regions that contain noise with the filtered signal, wherein mixing is performed based a composite blend map, wherein the composite blend map is generated by subtracting an edge detected signal from a proximity map, and wherein the composite blend map illustrates the one or more noisy regions that contain noise, wherein the one or more noisy regions are replaced with the filtered signal to generated an optimized signal.
an edge detector capable of receiving the filtered signal and detecting edges within the filtered signal, a proximity map generator capable of receiving the edge detected signal and generating the proximity map based on the edge detected signal.
a proximity map modifier capable of modifying the proximity map before subtracting the edge detected signal.
a edge detection modifier capable of modifying the edge detected signal before subtracting from the modified proximity map.
a proximity map modifier capable of modifying the proximity map signal before subtracting the edge detection signal. Description
This application claims priority to U.S. Provisional Patent Application No. 60/876,223, filed Dec. 20, 2006, and entitled “Noise Cancellation,” by Laurence A. Thompson, and is hereby incorporated herein by reference.
The consumer video industry has been undergoing significant changes over the past few years due to the transition from analog to digital transmission and storage and the adoption of new video formats, including high definition. In parallel, digital display technologies are rapidly converting from the old CRT to new digital technologies, including LCD, plasma, and DLP.
Consumer expectations for video image quality are rising and digital high definition displays are capable of resolving increasingly fine details. For this reason, consumers are becoming less tolerant of noise and distortion in video images.
The term “noise” is used informally as a catch-all term that when applied to video images means almost anything that looks unnatural or diminishes the clarity of the video image. The term has traditionally been used to describe stray or random electrical signals that are imposed on a video signal. A characteristic of this type of noise is that the unwanted noise signal is uncorrelated to the video signal. Traditional techniques for removing this type of uncorrelated noise include temporal averaging, in which multiple video frames are averaged to diminish the appearance of noise. This type of temporal averaging requires motion detection, because the temporal averaging must be applied only to areas of the video where there is no motion to prevent motion blurring. This also limits the noise suppression to areas in the video image where there is no motion.
Increasingly, video signals are transmitted digitally. Digital transmission requires the use of digital compression to reduce transmission bandwidth and storage requirements. In the United States, the FCC has mandated the MPEG2 video compression standard for digital terrestrial broadcast. Cable and satellite providers may use MPEG2 or they may use other standards, such as H.264. Most video compression standards are “lossy.” This means that the compressed video is not identical the pre-compressed video. As the compression ratio is increased, the lossy compression standards result in increasing distortion in the compressed video.
The distortion introduced by video compression is also informally referred to as “noise.” But it is actually an artifact of the video compression processing. It is unlike the random noise described earlier in that it is correlated to image details or to rapid motion. Since it is correlated to the image content of the video, the temporal averaging technique described above is not effective in removing noise due to compression processing because this type of noise is correlated to motion of feature elements of the video image.
Compression processing introduces a number of distortions in the compressed video signal. The most common types of distortion are referred to as “block noise” and “mosquito noise.” Both of these types of noise are objectionable to a viewer because they are clearly unnatural in appearance.
Block noise typically appears in compressed video in areas of rapid motion. If the video compression and transmission system can not provide enough new information to update rapid motion, then an entire DCT block of pixels may be temporarily assigned a single color. The term “DCT” means “Discreet Cosine Transform” which is a mathematical operation used in most compression standards. The size of a DCT block of pixels in the MPEG2 compression standard is 8�8 pixels, so in areas of rapid motion, an 8�8 block of pixels can be assigned a single color. This results in the temporary appearance of blocks of 8�8 in the video image.
Mosquito noise is another artifact of lossy compression processing. Mosquito noise appears in the video image as small dots or distortions in the luma value of pixels that are near the edges of objects. Object edges convert to high frequencies by the DCT process. Mosquito noise is the result of course quantization of the higher frequency components of a video image as a result of compression processing. Mosquito noise will appear in close proximity to object edges. Mosquito noise will be distributed within the same 8�8 block as the pixels that make up the object edge. This bounds the area where mosquito noise is visible.
In addition to the block noise and mosquito noise artifacts described previously, compression processing tends to remove detail from video images.
What is needed is a method that can remove compression artifacts and enhance video images.
A technique for reducing noise in a digital video signal involves the use of a composite blend map. An example of a method according to the technique involves receiving a digital signal. The digital signal can be filtered thereby generating a filtered signal. The digital signal and the filtered signal can be mixed according to the composite blend map thereby generating an optimized signal. The optimized signal can be provided as an output.
In alternate embodiments, an edge detector can be applied to the filtered signal thereby generating an edge detected signal. A predictor can be applied to the edge detected signal to generate a proximity map. The edge detected signal can be subtracted from the proximity map to generate the composite blend map. In some embodiments, the edge detector can generate the edge detected signal by calculating a gradient of the filtered signal by taking a first derivative of change in luminance in the filtered signal.
In additional embodiments, the predictor can include a 17�17 block. Applying the predictor can involve passing the 17�17 block over each location corresponding to a pixel location of the edge detected signal. For instance, in certain embodiments, the 17�17 block can have a center that is capable of detecting the value of first derivatives in the edge detected signal. The 17�17 block can also be capable of selecting the largest edge detection value within the pixel locations covered by the 17�17 block and assigning that value to the location corresponding to the center of the 17�17 block in the proximity map.
In other embodiments, the composite proximity map can be modified in order to further reduced noise. In additional embodiments, the edge detected signal can be modified in order to improve selectivity of filtered areas and disqualify areas where filtering is inappropriate. In further embodiments, the filtering step can involve applying a low pass filter or a Gaussian blur to the digital video signal.
An example of a system according to the technique includes a filter, a composite blend map generator and a mixing module. The filter can be capable of receiving and filtering a digital signal. The composite blend map generator can be capable of receiving the digital signal and generating a composite blend map based on the digital signal. The mixing module can be capable of mixing the digital signal and the filtered signal according to the composite blend map.
In further embodiments, the system can further include an edge detector and a proximity map generator. The edge detector can be capable of receiving the filtered signal and detecting edges within the filtered signal. The proximity map generator can be capable of receiving the edge detected signal and generating a proximity map based on the edge detected signal. The composite blend map can be generated by subtracting the edge detected signal from the proximity map.
An example of a method for enhancing a digital signal involves receiving a digital signal. The digital signal can be filtered thereby generating a filtered signal. The filtered signal can be subtracted from the digital signal thereby generating a modified signal. The modified signal can be added to the digital signal thereby generating an optimized signal which can be provided as an output.
The proposed system, method and device can offer, among other advantages, the reduction of compression artifacts including mosquito noise reduction and general noise reduction. Further, the proposed system, method and device can generally enhance signal quality and qualifiedly enhance signal quality. Advantageously, the proposed system, method and device can reduce noise and improve signal quality on a frame by frame basis. These and other advantages of the present invention will become apparent to those skilled in the art upon a reading of the following descriptions and a study of the several figures of the drawings.
FIG. 1 depicts an example of a system for optimizing a digital signal.
FIG. 2 depicts an alternate example of a system for optimizing a digital signal.
FIG. 3 depicts an alternate example of a system for optimizing a digital signal.
FIG. 4 depicts an example of a system for enhancing a digital signal.
FIG. 5 depicts an alternate example of a system for enhancing a digital signal.
FIG. 6 depicts a flowchart of an example of a method for optimizing a digital signal.
FIG. 7 depicts a flowchart of an alternate example of a method for optimizing a digital signal.
FIG. 8 depicts a flowchart of an alternate example of a method for optimizing a digital signal.
FIG. 9 depicts a flowchart of an alternate example of a method for optimizing a digital signal.
FIG. 10 depicts an alternate embodiment of the present invention.
FIG. 1 depicts a system 100 for optimizing a digital signal. In the example of FIG. 1, the system 100 includes a filter 102, a composite blend map generator 104, and a mixing module 106. The filter 102 is coupled to the composite blend map generator 104 and the mixing module 106. The composite blend map generator 104 is also coupled to the mixing module 106.
In operation, the filter 102 receives a digital signal as an input. The digital signal can be any convenient and/or known digital signal, including, but not limited to, a digital video signal. Further, the digital signal can be a compressed digital signal using any convenient and/or known compression technique, including for example and not limitation, MPEG2, MPEG4, and/or any other lossy or lossless compression scheme. The filter 102 can be any known and/or convenient filter capable of generating a filtered signal. In one embodiment, the filter 102 can be a low-pass filter which blocks the high frequency content of the digital signal. In other embodiments, the filter 102 can apply a Gaussian blur to the digital signal to generate the filtered signal.
The filtered digital signal is provided to the composite blend map generator 104. The composite blend map generator 104 utilizes the filtered digital signal to generate a composite blend map. The composite blend map can be generated in any convenient and/or known manner capable of generating a map for mixing the digital signal and the filtered digital signal. For example, and not limitation, the digital signal can be analyzed to determine the location of noise in the signal. In certain embodiments, the noise can be correlated with the brightness which can be found around the edges of an object in the digital signal due to the quantization of high frequencies.
The mixing module 106 can receive the composite blend map from the composite blend map generator 104 as well as the digital signal and the filtered digital signal. The mixing module 106 can mix the digital signal and the filtered digital signal according to the composite blend map and output an optimized signal. For example, and not limitation, the mixing module 106 can retain all the regions in the digital signal that do not contain noise and replace all regions that contain noise with the filtered digital signal. In this example, the composite blend map is used to determine the regions that contain noise and those without noise. In other embodiments, the determination of noisy regions can be facilitated using any known and/or convenient technique and the result can be used to mix the digital signal and the filtered signal.
FIG. 2 depicts a system 200 for optimizing a digital signal. In the example of FIG. 2, the system 200 includes a filter 202, an edge detector 204, a predictor 206, a subtractor 208, and a mixing module 210. The filter 202 is coupled to the edge detector 204 and the mixing module 210. The edge detector 204 is coupled to the predictor 206 and the subtractor 208. The predictor 206 is also coupled to the subtractor 208. The subtractor 208 is coupled to the mixing module 210.
In operation, the filter 202 receives a digital signal. The digital signal is filtered by the filter 202 and provided to the edge detector 204 and the mixing module 210. The edge detector 204 detects the edges of the objects in the filtered digital signal and provides the edge detected signal to the predictor 206 and the subtractor 208.
In certain embodiments, the predictor 206 generates a proximity map in order to determine the regions with unwanted signals within the digital signal. In certain embodiments, the predictor 206 includes a 17�17 block which is passed over each location corresponding to a pixel location of the edge detected signal. In one embodiment, the 17�17 block can be capable of detecting the value of the first derivatives in the edge detected signal. Following this example, the 17�17 block can select the largest edge detection value within the pixel location covered by the 17�17 block and assign the value to the location corresponding to the center. The result of the example is a proximity map of regions with unwanted signals.
As can be seen from the forgoing example, the regions with unwanted signals are in proximity to the detected edges in the digital signal. In other embodiments, any convenient and/or known technique to detect regions of unwanted signals can be utilized. For example, and not limitation, the proximity map can be generated by selecting all pixels within a predetermined threshold of the edges detected in the signal. In alternate embodiments, other attributes, such as object shape or movement rather than edges, can be utilized to determine unwanted signals.
The subtractor 208 receives the proximity map and the edge detected signal. In operation, the subtractor 208 subtracts the edge detected signal from the proximity map. The result can be a composite blend map. In this example, the composite blend map illustrates the areas in which noise is likely to be found. In other embodiments, the subtractor 208 can be replaced with any convenient and/or know device capable of producing a composite blend map. For example, and not limitation, the subtractor 208 can be replaced with an adder. In such an example, the edge detected signal can be inversed and combined with the proximity map to generate the composite blend map.
The mixing module 210 receives the composite blend map from the subtractor 208 as well as the digital signal and the filtered digital signal. The mixing module 210 mixes the digital signal with the filtered digital signal according to the composite blend map in order to generate an optimized signal. In one embodiment, the mixing module 210 replaces all areas in the digital signal that have been indicated on the composite blend map to contain noise with the filtered digital signal. In other embodiments, the mixing module 210 can generate an optimized signal using any convenient and/or known technique. For example, and not limitation, the mixing module 210 can include an additional input which modifies the extent to which the digital signal and the filtered digital signal are combined to generated the optimized digital signal.
FIG. 3 depicts a system 300 for optimizing a digital signal. In the example of FIG. 3, the system 300 includes a filter 302, an edge detector 304, a predictor 306, an edge detector modifier 308, a proximity map modifier 310, a subtractor 312, and a mixing module 314. The filter 302 is coupled to the edge detector 304 and the mixing module 314. The edge detector 304 is coupled to the predictor 306 and the edge detector modifier 308. The predictor 306 is coupled to the proximity map modifier 310. The edge detector modifier 308 and the proximity map modifier 310 are coupled to the subtractor 312. The subtractor 312 is coupled to the mixing module 314.
In the example of FIG. 3, the edge detector 304 detects the edges in the filtered signal received from the filter 302. The edge detector 304 provides the edge detected signal to the edge detector modifier 308. The edge detector modifier 308 can adjust the edge detected signal according to any convenient and/or known parameters. For example, and not limitation, the edge detector modifier 308 can be adjusted by the user and/or adjusted automatically.
Further, the edge detector modifier 308 can perform any convenient and/or known function. In one embodiment, the edge detection modifier can further enhance the detection of edges in the filtered digital signal. In another embodiment, the edge detection modifier can suppress the detection of edges in the filtered digital signal. In further embodiments, the edge detection modifier can selectively enhance and suppress edges in the filtered signal.
Additionally, in the example of FIG. 3, the predictor 306 generates a proximity map utilizing the edge detected signal provided by the edge detector 304. The predictor 306 provides the proximity map to the proximity map modifier 310. The proximity map modifier 310 can adjust the proximity map according to any convenient and/or known parameters. For example, and not limitation, the proximity map modifier 310 can be adjusted by the user and/or adjusted automatically.
Further, the proximity map modifier 310 can perform any convenient and/or known function. In one embodiment, the proximity map modifier 310 can expand the regions that were detected to have noise. In other embodiments, the proximity map modifier 310 can contract the regions that were detected to have noise. In further embodiments, the proximity map modifier 310 can selectively expand or contract regions that were detected to have noise. Further, in additional embodiments, the filter 302, subtractor 312, and mixing module 314 can each be connected to respective modifier modules in order to further optimize the digital signal.
FIG. 4 depicts a system for enhancing a digital signal. In the example of FIG. 4, the system includes a filter 402, a subtractor 404, and an adder 406. The filter 402 is coupled to the subtractor 404. The subtractor 404 is coupled to the adder 406.
In operation, the filter 402 receives a digital signal. The digital signal can be any convenient and/or known digital signal, including but not limited to, a digital video signal. The filter 402 can be any convenient and/or known filter capable of generating a filtered signal. In one embodiment, the filter 402 can be a low-pass filter which blocks the high frequency content of the digital signal.
The subtractor 404 receives the filtered signal from the filter 402, and the digital signal. In operation, the subtractor 404 subtracts the filtered signal from the digital signal in order to generate a modified signal. In certain embodiments, the subtractor can be any known and/or convenient device capable of combining the digital signal and filtered digital signal to produce a desired result. For example, and not limitation, the filtered signal can contain low frequency content of the digital signal. Following the example, the subtractor 404 subtracts the low-frequency content from the digital signal thereby generating the modified signal having high frequency content. In other embodiments, the filter 402 and subtractor 404 can be replaced with a high frequency filter or other suitable convenient and/or known device.
The adder 406 receives the modified signal from the subtractor 404, and the digital signal. In operation, the adder 406 combines the modified signal with the digital signal to generate an optimized signal. In certain embodiments, the adder 406 can be any known and/or convenient device capable of combing the modified signal and the digital signal to produce a desired result. For example, and not limitation, the modified digital signal can contain high frequency content. Following the example, the adder 406 combines the digital signal with the high frequency content thereby generating an optimized signal having the high frequency content enhanced. In other embodiments, the filter 402, the subtractor 404 and the adder 406 can be one device that enhances the high frequency content of the digital signal.
FIG. 5 depicts a system 500 for enhancing a digital signal. In the example of FIG. 5, the system 500 includes a low frequency filter 502, a subtractor 504, an edge detector 506, a modifier 508, a multiplier 510 and an adder 512. The low frequency filter 502 is coupled to the subtractor 504. The edge detector 506 is coupled to the modifier 508. The subtractor 504 and the modifier 508 are coupled to the multiplier 510. The multiplier is coupled to the adder 512.
In operation, the low frequency filter 502 receives a digital signal. The digital signal can be any convenient and/or known format, including for example and not limitation, interlaced, progressive or any broadcast format including, but not limited to NTSC, SECAM or PAL. It should be noted that in certain embodiments analog formats can be digitized and decoded using any convenient and/or known technique in order to generate the desired digital signal. However, in alternate embodiments, an analog signal can be received and a similar operation can be applied in order to generate a desired result. The low frequency filter 502 blocks the high frequency content of the digital signal and provides a signal with low frequency content to the subtractor 504.
The subtractor 504 receives the signal with low frequency content and the digital signal. The subtractor 504 subtracts the signal with low frequency content from the digital signal. The result is a signal with high frequency content. In other embodiments, the low frequency filter 502 and the subtractor 504 can be replaced with a high frequency filter. In further embodiments, any combination of known and/or convenient components can be combined to generate the desired signal. The subtractor 504 provides the resulting signal to the multiplier 510.
The edge detector 506 also receives the digital signal. The edge detector 506 analyzes the digital signal and detects the edges of the objects therein. The edge detector 506 can use any convenient and/or known technique to detect the edges within the digital signal, including for example, calculating a gradient and applying a known and/or convenient operator and/or technique, including but not limited to a Canny edge detector or modification thereof, Sobel technique or modification thereof and/or any other technique involving a high-pass frequency filter, low pass frequency filter, bandpass filter, difference of Gaussians, FIR filter, convolution, first order differential operators and/or second order differential operators. Further, the edge detector 506 can use the technique described in U.S. patent application Ser. No. 11/437,357 to Dale Adams entitled “Edge Detection” filed on May 19, 2006 which is incorporated herein by reference.
The edge detector 506 provides the edge detected signal to the modifier 508. The modifier 508 adjusts the signal in order to obtain a desired result. In some embodiments, the modifier 508 can be preset by a user to perform a default modification to the edge detected signal. In other embodiments, the user can manually adjust the modifier and/or the modifier can be adjusted automatically by the system 500. The modifier 508 provides the modified edge detected signal to the multiplier 510.
In the example of FIG. 5, the multiplier 510 receives the modified edge detected signal from the modifier 508 and the signal with high frequency content from the subtractor 504. The multiplier 510 multiplies the modified edge detected signal and the high frequency content signal to generate a modified high frequency signal. In some embodiments, the characteristics of the multiplier 510 can be adjusted manually and/or automatically in order to obtain a desired result. For example, and not limitation, the multiplier 510 can assign a greater weight to a particular signal to generate a desired result. The multiplier 510 provides the modified high frequency signal to the adder 512.
The adder 512 receives the modified high frequency signal from the multiplier 510 and the digital signal. The adder 512 combines the modified high frequency signal and the digital signal to generate an edge enhanced signal. The adder 512 can be any convenient and/or known device capable of combining the modified high frequency signal and the digital signal to generate an edge enhanced signal. For example, and not limitation, the adder 512 can include combinational logic to produce the desired signal, including but not limited to, an AND gate.
FIG. 6 depicts a flowchart 600 of an example of a method for optimizing a digital signal. FIG. 6 is intended to illustrate the removal of noise from a digital signal. In certain embodiments, noise can include any unwanted electrical signal that is found in the video signal, including but not limited to, general noise, compression artifacts, mosquito noise, and/or block noise which can be correlated to the image or uncorrelated to the image. In the example of FIG. 6, the flowchart 600 begins at module 602 where a digital signal is received. The digital signal can be received using any convenient and/or known interface, including but not limited to, component, composite, DVI, HDMI and/or SDI.
In the example of FIG. 6, the flowchart 600 continues at module 604 where the digital signal is filtered. The digital signal can be filtered using any known and/or convenient device capable of producing a desired signal. In one embodiment, the digital signal is filtered with a low-pass filter in order to produce a signal with low frequency content.
In the example of FIG. 6, the flowchart 600 continues at module 606 where the digital signal and the filtered signal are mixed according to a composite blend map. The digital signal and the filtered signal can be mixed using any known and/or convenient device capable of producing a desired signal. For example, and not limitation, the digital signal and filtered signal can be mixed according to the composite blend map using combination logic. In one embodiment, the regions surrounding the edges in the digital signal can be replaced with the filtered signal in order to reduce noise, such as for example, mosquito noise that results from compression artifacts. At module 608, the optimized digital signal is provided as an output.
FIG. 7 depicts a flowchart 700 of an example of a method for optimizing a digital signal. FIG. 7 is intended to illustrate the removal of noise from a digital signal. In the example of FIG. 7, the flowchart 700 begins at module 702 where a digital signal is received. The digital signal can be received from any convenient and/or known source. For example, and not limitation, the digital signal can be received using an antenna, set-top box, receiver, DVD player or any other type of device capable of producing a digital signal.
In the example of FIG. 7, the flowchart 700 continues at module 704 where the digital signal is filtered. Once filtered, edges are detected in the filtered digital signal at module 706. At module 708, a predictor is applied to the edge detected signal to generate a proximity map. In one embodiment, the predictor can include a 17�17 block which is passed over each pixel location in the digital signal. In other embodiments, the predictor can include a block of any convenient and/or known size capable of predicting the location of noise.
In the example of FIG. 7, the flowchart 700 continues at module 710 where the edge detected signal is subtracted from the proximity map to generate a composite blend map. At module 712, the digital signal and the filtered signal are combined according to the composite blend map. The result is a digital signal with reduced noise. In one embodiment, the digital signal has reduced mosquito noise around the edges of the objects in the digital signal.
FIG. 8 depicts a flowchart 800 of an example of a method for optimizing a digital signal. FIG. 8 is intended to illustrate the enhancement of a digital signal. In the example of FIG. 8, the flowchart 800 begins at module 802 where a digital signal is received. The flowchart 800 continues at module 804 where the digital signal is filtered.
In the example of FIG. 8, the flowchart 800 continues at module 806 were the filtered signal is subtracted from the digital signal. In certain embodiments, the subtraction step is not necessary if the filter produces a desired signal. For example, and not limitation, if high frequency content is desired, the filter can be a low pass filter and the subtraction step would be necessary. However, for example and not limitation, if high frequency content is desired, the filter can be a high pass filter and the subtraction step would not be necessary. In additional embodiments, any number of convenient and/or known devices and/or steps can be implemented to produce the desired result.
In the example of FIG. 8, the flowchart 800 continues at module 808 where the modified signal is added to the digital signal. In one embodiment, the modified signal contains high frequency content. At module 810, the optimized signal is provided as an output. In one embodiment, the optimized signal can contain enhanced high frequency content.
FIG. 9 depicts a flowchart 900 of an example of a method for optimizing a digital signal. FIG. 9 is intended to illustrate enhancing a digital signal. In the example of FIG. 9, the flowchart begins at module 902 where a digital signal is received. At module 904, a low frequency filter is applied to the digital signal. Edges are detected in the filtered signal at module 906. At module 908, the edge detected signal is modified. The modification can be automatic and/or manual according to the desired result of the signal.
In the example of FIG. 9, the flowchart 900 continues at module 910 where the filtered signal is subtracted from the digital signal. At module 912, the result of module 910 is multiplied by the modified edge detected signal from module 908. The result of module 912 is added to the digital signal at module 914 thereby generating the enhanced signal. In other embodiments, steps may be added, modified or removed according to desired functionality of the system and/or output of the signal.
FIG. 10 depicts and example of a system 1000 for enhancing and optimizing a digital signal. In this example, as well as previous examples, the systems and/or methods can be implemented in software, hardware and/or firmware. Further, the systems and/or methods can be built into a set top box or a television.
In the example of FIG. 10, the system 1000 includes line memory modules 1002 1008 1016 1026 1028, 2D Gaussian filters 1004 and 1006, a gradient module (edge detection) 1010, a proximate map modification module 1012, a proximity map generation module 1018, a proximate map modification module 1020, a background map generation module 1022, an edge modification module 1024, a composite blend map module 1030, an enhancement module 1014, and a final mixing module 1032.
The 2D Gaussian filter 1004 is coupled to the gradient module (edge detection) 1010 as well as line memory 1008. The 2D Gaussian filter 1006 is coupled to the enhancement module 1014 and line memory 1026. The line memories 1002 are connected to the 2D Gaussian filter 1004 and 1006 as well as the enhancement module 1014. The gradient module 1010 is coupled to the proximity map generation module 1018 as well as line memory 1016. The line memories 1016 are coupled to the proximity map generation module 1018, the edge modification module 1024, and the proximate map modification module 1012. The proximity map generation unit 1018 is coupled to the proximate map modification module 1020 and the background map generation module 1022. The proximate map modification module 1020 and the background map generation module 1022, and the edge modification module 1024 are coupled to the composite blend map module 1030. The composite blend map module 1030 is coupled to the final mixing module 1032. The proximate map modification module 1012 is coupled to the enhancement module 1014. The enhancement module 1014 is coupled to line memory 1028. The line memories 1026 and 1028 are coupled to the final mixing module 1032.
In the example of FIG. 10, the Gaussian filter 1004 receives a video input signal as well as inputs from line memories 1002. The Gaussian filter 1006 receives inputs from line memories 1002. The filtered signal from the 2D Gaussian filter 1004 is provided to the gradient module 1010 and the filtered signal from the 2D Gaussian filter 1006 as well as delayed Y signal are provided to the enhancement module 1014. The enhancement module 1014 uses an unsharp mask and the output signal is provided to the line memories 1028. The signal from the gradient module (edge detection) is sent to line memories 1016 and also used to generate a proximity map in module 1018, which is provided to module 1020 for proximate map modification and module 1022 for background map generation. The signal from a line memory 1016 is provided to the edge modification module 1024. The signals from modules 1020, 1022, and 1024 are provided to generate a composite blend map in module 1030 by combining the background signal with the result of subtraction of the edge modification signal from the modified proximate map signal. The 2D Gaussian filter 1006 signal goes through line memories 1026, which is provided to the final mixing module 1032 together with the enhancement module 1014 signal through line memories 1028 and the composite blend map signal from module 1030 to generate an enhanced signal with reduced noise.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4997321May 7, 1990Mar 5, 1991Adams Dale RApparatus for aligning and reboring a damaged throughbore of a housingUS5357606Feb 25, 1992Oct 18, 1994Apple Computer, Inc.Row interleaved frame bufferUS5532751Jul 31, 1995Jul 2, 1996Lui; SamEdge-based interlaced to progressive video conversion systemUS5550592Apr 25, 1995Aug 27, 1996Texas Instruments IncorporatedFilm mode progressive scan conversionUS5689301Dec 30, 1994Nov 18, 1997Thomson Consumer Electronics, Inc.Method and apparatus for identifying video fields produced by film sourcesUS5790269 *Dec 12, 1995Aug 4, 1998Massachusetts Institute Of TechnologyMethod and apparatus for compressing and decompressing a video imageUS5796875Aug 13, 1996Aug 18, 1998Sony Electronics, Inc.Selective de-blocking filter for DCT compressed imagesUS5852475 *Jun 6, 1995Dec 22, 1998Compression Labs, Inc.Transform artifact reduction processUS5857118Nov 18, 1996Jan 5, 1999Apple Computer, Inc.shared video display in a multiple processor computer systemUS5920356Jun 6, 1996Jul 6, 1999Compressions Labs, Inc.Coding parameter adaptive transform artifact reduction processUS6055018Nov 4, 1997Apr 25, 2000Ati Technologies, Inc.System and method for reconstructing noninterlaced captured content for display on a progressive screenUS6064776 *Oct 24, 1996May 16, 2000Kabushiki Kaisha ToshibaImage processing apparatusUS6069664Jun 4, 1997May 30, 2000Matsushita Electric Industrial Co., Ltd.Method and apparatus for converting a digital interlaced video signal from a film scanner to a digital progressive video signalUS6167164Mar 10, 1998Dec 26, 2000Samsung Electronics Co., Ltd.One-dimensional signal adaptive filter for reducing blocking effect and filtering methodUS6219747Jan 6, 1999Apr 17, 2001Dvdo IncMethods and apparatus for variable length SDRAM transfersUS6285801May 29, 1998Sep 4, 2001Stmicroelectronics, Inc.Non-linear adaptive image filter for filtering noise such as blocking artifactsUS6380978Oct 6, 1998Apr 30, 2002Dvdo, Inc.Digital video system and methods for providing sameUS6385692Mar 12, 2001May 7, 2002Silicon Image, Inc.Methods and apparatus for variable length SDRAM transfersUS6393505Jan 6, 1999May 21, 2002Dvdo, Inc.Methods and apparatus for data bus arbitrationUS6421090Aug 27, 1999Jul 16, 2002Trident Microsystems, Inc.Motion and edge adaptive deinterlacingUS6459455Aug 31, 1999Oct 1, 2002Intel CorporationMotion adaptive deinterlacingUS6473476Jan 6, 1999Oct 29, 2002Dvdo, IncorporatedMethod and apparatus for providing deterministic resets for clock divider systemsUS6489998Aug 11, 1999Dec 3, 2002Dvdo, Inc.Method and apparatus for deinterlacing digital video imagesUS6515706Sep 15, 1999Feb 4, 2003Dvdo, Inc.Method and apparatus for detecting and smoothing diagonal features video imagesUS6587158Jul 22, 1999Jul 1, 2003Dvdo, Inc.Method and apparatus for reducing on-chip memory in vertical video processingUS6621937 *Oct 8, 1999Sep 16, 2003Eastman Kodak CompanyRemoving chroma noise from digital images by using variable shape pixel neighborhood regionsUS6681059Jul 28, 1999Jan 20, 2004Dvdo, Inc.Method and apparatus for efficient video scalingUS6700622Oct 1, 1999Mar 2, 2004Dvdo, Inc.Method and apparatus for detecting the source format of video imagesUS6757022Sep 10, 2001Jun 29, 2004Pixelworks, Inc.Method and apparatus for motion adaptive deinterlacingUS6757442 *Nov 22, 2000Jun 29, 2004Ge Medical Systems Global Technology Company, LlcImage enhancement method with simultaneous noise reduction, non-uniformity equalization, and contrast enhancementUS6859237Mar 14, 2003Feb 22, 2005Genesis Microchip Inc.Film source video detectionUS6867814Apr 18, 2001Mar 15, 2005Silicon Image, Inc.Method, system and article of manufacture for identifying the source type and quality level of a video sequenceUS6975776 *Feb 14, 2002Dec 13, 2005Tektronix, Inc.Predicting human vision perception and perceptual differenceUS6999047Mar 15, 1999Feb 14, 2006Koninklijke Philips Electronics N.V.Displaying video on a plasma display panelUS7023487Jan 25, 2002Apr 4, 2006Silicon Image, Inc.Deinterlacing of video sources via image feature edge detectionUS7027099Sep 19, 2002Apr 11, 2006Silicon ImageMethod and apparatus for deinterlacing digital video imagesUS7089577Jun 23, 2000Aug 8, 2006Terayon Communication Systems, Inc.Process for supplying video-on-demand and other requested programs and services from a headendUS7126643Dec 22, 2003Oct 24, 2006Samsung Electronics Co., Ltd.Progressive scan method used in display using adaptive edge dependent interpolationUS7136541Oct 18, 2002Nov 14, 2006Sony CorporationMethod of performing sub-pixel based edge-directed image interpolationUS7154556Mar 20, 2003Dec 26, 2006Pixelworks, Inc.Weighted absolute difference based deinterlace method and apparatusUS7206025Mar 23, 2001Apr 17, 2007Lg Electronics Inc.Device and method for converting format in digital TV receiverUS7236209Mar 14, 2003Jun 26, 2007Monster Cable Products, Inc.Method and apparatus for automatic selection of video interfaceUS7257272 *Apr 16, 2004Aug 14, 2007Microsoft CorporationVirtual image generationUS7345708Dec 23, 2003Mar 18, 2008Lsi Logic CorporationMethod and apparatus for video deinterlacing and format conversionUS7349028Sep 21, 2004Mar 25, 2008Broadcom CorporationMethod and system for motion adaptive deinterlacer with integrated directional filterUS7362376Dec 23, 2003Apr 22, 2008Lsi Logic CorporationMethod and apparatus for video deinterlacing and format conversionUS7391468Sep 24, 2004Jun 24, 2008Magnum Semiconductor, Inc.Telecine conversion detection for progressive scan playbackUS7400359Jan 7, 2004Jul 15, 2008Anchor Bay Technologies, Inc.Video stream routing and format conversion unit with audio delayUS7412096Sep 21, 2004Aug 12, 2008Broadcom CorporationMethod and system for interpolator direction selection during edge detectionUS7414671Jun 30, 2005Aug 19, 2008Magnum Semiconductor, Inc.Systems and methods for display object edge detection and pixel data interpolation in video processing systemsUS7417686Feb 24, 2005Aug 26, 2008Hnaya Microelectronics, LtdConversion of interlaced video streams into progressive video streamsUS7474354Apr 30, 2003Jan 6, 2009Panasonic CorporationImage angle detection device and scan line interpolation device having the sameUS7515205Nov 30, 2006Apr 7, 2009Pixelworks, Inc.Weighted absolute difference based deinterlace method and apparatusUS7519332Mar 31, 2000Apr 14, 2009Sharp Kabushiki KaishaRadio-frequency radiocommunication apparatusUS7529426Sep 21, 2004May 5, 2009Broadcom CorporationCorrelation function for signal detection, match filters, and 3:2 pulldown detectionUS7551800 *Aug 9, 2007Jun 23, 2009Fotonation Vision LimitedDetection of airborne flash artifacts using preflash imageUS7557861Sep 21, 2004Jul 7, 2009Broadcom CorporationReverse pull-down video using corrective techniquesUS7605866Oct 20, 2009Realnetworks, Inc.Automatic deinterlacing and inverse telecineUS7657098 *Feb 2, 2010Samsung Electronics Co., Ltd.Method and apparatus for reducing mosquito noise in decoded video sequenceUS7659939Feb 9, 2010Lsi CorporationMethod and apparatus for video deinterlacing and format conversionUS7667773Jun 10, 2005Feb 23, 2010Samsung Electronics Co., Ltd.Apparatus and method of motion-compensation adaptive deinterlacingUS7710501Jul 12, 2004May 4, 2010Anchor Bay Technologies, Inc.Time base correction and frame rate conversionUS7865035 *Oct 6, 2005Jan 4, 2011Samsung Electronics Co., Ltd.Video quality adaptive coding artifact reductionUS7940992 *May 10, 2011Siemens AktiengesellschaftVisual discrimination model for single image applicationsUS7969511 *Jun 28, 2011Samsung Electronics Co., Ltd.Method and apparatus for improving quality of composite video signal and method and apparatus for decoding composite video signalUS7986854 *Jun 11, 2007Jul 26, 2011Industry-University Cooperation Foundation Sogang UniversityMethod and apparatus for improving quality of composite video signal and method and apparatus for removing artifact of composite video signalUS20020149685 *Mar 21, 2002Oct 17, 2002Nec Viewtechnology, Ltd.Method of and apparatus for improving picture qualityUS20020149703Apr 18, 2001Oct 17, 2002Adams Dale R.Method, system and article of manufacture for identifying the source type and quality level of a video sequenceUS20040042673Aug 22, 2003Mar 4, 2004Boon Choong SengDigital image coding/decoding method and data recording mediumUS20040189877Jan 7, 2004Sep 30, 2004Choi Byung TaeApparatus for detecting a common frame in an interlaced imageUS20050122433 *Nov 5, 2004Jun 9, 2005Oki Electric Co., Ltd.Noise reduction circuit and methodUS20050128360Sep 8, 2004Jun 16, 2005Sunplus Technology Co., Ltd.Bad editing detection deviceUS20060072037Dec 21, 2004Apr 6, 2006Wyman Richard HDetection and correction of irregularities while performing inverse telecine deinterlacing of videoUS20070052845May 19, 2006Mar 8, 2007Adams Dale REdge detectionUS20070103588Nov 10, 2005May 10, 2007Alexander MaclnnisSystem and method for adjacent field comparison in video processingUS20070223835Aug 1, 2006Sep 27, 2007Fujitsu LimitedBlock noise detecting method and apparatus, and block noise reducing method and apparatusUS20080123998Apr 21, 2005May 29, 2008Sony CorporationImage Processing Apparatus, Image Processing Method, Program of Image Processing Method, and Recording Medium in Which Program of Image Processing Method Has Been RecordedUS20080143873 *Dec 15, 2006Jun 19, 2008Darren NeumanTv user interface and processing for personal video playersUS20080151103Nov 21, 2005Jun 26, 2008Masako AsamuraVideo Signal Processing Apparatus, Video Signal Processing Method, and Video Signal Display ApparatusEP0881837A1May 30, 1997Dec 2, 1998SGS-THOMSON MICROELECTRONICS S.r.l.Post-processing method for reducing artifacts in block-coded digital images, and post-processing device for actuating such methodEP1039760A2Mar 13, 2000Sep 27, 2000Victor Company Of Japan, LimitedApparatus and method of block noise detection and reductionEP1434438A2Nov 14, 2003Jun 30, 2004Samsung Electronics Co., Ltd.Image signal format detection apparatus and methodEP1492344A1Mar 28, 2003Dec 29, 2004Matsushita Electric Industrial Co., Ltd.Field interpolation mehtod determination deviceJP2001245155A Title not availableJP2005122361A Title not availableJP2007213125A Title not available* Cited by examinerNon-Patent CitationsReference1"Analog Devices CMOS 180 MHz DDS/DAC Synthesizer", AD9851, Rev. C, Analog Devices, Inc., www.analog.com.pp. 1-23 (1999).2Co-pending U.S. Appl. No. 10/753,909, filed Jul. 7, 2004.3Co-pending U.S. Appl. No. 10/889,855, filed Jul. 12, 2004.4Co-pending U.S. Appl. No. 11/437,357, filed May 19, 2006.5Co-pending U.S. Appl. No. 11/487,144, filed Jul. 13, 2006.6Co-pending U.S. Appl. No. 11/512,754, filed Aug. 29, 2006.7Co-pending U.S. Appl. No. 12/204,760, filed Sep. 4, 2008.8Co-pending U.S. Appl. No. 12/703,623, filed Feb. 10, 2010.9Final Office Action mailed Apr. 19, 2010 for U.S. Appl. No. 11/487,144, filed Jul. 13, 2006.10International Preliminary Report on Patentability PCT/US2009/054427 dated Mar. 17, 2011, pp. 1-5.11International Search Report and Written Opinion from PCT Application No. PCT/US2007/084881, Search report dated Mar. 25, 2008, 10 pages.12International Search Report PCT/US2006/034785 dated Apr. 4, 2007.13International Search Report PCT/US2009/054427 dated Mar. 17, 2010 pp. 1-3.14International Search Report PCT/US2010/023775 dated Oct. 11, 2010, pp. 1-3.15Non-Final Office Action for U.S. Appl. No. 11/512,754 mailed May 11, 2011.16Non-Final Office Action mailed Aug. 20, 2010 for U.S. Appl. No. 11/487,144, filed Jul. 13, 2006.17Non-Final Office Action mailed Dec. 13, 2010 U.S. Appl. No. 11/512,754, filed Aug. 29, 2006.18Non-Final Office Action mailed Dec. 31, 2009 for U.S. Appl. No. 11/487,144, filed Jul. 13, 2006.19Non-Final Office Action mailed Jun. 25, 2010 U.S. Appl. No. 11/512,754, filed Aug. 29, 2006.20Non-Final Office Action mailed Mar. 18, 2008 for Issued Patent No. 7,400,359, U.S. Appl. No. 10/753,909, filed Jan. 7, 2004.21Non-Final Office Action mailed Sep. 11, 2009 for Issued Patent U.S. Patent No. 7,710,501, U.S. Appl. No. 10/889,855, filed Jul. 12, 2004.22Non-Final Office Action mailed Sep. 23, 2010 for U.S. Appl. No. 11/437,357, filed on May 19, 2001.23Notice of Allowance mailed Dec. 1, 2010 U.S. Appl. No. 11/437,357, filed May 19, 2006.24Notice of Allowance mailed Dec. 9, 2010 U.S. Appl. No. 11/487,144, filed Jul. 13, 2006.25Notice of Allowance mailed Feb. 7, 2011 U.S. Appl. No. 11/512,754, filed Aug. 29, 2006.26Notice of Allowance mailed Feb. 8, 2010 for Issued Patent No. 7,710,510, U.S. Appl. No. 10/889,855, filed Jul. 12, 2004.27Notice of Allowance mailed Mar. 21, 2011 U.S. Appl. No. 11/487,144, filed Jul. 13, 2006.28Notice of Allowance mailed Mar. 9, 2011 U.S. Appl. No. 11/437,357, filed May 19, 2006.29Notice of Allowance mailed May 30, 2008 for Issued Patent No. 7,400,359, U.S. Appl. No. 10/753,909, filed Jan. 7, 2004.30Notice of of Allowance mailed Sep. 3, 2010 for U.S. Appl. No. 11/512,754, filed Aug. 29, 2006.31Office Action mailed Aug. 18, 2010 CA Patent Application No. 2,620,820, filed on Sep. 8, 2006.32Office Action mailed Feb. 10, 2011 EP Patent Application No. 06814256.1, filed on Sep. 8, 2006.33Office Action mailed Jul. 16, 2010 EP Patent Application No. 06814256.1, filed on Sep. 8, 2006.34Restriction Requirement mailed Feb. 25, 2010 for U.S. Appl. No. 11/437,357, filed May 19, 2006.35Supplementary European Search Report 06 814 256 dated Mar. 31, 2010, pp. 1-7.36Written Opinion PCT/US2006/34785 dated Apr. 4, 2007, pp. 1-4.37Written Opinion PCT/US2007/084881 dated Mar. 25, 2008 pp. 1-4.38Written Opinion PCT/US2009/054427 dated Mar. 17, 2010 pp. 1-3.39Written Opinion PCT/US2010/023775 dated Oct. 11, 2010, pp. 1-5.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8446525Jun 3, 2011May 21, 2013Silicon Image, Inc.Edge detectionClassifications U.S. Classification382/275, 382/181, 382/264, 382/190, 382/260, 382/199International ClassificationG06K9/46, G06K9/66, G06K9/48, G06K9/40, G06K9/00Cooperative ClassificationH04N5/21, H04N5/208European ClassificationH04N5/208, H04N5/21Legal EventsDateCodeEventDescriptionJan 10, 2008ASAssignmentOwner name: ANCHOR BAY TECHNOLOGIES, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:THOMPSON, LAURENCE A.;REEL/FRAME:020426/0348Effective date: 20080110Dec 7, 2010ASAssignmentOwner name: ABT-AC, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ANCHOR BAY TECHNOLOGIES, INC.;REEL/FRAME:025461/0563Effective date: 20101203Feb 16, 2011ASAssignmentOwner name: SILICON IMAGE, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ABT-AC, INC.;REEL/FRAME:025820/0249Effective date: 20110214Mar 19, 2015ASAssignmentOwner name: JEFFERIES FINANCE LLC, NEW YORKFree format text: SECURITY INTEREST;ASSIGNORS:LATTICE SEMICONDUCTOR CORPORATION;SIBEAM, INC.;SILICON IMAGE, INC.;AND OTHERS;REEL/FRAME:035226/0289Effective date: 20150310Jun 29, 2015FPAYFee paymentYear of fee payment: 4Aug 21, 2015ASAssignmentOwner name: LATTICE SEMICONDUCTOR CORPORATION, OREGONFree format text: MERGER;ASSIGNOR:SILICON IMAGE, INC.;REEL/FRAME:036419/0792Effective date: 20150513RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services