System and method for feature emphasis and de-emphasis in image processing

A system and method for emphasizing and/or de-emphasizing features in a digital image. In an embodiment of the invention, the image is processed by a series of filters, which decompose the image into a series of sub-bands. Each sub-band is then processed by an emphasis circuit. In an embodiment of the invention, the emphasis circuit is connected to an input representing a particular sub-band. Each filter, therefore, is connected to its own respective emphasis circuit. The result is a series of emphasis circuit outputs, which are then combined. The result of the combination is the processed image, containing one or more emphasized or deemphasized features. In an alternative embodiment of the invention, the sub-bands are combined in a weighted fashion. The sum of the weighted sub-bands is then applied to a single emphasis circuit. The output of the emphasis circuit, in this embodiment, is a processed image having one or more emphasized or deemphasized features.

The above stated application is hereby incorporated herein by reference in its entirety.

FIELD OF THE INVENTION

Certain embodiments of the invention relate to image processing. More specifically, certain embodiments of the invention relate to a method and system for emphasis and de-emphasis in image processing

BACKGROUND OF THE INVENTION

Feature emphasis and de-emphasis are common image processing operations. A feature in an image can be as straightforward as an edge or as complicated as a complex fractal pattern. Image sharpening is one example of feature emphasis. Image smoothing is an example of the opposite process, feature de-emphasis.

One purpose of emphasizing a feature may be to make the image look better or to explicitly exaggerate the feature. A useful application of feature emphasis is edge enhancement. A useful application of feature de-emphasis is noise suppression. Sharpening an image may be required for several reasons. An image that was sharp at its native resolution can often lack sharpness once it has been enlarged, so that sharpening may be needed after enlargement. Alternatively, perceived image sharpness is also a function of the observer's physical distance from the screen. In order to make an image appear adequately sharp, one must be cognizant of the observer's position relative to the screen. Whatever the reason, it is of fundamental importance to be able to adjust the level of sharpness in an image to make it perceptually pleasing.

There are many approaches to feature emphasis and de-emphasis. Of the many ways to perform some sort of feature enhancement, image sharpening is among them. Sharpening algorithms typically perform some sort of edge manipulation. Many approaches, however, are linear methods. Linear approaches to sharpening are fraught with problems. For instance, a constant poly-phase finite impulse response (FIR) filter will act the same way on all image content, so it will exacerbate noise as well as make edges more “sharp.” But so-called sharpening linear filters—those that have some gain in the frequency mid-band—also come with another often unwanted side-effect—Gibb's phenomenon, which is more commonly referred to as ringing. The performance of linear filters that sharpen is particularly poor in regions of the image where there is relatively little content, that is, areas of the image that are very flat. In these regions a sharpening filter only serves to enhance noise. While some overshoot and undershoot is necessary to create an effect that sharpens an image, when ringing occurs, there are many repetitive artifacts close to an edge that are visually unappealing.

It would therefore be desirable to have a feature emphasis/de-emphasis circuit that operates in accordance with the following principles. Such a circuit: 1) should not aggravate areas when the feature is not present; 2) should identify the feature in question; 3) should not alter the feature's fundamental properties, e.g., the size of the feature, although such a circuit may alter the feature's range, i.e., contrast; and 4) should not contain heuristic components. In the more specific context of image sharpening, such a circuit or process should: 1) be benign in regions that are flat or relatively flat; 2) add overshoot and undershoot (emphasis and de-emphasis) in a controlled fashion; 3) not ring; and, 4) not contain any arbitrary user-defined thresholds. Depending on the exact nature of the emphasis/de-emphasis problem, these principles may assume a more specific connotation.

BRIEF SUMMARY OF THE INVENTION

A system and/or method is provided for emphasis and de-emphasis in image processing, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.

DETAILED DESCRIPTION OF THE INVENTION

A preferred embodiment of the present invention is now described with reference to the figures, where like reference numbers indicate identical or functionally similar elements. Also in the figures, the leftmost digit of each reference number corresponds to the figure in which the reference number is first used. While specific configurations and arrangements are discussed, it should be understood that this is done for illustrative purposes only. A person skilled in the relevant art will recognize that other configurations and arrangements can be used without departing from the spirit and scope of the invention. It will be apparent to a person skilled in the relevant art that this invention can also be employed in a variety of other systems and applications.

Embodiments of a sharpening system and associated process are described herein. This system and circuit are used to sharpen images by altering the overshoot and undershoot (emphasis and de-emphasis) in the edges of the image content. The same circuit can be used to selectively reduce what is deemed to be noise, i.e., regions of the image where the local variability is low. The circuit and process are non-linear in an embodiment of the invention.

Embodiments of the circuit and process described herein can be used in an arbitrary dimensional space. For example, in audio processing they can be used to sharpen transition and suppress noise in the audio signal; in a two-dimensional application such as image processing, they can be used to sharpen the image and suppress spurious detail; and in three-dimensional mappings of a solid, e.g., computed axial tomography (CAT) scans, the same circuit can be used to create sharper three dimensional images.

The invention described herein includes a system and method for emphasizing and/or de-emphasizing features in a digital image. In an embodiment of the invention, the image is processed by a series of filters, which decompose the image into a series of sub-bands. Each sub-band is then processed by an emphasis circuit. In an embodiment of the invention, the emphasis circuit is connected to an input representing a particular sub-band. Each filter, therefore, is connected to its own respective emphasis circuit. The result is a series of emphasis circuit outputs, which are then combined. The result of the combination is the processed image, containing one or more emphasized or deemphasized features.

In an alternative embodiment of the invention, the sub-bands are combined in a weighted fashion. The sum of the weighted sub-bands is then applied to a single emphasis circuit. The output of the emphasis circuit, in this embodiment, is a processed image having one or more emphasized or deemphasized features.

The emphasis circuit comprises an emphasis orientation module, a feature detector module, and an edge orientation module, in an embodiment of the invention. These modules generate an emphasis orientation information, feature detection information, and edge orientation information. An output that is a function of the emphasis orientation information, the feature detector information, and the edge orientation information, is then combined with a first user input. The difference between this output and the original image is then determined. This difference is then combined with a second user input and passed through a limiter. The limiter serves to regulate the amount of emphasis or de-emphasis placed on the image. In an embodiment of the invention, the limiter output can be passed through a pattern injection circuit. This circuit can be used to mark a specific feature in the image. The output of the pattern injection circuit, therefore, includes a feature bearing some sort of marking or pattern. If the pattern injection circuit is used, the output of the pattern injection circuit can be used as the final processed image. Alternatively, if a pattern injection circuit is not used, the limiter output can be used as the final processed image.

An embodiment of the system of the invention is illustrated inFIG. 1. An image110is passed to each of a plurality of filters, identified as1201to120m. Each filter generates a respective sub-band of the original image110. Any image can be decomposed into a series of sub-bands. Sub-bands, in their most common form, can correspond to frequency bands. InFIG. 1, any sub-band can be acted upon independently. Filter1201therefore generates a sub-band1251for example. Each sub-band125is then input to its own respective emphasis circuit. These are shown as emphasis circuits1301through130m. The respective outputs of the emphasis circuits are shown as emphasis outputs1351through135m. The emphasis circuit outputs are then combined in combiner140. The output of combiner140is processed image145.

FIG. 2illustrates an alternative embodiment of the invention. As before, image110is input to a plurality of filters, illustrated inFIG. 2as filters1701through170m. In this embodiment, the sub-bands that are output from the filters are shown as sub-bands1751through175m. A weighted sum of the sub-bands is created in combiner190. The output of combiner190is then input into emphasis circuit195. The output of emphasis circuit195is processed image197.

Note that the embodiment ofFIG. 2allows for band selectivity. In an embodiment of the invention, band selectivity is a user-defined option that can be programmed to activate one or more bands for sharpening or for noise suppression. Band selection can be a function of factors including, but not limited to, image resolution.

The emphasis circuit of the previous two figures is illustrated in greater detail inFIG. 3, according to an embodiment of the invention. This circuit includes several functions to collectively accomplish feature emphasis/de-emphasis. These functions include a feature detection function, an emphasis orientation function, and an edge orientation function. The illustrated circuit includes components corresponding to these functions and operating in concert, along with additional input control parameters, to perform the sharpening function.

Note that a feature can have many orientations. Three are of primary interest: the feature is present, the feature is absent, or the feature is present by virtue of its surroundings. The latter requires some further explanation. Suppose an emphasis orientation function is defined to be local gradient. The gradient can be positive, zero, or negative. The negative gradient is an example of the feature being present by virtue of its surroundings. Similarly, convexity may be positive, absent (i.e., zero), or negative. These quantities can lie anywhere in the domain of real numbers, but the real number line is still defined by three regions—positive, zero and negative numbers.

Referring toFIG. 3, the data that is input to the emphasis circuit is shown as information305. This information is shown as information related to a sub-band. In the A filter can be used to signal whether the local image content has a positive or negative emphasis orientation in an embodiment of the invention. One example of emphasis orientation is curvature. A test for curvature is to measure convexity, although in other embodiments of the invention more sophisticated tests can be used. Convexity can be computed by applying a time-invariant finite impulse response filter to the data context of the embodiment ofFIG. 1, information305represents the output of one of the filters. In the context of the embodiment ofFIG. 2, information305represents the output of combiner190. The emphasis circuit comprises an emphasis orientation module320, a feature detector module315, and an edge orientation module310. Information305is input to each of these three modules. Edge orientation module310receives information305, and outputs edge orientation information312. Edge orientation information312represents the orientation of an edge, as it appears in information305.

The function of the edge orientation module310is to determine the direction of the edge in image content. For example, in an embodiment of the invention, anisotropic diffusion could be used to accomplish this, as would be known to a person of skill in the art. Alternatively, in a different implementation, anisotropic diffusion is not required. In a two-pass feature emphasis approach, for example, the results of the first and second pass can be combined together in a weighted additive fashion. The output of this module, edge orientation information312, is input into feature detector module315, along with information305. Feature detector module315is used to identify which specific parts of the image content are to be acted upon. Such parts could be areas that have a specific signature, flat, smooth, film-grain noise, etc. These are detected using the feature detector module315and are subject to enhancement in subsequent processing. Feature detector module315outputs feature detector information317, and will be described in greater detail below.

Edge orientation information312is also input into emphasis orientation module320, along with information305. This module produces emphasis orientation information322. Emphasis orientation information322represents information regarding the orientation of emphasis or de-emphasis, to be applied to the processing of image110. The emphasis orientation module320measures the orientation of the feature in local image content, consistent with the direction specified by the edge orientation information312.

A filter can be used to signal whether the local image content has a positive or negative emphasis orientation in an embodiment of the invention. One example of emphasis orientation is curvature. A test for curvature is to measure convexity, although in other embodiments of the invention more sophisticated tests can be used. Convexity can be computed by applying a time-invariant finite impulse response filter to the data such as [−1 2 −1] in one dimension. In general, however, the curvature could be determined in as sophisticated and selective a manner as desired.

Emphasis orientation information322is combined with feature detector information317, and edge orientation information305in combiner325.

The output of combiner325is entered into combiner330, along with user-defined input335. This input, as well as input350described below, can be viewed as a user defined gain. These serve to amplify or attenuate the degree of sharpening or noise reduction. The output of combiner330is then used, along with image110, as inputs to difference logic340. The difference between image110and the output of combiner330is then used as input to combiner345, along with user defined input350. The output of combiner345is then input to limiter355. Limiter355serves to regulate the amount of emphasis or de-emphasis of a feature. The limiter355can be a non-linear function (mapping) that can be used to arbitrarily regulate the amount of undershoot or overshoot added to an edge. The result is limiter output360.

In the illustrated embodiment, limiter output360is directed to multiplexer365. Limiter output360is also directed to pattern injection circuit370. The pattern injection circuit370is used to add a predefined pattern to the content in strength proportional to detected feature. Pattern injection circuit370may, for example, mark a particular feature with some icon pattern or other form of highlighting. The output of pattern injection circuit370is then input to multiplexer365. Whether the output of pattern injection circuit370or limiter output360is ultimately output from multiplexer365is determined by user input375. The output of multiplexer365is emphasis circuit output380, which corresponds to the processed image.

Feature detector315is illustrated in greater detail inFIG. 4, according to an embodiment of the invention. The feature detector315operates on information305, but in particular, operates on pairs of pixels taken from information305. The illustrated embodiment shows n+1 pairs of pixels (p, q). These are shown as pairs (po, qo) through (pn, qn). For any given pair of pixels piand qi, these two pixels represent pixels on opposite sides of a give feature. As shown inFIG. 4, the difference between the pixels in each pair is determined. For example, the difference between poand qois shown as Δx0. Each Δx is then input to a function f. In an embodiment of the invention, the function f is the absolute value function. In alternative embodiments, a user may choose other functions for f. The values of f are then combined in a weighted combiner operation, resulting in feature detector information317. Here the weights W(i) provide selectivity towards different types of patterns that we wish to identify and emphasis or de-emphasis as the case may be.

FIG. 5illustrates the application of the invention to an image. The feature strength mapping circuit is designed to map the feature strength and confine it to a range that is predetermined. The mapping can be arbitrarily complex or it can be a simple piecewise-linear curve. The one shown inFIG. 5has both negative and positive components. When the feature strength is in a certain region of the mapping, the feature is de-emphasized (i.e. attenuated) and when it is in another region, the feature is emphasized. The zero crossing point can be considered as an indication of the noise ceiling, or the point at which the system of the invention can begin suppressing unwanted detail or texture. In this case, it is entirely possible for the mapping to be adaptive so that its composition is controlled by the influence of several factors—some related to noise, others to user-preferences, and so on. The X axis510represents the strength of a feature. The Y axis520represents the amount of emphasis or de-emphasis (i.e., the amount of control) applied as a function of the feature strength. If the feature strength exceeds the point indicated by530, emphasis is applied. For features having the strength less than530, de-emphasis or attenuation takes place.

FIG. 6illustrates the processing of the invention, according to an embodiment. The process begins at step610. At step620, the image is received. In step630, the image is decomposed into sub-bands. In step640, feature emphasis is performed. This step is described in greater detail below. In step650, the processed image is output. The process concludes at step660.

The step of feature emphasis is illustrated in greater detail inFIG. 7. Here, the process begins at step710. In step715, edge orientation is performed. In step720, feature detection is preformed. In step725, emphasis orientation is performed. The outputs of steps715-725are combined in step730. In an embodiment of the invention, these outputs are combined according to the logic illustrated inFIG. 3.

In step735, a first user defined input is applied to the result of step730. In step740, the difference between the image and the output of step735is determined. A second user defined input is then applied in step745. In step750, the extent of emphasis or de-emphasis that is applied to the feature is limited. In step755, another user defined input is applied. This particular input is used to determine whether or not the emphasis circuit output includes an injected pattern. The process concludes at step760.

The method and system described above can act to achieve both sharpening and noise suppression. They can also do these functions concurrently. Depending on the degree of complexity, the circuit may be very differentiated or not at all. That is, if only one band is deemed important from a sharpening point-of-view, then the circuit can be modified to perform the sharpening operation on only that band. Moreover, noise suppression may act on the same band, or on a separate band. It is possible to perform only the noise suppression option without sharpening.

Noise suppression can be handled in a similar manner to feature enhancement, except that the orientation of the enhancement correction is reversed thereby causing a de-emphasis of the feature in question. In this way it is possible to selectively de-emphasis edges or any other feature in question so the circuit can act as a smoother as well. The negative portion of the mapping shown inFIG. 5is the part of the mapping that causes the de-emphasis.

It is also possible within the framework of the present invention to envision a circuit that adapts its parameters depending on the type of content. For example, a deinterlaced source may be treated differently from progressive, or there may be a component of the circuit that is resolution dependent.