IMAGE PROCESSOR AND OPERATION METHOD THEREOF

An operation method performed by an image processor is provided, including the following operations: receiving a mixed image including an input image and a first on-screen-display (OSD) pattern; generating a second OSD pattern, different from the first OSD pattern, based on the mixed image. The second OSD pattern encompasses protection regions of the input image, and the protection regions are in close proximity of the first OSD pattern. The method further includes operations of performing image processing to remaining regions, different from the protection regions, of the input image and generating an output image. The output image includes the second OSD pattern and the remaining regions of the input image, being performed by the image processing.

BACKGROUND

Field of Invention

The disclosure relates to an image processor and an operation method thereof. More particularly, the disclosure relates to an image processor and an operation method involving optimized on screen display resolution for image processing.

Description of Related Art

The existing OSD bits are not very accurate since they were designed to work in blocks, which can lead to issues in image processing. The low precision of OSD bits results in unnecessary protection at the edges of the user interface, leading to unnatural and discontinuous areas once the background passes through the image processing. Additionally, due to block sampling, some small user interface elements may be missed. Methods like igamma, CCE, SR, 3D LUT, etc., require pixel-wise accuracy, and missing or redundant OSDs can cause significant errors.

SUMMARY

The disclosure provides an operation method performed by an image processor, including the following operations: receiving a mixed image including an input image and a first on-screen-display (OSD) pattern; generating a second OSD pattern, different from the first OSD pattern, based on the mixed image. The second OSD pattern encompasses protection regions of the input image, and the protection regions are in close proximity of the first OSD pattern. The method further includes operations of performing image processing to remaining regions, different from the protection regions, of the input image and generating an output image. The output image includes the second OSD pattern and the remaining regions of the input image, being performed by the image processing.

It is to be understood that both the foregoing general description and the following detailed description are demonstrated by examples, and are intended to provide further explanation of the invention as claimed.

DETAILED DESCRIPTION

Reference is now made toFIG.1. The electrical device ofFIG.1includes an application processor (AP)110and an image processor120. In some embodiments, the application processor110is configured to generate mixed images by blending on a screen display (OSD) pattern, a designed intermediate layer, and images of an input video and further to provide the mixed images to the image processor120. The image processor120is configured to generate output images to include a pixel-wise OSD pattern based on analyzed OSD map associated with the OSD pattern and blending images that are decoded according to the mixed images. Due to its pixel-wise accuracy, the OSD pattern can be applied in a broad range of image processing effect protection scenarios, providing significant benefits over alternative methods that lack the same level of precision.

Reference is now made toFIG.2.FIG.2is a schematic diagram of a flowchart of an operation method20of the electronic device10corresponding toFIG.1, in accordance with some embodiments of the disclosure. It is understood that additional operations can be provided before, during, and after the processes shown byFIG.2, and some of the operations described below can be replaced or eliminated, for additional embodiments of the method. The order of the operations/processes may be interchangeable. Throughout the various views and illustrative embodiments, like reference numbers are used to designate like elements. The operation method20includes portions210and220that are described below with reference to the electronic device10ofFIG.1andFIGS.3-10.

According to some embodiments, in the portion210, the application processor110of the electronic device10generates a mixed image215by blending an initial OSD pattern211and an input image213with an intermediate layer212during the operation214. Alternatively stated, the mixed image215includes the initial OSD pattern211and the input image213.

For example, as shown inFIG.3illustrating a schematic diagram of the operation214in the operation method20corresponding toFIG.2, in accordance with some embodiments of the disclosure, the initial OSD pattern211blended by OSD pattern data211aand alpha channel data211bis further combined with image data of the intermediate layer212and image data of the input image213to obtain the mixed image215. In some embodiments, the initial OSD pattern211includes user interface (UI) information overlaying on the input image213and is also referred to as a UI layer.

In some embodiments, each of the OSD pattern data211aand the input image213includes data of color channels (such as red(R), green(G), and blue(B)). The alpha channel data211bis used to define the transparency by grayscale values. The intermediate layer212also includes an alpha channel data and data of color channels. Accordingly, in some embodiments of the mixed image215having RGB channels, a pixel value of one color channel for each pixel in the mixed image215is represented by the equation (1) as below:

in which Linputcorresponds to a value of one color channel for a corresponding pixel in the input image213, Li corresponds to a value of one color channel for a corresponding pixel in the intermediate layer212, at corresponds to an alpha value of the alpha channel for a corresponding pixel in the intermediate layer212, LUIcorresponds to a value of one color channel for a corresponding pixel in the initial OSD pattern211, and αUIcorresponds to an alpha value of the alpha channel for a corresponding pixel in the initial OSD pattern211.

The configurations of color channels are given for illustrative purposes. Various implements are within the contemplated scope of the present disclosure. For example, in some embodiments, color models may include CMYK (stands for Cyan, Magenta, Yellow, and Black) model, HSV (stands for Hue, Saturation, and Value) model, YUV (stands for Luminance (Y), and the two chrominance components (U and V)), or other suitable color models.

Reference is now made toFIGS.4-5.FIG.4is a schematic diagram of an input image413, andFIG.5is a schematic diagram of a mixed image415, in accordance with some embodiments of the disclosure. In some embodiments, the input image413is configured with respect to, for example, the input image213, and the mixed image415is configured with respect to, for example, the mixed image215.

For illustration, the mixed image415includes OSD pattern411, the input image413, and an intermediate layer412including reference points schematically symbolized by dots.

After the mixed image is generated, with reference back toFIG.2again, in portion220, the image processor120receives the mixed image215(e.g., the mixed image415inFIG.5) and generates a raw OSD map222and a blending image223.

Specifically, in the operation221, the image processor120decodes the mixed image215to extract OSD information and color image information. For example, the image processor120adjusts the alpha values at to have a value (e.g., “1”) in the equation (1) and the pixel value for each pixel is represented as below:

in which LROSDcorresponds to a value for a corresponding pixel in the raw OSD map222(e.g., schematically shown a raw OSD map822ofFIG.8generated based on the mixed image415).

In some embodiments, as shown inFIG.6illustrating part of the raw OSD map222, the raw OSD map222includes multiple blocks601that each block601consists of 8 pixels, with a length of 4 pixels and a width of 2 pixels. According to some embodiments, the pixel value (e.g., LROSD) of the reference (situated on top left(LT) of one block601) pixel611in one block601includes the OSD information and represents the OSD information of other pixels612in the same block601. In other words, the remaining pixels612in block601possess an identical pixel value to that of the reference pixel611. It is also noteworthy that all pixels within the block share the same alpha value. To illustrate, when a pixel, having the same pixel coordination as the reference pixel611, in the mixed image415is within the OSD pattern (e.g., the OSD pattern411), the 1-bit pixel value of the reference pixel611may be “1.” Consequently, the entire block601is rendered as “bright,” as depicted in the various embodiments of the raw OSD map822presented inFIG.8. On the contrary, when the pixel in the mixed image415is outside of the OSD pattern, the 1-bit pixel value of the reference pixel611may be “0.” Consequently, the entire block601is rendered as “dark,” as shown inFIG.8.

Moreover, the image processor120adjusts the alpha values αito have a zero value in the equation (1) and the pixel value for each pixel is represented as below:

in which LBLENDcorresponds to a pixel value of one color channel in the blending image223(e.g., a blending image823ofFIG.9generated based on the mixed image415). In some embodiments, regions in the blending image223including no OSD information (i.e., LUIαUIequals to 0,) have same pixel values of corresponding regions in the input image213. For example, with reference toFIGS.4and9, the corresponding regions in the input image413and regions in the blending image823that do not contain OSD information have identical pixel values.

Referring toFIG.2again, in operation224, the image processor120further generates a refined OSD map225based on the raw OSD map222and the blending image223. In some embodiments, a pixel value of a pixel being verified in the blending image223is compared with pixel values of pixels in neighbor blocks (for example, six blocks) in raw OSD map222that are around it and correspond to the reference pixels in the raw OSD map222in order to generate a value for a pixel has the same pixel coordination in the refined OSD map225. In some embodiments, the pixel having non-zero value in the refined OSD map225indicates that a pixel having the same pixel coordination in the blending image223belongs to a refined OSD pattern (e.g., schematically shown a refined OSD map825ofFIG.10generated based on the mixed image415).

Specifically, for example, with reference toFIG.7illustrating part of the blending image223, the pixels situated on the top left in each of blocks701are referred to as reference pixels and key pixels (circled ones)711-714are identified by the raw OSD map222as belonging to the OSD pattern (e.g., the OSD pattern411). In some embodiments, the pixel values of the key pixels711-714in the raw OSD map222are “1,” and the pixel values of non-key pixels (not-circled ones)715-716in the raw OSD map222are “0.”

The image processor120identifies whether the pixel belongs to the OSD pattern based on two rules. In some embodiments, the pixels fulfilled the rules in the blending image223(and/or in the mixed image215) have similar color. The first rule indicates that a difference between a maximum of pixel values of key pixels and a maximum of pixel values of non-key pixels is within an adaptive range, for example, smaller than a predetermined value (e.g.,220.) The second rule indicates that a difference between a maximum of pixel values of the key pixels711-714and the pixel value of a pixel being verified, for example, a pixel721is within an adaptive range, for example, smaller than a half of a maximum in pixel values of key pixels and a maximum in pixel values of non-key pixels.

For example, as shown inFIG.7, regarding the first rule, the image processor120compares a maximum of the pixel values of the key pixels711-714and a maximum of the pixel values of the non-key pixels715-716to obtain a difference. The difference between the maximum of the pixel values of the key pixels711-714and the maximum of the pixel values of the non-key pixels715-716equals to 110 (i.e., the pixel value of “120” of the key pixel714minus the pixel value of “10” of the non-key pixel715,) which is smaller than the predetermined value of “220.” Accordingly, the first rule is fulfilled.

Regarding the second rule, the maximum of the pixel values of the key pixels711-714is 120, and the maximum of the pixel values of the non-key pixels715-716is 10. The difference between the maximum (i.e., “120”) of the pixel values of the key pixels711-714and the pixel value (i.e., “100”) of the pixel721equals to 20, which is smaller than the half (i.e., “55”) of the difference between the maximum of the pixel values of the key pixels711-714and the maximum of the pixel values of the non-key pixels715-716. Accordingly, the second rule is fulfilled.

Based on the discussion above, as the pixel value of the pixel721matches the first and second rules, the pixel721is verified as included in the OSD pattern. Consequently, the pixel corresponding to the pixel721in the refined OSD map225has non-zero value and is rendered as “bright,” as depicted in the various embodiments of the refined OSD map225presented inFIG.9. Alternatively stated, the refined OSD map225indicates positions of pixels in the blending image223wherein differences between the pixel values of the pixels in the blending image223and the maximum of the pixel values of the key pixels711-714are within an adaptive range.

Moreover, in some embodiments, as shown inFIGS.9and10, with the configurations of the present disclosure, the pixels (e.g., around the alphabet “S”) that are in close proximity of the OSD pattern411and have color similar to the OSD pattern411are identified as belonging to OSD pattern and referred to as protection regions. On the contrary, for pixels do not match the first and second rules are referred to as remaining regions. For example, when the difference between the maximum and the pixel value of the pixel721is out of the adaptive range, for example, greater than the half of a maximum difference and a minimum difference between the pixel values of the key pixels711-714and the non-key pixels715-716. The pixel721belongs to the remaining regions.

The configurations of the adaptive range are given for illustrative purposes. Various implements are within the contemplated scope of the present disclosure.

After the refined OSD map225is obtained, the image processor120performs the operation226to generate a refined OSD pattern including protection regions based on the blending image223and the refined OSD map225and further executes the operation227to perform image processing to remaining regions of the blending image223for generating an output image228. In some embodiments, performing image processing includes performing color changing processing to the remaining regions. In various embodiments, the image processing includes image stylization, color correction, image filtering, or other suitable processing.

For example, with reference toFIGS.9to11, a refined OSD pattern829included in the output image828ofFIG.11is generated according to the blending image823ofFIG.9and the refined OSD map825ofFIG.10. In some embodiments, the refined OSD pattern829is different from the OSD pattern411. In various embodiments, a number of pixels included in the refined OSD pattern829is different from a number of pixels included in the OSD pattern411.

Specifically, as illustratively shown inFIG.11, the refined OSD pattern829includes the OSD pattern411and protection regions830in close proximity of part of the OSD pattern411. The number of pixels included in the refined OSD pattern829is greater than the number of pixels included in the OSD pattern411. InFIG.11, the protection regions830are surrounded by the OSD pattern411. The pixel values of pixels of the protection regions830are the same as the pixel values of pixels having the same pixel coordination in the blending image823. In the embodiments of the corresponding regions in the input image413and regions in the blending image823that do not contain OSD information having identical pixel values, the pixel values of pixels of the protection regions830are the same as the pixel values of pixels having the same pixel coordination in the input image413. Alternatively stated, some regions, corresponding to the protection regions830, of the input image413are “protected” and the image data thereof are retained in the output image828.

In some embodiments, the output image828includes remaining regions831being performed by image processing. As shown inFIGS.9and11, the remaining regions inFIG.9, apart from those that correspond to the refined OSD pattern829, are darkened and appear as the remaining regions831inFIG.11.

In some approaches, the transmission of user interface information includes OSD (on-screen display) information, which is integrated into 4×2 blocks. This integration method is not very accurate because it marks the remaining 7 points as long as there is UI in the upper left corner of the 4×2 block. Additionally, the sampling method used for integration is done only once every 4×2 blocks, so delicate UI information may be ignored. As a result, the remaining 7 points are determined based on the presence or absence of UI in the upper left corner of the OSD, leading to a high likelihood of redundant or missing information.

Compared with some approaches, the present disclosure involves optimizing the OSD map and blending it with the color map, for example, the input image. By utilizing the fact that the upper left corners of the blocks of the raw OSD map include accurate OSD information and comparing color image data, every pixel is verified by of the neighboring key pixels corresponding to OSD pattern, and thus pixel-wise OSD pattern with significant accuracy is provided.