Image processing method and device

An image processing method and device for processing multiple rows of pixels of an image simultaneously with a single instruction. The processing includes selecting a pixel window having a plurality of pixels of an image spanning across multiple rows and columns, building vertical and horizontal load registers to include the plurality of pixels of the selected pixel window, and simultaneously processing selected pixels of the plurality of pixels included in the vertical and horizontal load registers using a single instruction, wherein the vertical and horizontal load registers are shifted when the selected pixels are processed. Accordingly, a method and device for efficient processing of an image is provided.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to processing an image. More particularly, the present invention relates to processing multiple rows of pixels of an image simultaneously with a single instruction.

2. Description of the Related Art

Generally, analyzing and processing of a selected pixel of an image includes building a pixel window having neighboring pixels of the selected pixel and processing the selected pixel with respect to the neighboring pixels in the pixel window. Accordingly, pixels in a pixel window are processed with respect to one another.

A typical hardware implementation of image processing utilizes a large buffer to store multiple pixels of image data. This requires the use of an application specific integrated circuit (ASIC) at a substantial cost for implementation. Further, a typical software based implementation of an imaging process requires multiple instruction cycles for building pixel windows, processing the pixel windows and loading pixels to the pixel windows, thereby consuming a higher processor bandwidth and requiring use of larger and faster processors. Unfortunately, the typical hardware and software solutions for image processing are not cost-efficient.

Accordingly, it is important to provide an image processing method and device without requiring high processor bandwidth. It is also important to provide an image processing method and device to eliminate the need to build pixel windows manually. Therefore, there is a need for executing image processing without having to utilize faster and larger processors.

SUMMARY OF THE INVENTION

The present invention provides a method of processing multiple rows of pixels of an image simultaneously with a single instruction. According to an aspect of the present invention, the processing includes building the multiple rows of pixels of the image in horizontal and vertical load registers that are arranged with respect to each other to correspond with the image. Further, the processing includes processing of multiple rows of pixels that are stored in non-sequential locations in a memory.

The present invention also provides a method of selecting pixel windows having a plurality of pixels of an image spanning across multiple rows and columns, building vertical and horizontal load registers to include the plurality of pixels of the selected pixel windows, and simultaneously processing selected pixels of the plurality of pixels included in the vertical and horizontal load registers using a single instruction, wherein the vertical and horizontal load registers are shifted when the selected pixels are processed. Further, the shifting of the vertical and horizontal load registers includes shifting a pixel in each of the horizontal load registers into at least one of the vertical load registers when the selected pixels are processed. The present invention also includes changing the plurality of pixels in the selected pixel windows each time the selected pixels are processed.

The present invention also provides an image processing device including a processor for processing multiple rows of pixels of an image simultaneously with a single instruction, where the multiple rows of pixels of the image are arranged in vertical and horizontal load registers. Further, the present invention provides a number of the horizontal and vertical load registers based on an imaging algorithm used.

Further, a method of the present invention includes building vertical and horizontal load registers to include a plurality of pixels of an image spanning across multiple rows and columns, and simultaneously processing selected pixels of the plurality of pixels spanning across the multiple rows using a single instruction, wherein most significant bits of each of the horizontal load registers are shifted into the vertical load registers when the selected pixels of the image are processed, and at least one of the vertical load registers is overwritten.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1is a flow chart illustrating an imaging process200, according to an embodiment of the present invention. As shown inFIG. 1, process200includes operations70,80,90and100. Accordingly, operation70includes selecting pixel windows having a plurality of pixels of an image spanning across multiple rows and columns. From operation70, process200moves to operation80, where vertical and horizontal load registers are built to include the plurality of pixels of the selected pixel windows. From operation80, process200moves to operation90, where selected pixels of the plurality of pixels included in the vertical and horizontal load registers are simultaneously processed using a single instruction. From operation90, process200moves to operation100, where the vertical and horizontal load registers are shifted when the selected pixels are processed in operation90.

FIG. 2is a diagram illustrating pixel windows, according to an embodiment of the present invention.FIG. 2illustrates pixel windows10and10ahaving a plurality of pixels of an image spanning across multiple rows and columns. The pixel windows10aand10bare indicated using gray shading to respectively illustrate pixel contents of the pixel windows10aand10b. The pixel windows10and10aare constructed to include neighboring pixels of selected pixels20and20a, respectively, such that processing of pixels20and20ais implemented with respect to corresponding neighboring pixels. For example, the neighboring pixels with respect to pixel20are those pixels in rows above and below and pixels in columns to the right and left of pixel20.

Referring toFIG. 2, the pixel window10and10ainclude pixels that span across 5 rows and 3 columns. WhileFIG. 2is explained using the pixel windows10and10ahaving pixels that span across 5 pixel rows and 3 pixel columns, the present invention is not limited to this size of a pixel window. For example, 4 pixel rows by 4 pixel columns, 10 pixel rows by 4 pixel columns, etc., can be processed using the present invention. For ease of explanation, it is assumed that each pixel represents one bit, however, the present invention is not limited to such and can be implemented to process pixels having more than one bit.

As shown inFIG. 2, pixels of an image are located within different byte boundaries30a,30band30cin a memory (not shown). InFIG. 2, pixel window10aillustrates one of the subsequent positions of pixel window10when imaging process200ofFIG. 1is applied to pixel window10, according to an embodiment of the present invention. The pixel window10moves or shifts across byte boundaries30aand30bas denoted by arrow11based on execution of imaging process200ofFIG. 1. The operation resulting in the shifting of pixel window10is described in detail in the following paragraphs.

FIG. 3is a diagram illustrating vertical and horizontal load registers of a pixel window, according to an embodiment of the present invention. InFIG. 3, vertical load registers40aand40band horizontal load registers50a,50b,50c,50dand50eare loaded when pixel window10is to be processed according to imaging process200ofFIG. 1. Specifically, horizontal load registers50a,50b,50c,50dand50ecorrespond to registers1through5(Reg1through Reg5inFIG. 3). The horizontal load registers50a,50b,50c,50dand50econtain all the pixels within byte boundaries30aand30bofFIG. 2while vertical load registers40aand40bcontain 2 pixel columns prior to byte boundary30a, which are the remaining pixels in pixel window10. Most significant bits60of the horizontal load registers50a,50b,50c,50dand50eare placed closest to the vertical load registers40aand40b.

FIGS. 4A,4B and4C are diagrams illustrating pixel windows and corresponding vertical and horizontal load registers, according to an embodiment of the present invention.FIG. 4illustrates application of operation100ofFIG. 1to pixel window10inFIG. 2. The vertical load registers40aand40band the horizontal load registers50a,50b,50c,50dand50einFIG. 3are shifted when pixels in pixel window10are processed based on operation90of imaging process200inFIG. 1. Accordingly, when pixel window10is processed, pixels in vertical load register40ainFIG. 3are overwritten by pixels in vertical load register40b. Accordingly, the contents of vertical load register40bare transferred to vertical load register40a. Further, horizontal load registers50a,50b,50c,50dand50eare also shifted to the left so that most significant bits60of the horizontal load registers50a,50b,50c,50dand50eare transferred to vertical load register40b.

FIG. 4Ashows pixel window10bthat results when pixel window10ofFIG. 3is processed and horizontal load registers50a,50b,50c,50dand50eand vertical load registers40aand40bare all shifted to the left according to imaging process200ofFIG. 1. As denoted by arrow11inFIG. 2, once pixel window10is processed resulting in shifting of horizontal load registers50a,50b,50c,50dand50eand vertical load registers40aand40b, pixel window10binFIG. 4Ais constructed. Accordingly, pixel contents of horizontal load registers50a,50b,50c,50dand50eand vertical load registers40aand40bofFIG. 3changes. As shown inFIG. 4A, horizontal load registers51a,51b,51c,51dand51eand vertical load registers41aand41bhave different pixel contents than horizontal load registers50a,50b,50c,50dand50eand vertical load registers40aand40bofFIG. 3. Further, when pixel window10is processed and pixels in the horizontal load registers50a,50b,50c,50dand50eare shifted towards the vertical load registers40aand40binFIG. 3, the resulting horizontal load registers51a,51b,51c,51dand51eare loaded with values that are not required for the processing of pixel window10b. For example, column A containing X values inFIG. 4Aillustrates that the last bits of horizontal load registers51a,51b,51c,51dand51econtain values that are not required for processing pixel window10b.

Referring toFIG. 4B, when pixel window10bofFIG. 4Ais processed, horizontal load registers51a,51b,51c,51dand51eand vertical load registers41aand41bshift to the left, resulting in pixel window10cas shown inFIG. 4B. Specifically, pixels in vertical load register41aofFIG. 4Aare overwritten by pixels in vertical load register41b, and the most significant bits60of each horizontal load registers51a,51b,51c,51dand51eare loaded into vertical load register41b. As described above, as most significant bits60of the horizontal load registers continue to be shifted or transferred to at least one of the vertical load registers, columns A and B inFIG. 4Bcontain values that are not required for processing of pixel window10c. Similarly, when horizontal load registers52a,52b,52c,52d,52eand vertical load registers42aand42binFIG. 4Bare shifted to the left, horizontal load registers53a,53b,53c,53d,53eand vertical load registers43aand43bresult as shown inFIG. 4C. Further, when imaging process200is applied to pixel window10cinFIG. 4B, pixel column and rows of pixel window10cchanges to pixel window10d, according to an embodiment of the present invention. As illustrated inFIG. 4C, columns A, B and C contain values that are not required for processing pixel window10d.

FIGS. 5A,5B and5C are diagrams illustrating respective shifting operations of the pixel windows10b,10cand10dofFIGS. 4A,4B and4C, according to an embodiment of the present invention.FIG. 5illustrates the shifting movement of pixel windows upon processing of pixel windows10,10band10c. Specifically, upon processing pixels in pixel window10bofFIG. 5A, content of pixel window10bis sequentially changed to pixel windows10cand10dinFIGS. 5B and 5C. As is evident fromFIGS. 5A,5B and5C, pixel window10dinFIG. 5Cis closer to byte boundary30bthan pixel windows10band10cofFIGS. 5A and 5B.

FIGS. 6A and 6Bare diagrams illustrating an exemplary pixel window and vertical and horizontal load registers, according to an embodiment of the present invention. InFIG. 6, pixel window10eis formed of pixels in the last 3 pixel columns before the byte boundary30binFIG. 2. As shown inFIG. 6A, horizontal load registers54a,54b,54c,54dand54eand vertical registers44aand44bare built for pixel window10e. Then, as shown inFIG. 6B, the vertical load registers45aand45bkeep getting reloaded with pixels until shifting of all the pixels in horizontal load registers54a,54b,54c,54dand54eis completed. According to an embodiment of the present invention, an imaging process90is not executed because only vertical load registers45aand45bcontain pixels while the most significant bits60of horizontal load registers55a,55b,55c,55dand55eare not loaded with pixels.

FIG. 7is a diagram illustrating shifting operation of the pixel window10eofFIG. 6A, according to an embodiment of the present invention. As mentioned above, pixel window10econtains the far right three pixel columns within boundary30a. Accordingly, when operation90ofFIG. 1is applied to pixel window10e, corresponding horizontal load registers54a,54b,54c,54dand54eand vertical registers44aand44bare shifted to the left so that most significant bits60of54a,54b,54c,54dand54einFIG. 6Aare transferred to vertical register44bwhile pixels in vertical register44bare shifted to vertical register44a. Further, when pixel window10eofFIG. 6Ahaving the last 3 pixel columns before byte boundary30bis processed and horizontal load registers55a,55b,55c,55dand55eand vertical load registers45aand45bofFIG. 6Bare loaded, pixel columns subsequent to boundary byte30bcontinue to be loaded into the horizontal and vertical load registers as described below in relation toFIGS. 8A and 8B.

FIGS. 8A and 8Bare diagrams illustrating pixel windows and corresponding vertical and horizontal load registers, according to an embodiment of the present invention.FIG. 8Ashows pixel window10fhaving initial 2 pixel columns from pixel columns before byte boundary30band 1 pixel column having pixels after byte boundary30bas shown inFIG. 9A. Further,FIG. 8Bshows pixel window10ghaving 1 pixel column before byte boundary30band 2 pixel columns after byte boundary30bas shown inFIG. 9B. As shown inFIGS. 8A and 8B, vertical and horizontal load registers are built for pixel window10fand pixel window10g, respectively. For pixel window10f, horizontal load registers56a,56b,56c,56d,56eare constructed, and for pixel window10g, horizontal load registers57a,57b,57c,57d,57eare constructed. Further, vertical load registers46a,46binFIG. 8Aand vertical load registers47aand47bare respectively constructed for pixel window10fand pixel window10g.

As mentioned above,FIG. 9Ashows that pixel window10fhas 2 initial pixel columns before byte boundary30band 1 pixel column subsequent to byte boundary30b. AndFIG. 9Bshows that pixel window10ghas the initial pixel column before byte boundary30band 2 pixel columns after byte boundary30b. Accordingly,FIGS. 9A and 9Bshow shifting of pixel window10fto pixel window10gas imaging process200is applied to pixel window10f.

FIG. 10is a diagram illustrating a pixel window, according to an embodiment of the present invention. As shown inFIG. 10, pixel window10hhas 8 pixel rows by 3 pixel columns. As discussed above, byte boundaries30a,30band30crepresent byte boundaries in a memory within which pixels of an image are stored. WhileFIG. 10is explained using pixel window10hhaving 8 pixel rows by 3 pixel columns, the present invention is not limited to processing a pixel window of that size.

FIG. 11is a diagram illustrating vertical and horizontal registers of the pixel window ofFIG. 10, according to an aspect of the present invention. As shown inFIG. 11, multiple pixel windows10hI, II, III and IV having 3 pixel columns by 5 pixel rows are constructed for pixel window10hofFIG. 10that has 8 pixel rows by 3 pixel columns. Accordingly, for each of the multiple pixel windows10hI, II, III and IV, a corresponding instruction is implemented to process pixel columns and rows in pixel windows10hI, II, III and IV where only pixel window10hIV shifts horizontal load registers58a,58b,58c,58d,58e,58f,58gand58hand vertical load registers48aand48b. This allows the present invention to implement efficient image processing by serially executing multiple pixel windows of a larger pixel window with corresponding instructions. As shown inFIG. 11, horizontal load registers58a,58b,58c,58d,58e,58f,58gand58hand vertical load registers48aand48bare constructed for pixel window10hofFIG. 10. Upon processing of pixels in pixel windows10hIV, most significant bits60of horizontal load registers58a,58b,58c,58d,58e,58f,58gand58hare shifted into vertical load register48b. Moreover, pixels in vertical load register48bare shifted to the left to overwrite pixels of vertical load register48a. Further, according to another embodiment of the present invention, pixel windows10hI, II, III and IV are processed simultaneously using a single instruction, causing the horizontal load registers58a,58b,58c,58d,58e,58f,58gand58hand the vertical load registers48aand48bto be shifted.

For example, central processing units such as, Tensilica XTensa T1050, ARC600 and ARC700, MIPS Pro series can be used to process multiple rows of pixels of an image simultaneously with a single instruction, according to the present invention. However, while the imaging process of the present invention can be utilizing Tensilica XTensa T1050, ARC600 and ARC700, MIPS Pro series central processing units, the present invention is not limited to use by these processing units.

Accordingly, the method of the present invention enables processing of multiple rows of pixels of an image simultaneously with a single instruction. The method further includes selecting a pixel window having a plurality of pixels of an image spanning across multiple rows and columns, building vertical and horizontal load registers to include the plurality of pixels of the selected pixel window, where selected pixels are simultaneously processed and the vertical and horizontal load registers are shifted when the selected pixels are processed.

The present invention also provides processing of multiple rows of pixels that are stored in non-sequential locations in a memory. Further, the present invention includes shifting most significant bits of each of the horizontal load registers into the vertical load registers when the multiple rows of pixels of the image are processed, where at least one of the vertical load registers is overwritten.

The present invention also provides an image processing device having a processor for processing multiple rows of pixels of an image simultaneously with a single instruction, wherein the multiple rows of pixels of the image are arranged in vertical and horizontal load registers. Further, the processing of the multiple rows of pixels of the image includes shifting of the pixels of the image that are arranged in the vertical and horizontal load registers when the multiple rows of pixels are processed.

Additionally, the present invention provides building vertical and horizontal load registers to include a plurality of pixels of an image spanning across multiple rows and columns, and simultaneously processing selected pixels of the plurality of pixels spanning across the multiple rows using a single instruction, wherein most significant bits of each of the horizontal load registers are shifted into the vertical load registers when the selected pixels of the image are processed, and at least one of the vertical load registers is overwritten.