Image processing apparatus

An image processing apparatus for detecting a desired image-element from a given image, includes: a storing unit for storing a plurality of first templates indicating the desired image-element and a plurality of second templates stored corresponding to the first templates, each of the second templates indicating a part of a corresponding one of the first templates; the first comparing unit for selecting any of the first templates and comparing the selected first template with the given image; and the second comparing unit for comparing a second template corresponding to the selected first template selected by the first comparing unit with the given image; and a detecting unit for detecting the desired image-element from the given image based on the comparison results by the first and second comparing units.

This patent application claims priority from a Japanese patent application No. 2002-006809 filed on Jan. 15, 2002, the contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus. More particularly, the present invention relates to an image processing apparatus for detecting a desired image-element from a given image.

2. Description of the Related Art

As an image recognition apparatus that identifies an image to be recognized, an apparatus disclosed in Japanese Patent Application Laid-Open No.5-174149 has been conventionally known. This image recognition apparatus converts an object image, that is an object of recognition, into a coarse-mosaic image, and scans an unknown image by using this coarse-mosaic image as a feature of the object image so as to roughly find the position and size of the object image. Then, the image recognition apparatus produces a fine-mosaic image from the object image and scans an area around the position in the unknown image, that was roughly found, so as to find the object image.

In the conventional image recognition apparatus, the number of pixels of the mosaic image used as the feature of the object image is changed in three stages, so as to calculate the distance between the mosaic image and the unknown image in each stage. This image recognition apparatus detects the object image to be recognized based on the thus calculated distance.

The conventional image recognition apparatus, however, stores a plurality of templates for finding the object image to be recognized, and compares the templates with the object image one by one so as to detect the object image to be recognized. Thus, it spends much time to compare the templates with the object image by the conventional image recognition apparatus.

SUMMARY OF THE INVENTION

Therefore, it is an object of the present invention to provide an image processing apparatus, which is capable of overcoming the above drawbacks accompanying the conventional art. The above and other objects can be achieved by combinations described in the independent claims. The dependent claims define further advantageous and exemplary combinations of the present invention.

According to the first aspect of the present invention, an image processing apparatus for detecting a desired image-element from a given image, comprises: a storing unit operable to store a plurality of first templates indicating the desired image-element and a plurality of second templates stored corresponding to the plurality of first templates, each of the second templates indicating a part of a corresponding one of the plurality of first templates; a first comparing unit operable to select any one of the first templates and compare the selected one first template with the given image; and a second comparing unit operable to compare a second template corresponding to the selected one first template selected by the first comparing unit with the given image; and a detecting unit operable to detect the desired image-element from the given image based on the comparison results by the first and second comparing units.

The storing unit may store at least one first template that corresponds to the plurality of second templates.

The storing unit may store at least one second template that corresponds to other one of the second templates.

The second comparing unit may compare each of a plurality ones of the second templates, that correspond to the selected one first template selected by the first comparing unit, with the given image.

The storing unit may store a second template that has a higher resolution than that of the corresponding first template.

The summary of the invention does not necessarily describe all necessary features of the present invention. The present invention may also be a sub-combination of the features described above. The above and other features and advantages of the present invention will become more apparent from the following description of the embodiments taken in conjunction with the accompanying drawings.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1illustrates an exemplary arrangement of an image processing apparatus100according to the present invention. The image processing apparatus100detects a desired image from a given image, and includes a template storing unit10, a mosaic-image producing unit20, a detecting unit60, a history storing unit70and a selecting unit80.

The template storing unit10stores a plurality of templates showing the entire desired image-element or a part thereof. It is preferable that the template storing unit10stores a plurality of templates each has a lower resolution than that of the desired image-element, that are produced based on the desired image-element. For example, the template storing unit10stores a mosaic image of the desired image-element as the template. In this example, the template storing unit10stores a plurality of first templates each showing the entire desired image-element and a plurality of second templates each having a lower resolution than that of the first template and showing a part of the first template, and a plurality of third templates each having a lower resolution than that of the second template and showing a part of the second template.

The template storing unit10stores the second templates in such a manner that each second template is stored so as to be associated with any of the first templates. Moreover, the template storing unit10stores the second template that is made to correspond to a plurality of first templates and that shows a part of any of the first templates. For example, the template storing unit10stores each second template so as to be associated with a plurality of first templates. Similarly, the template storing unit10stores each of the third templates so as to be associated with any of the second templates.

The template storing unit10may each third template so as to be associated with a plurality of second templates. In this example, the template storing unit10stores a plurality of first templates showing faces of a plurality of persons; stores second templates showing elements of face such as an eye, a nose, and a mouth in such a manner that each second template is associated with any of the first templates; and stores third templates showing details of the eye, nose and mouth in such a manner that each third template is associated with any of second templates.

The mosaic-image producing unit20produces, based on a given image, an image having a lower resolution than that of the given image, i.e., a mosaic image. The mosaic-image producing unit20may produce a plurality of mosaic images having different resolutions. For example, the mosaic-image producing unit20may produce the first mosaic image having approximately the same resolution as that of the first template, the second mosaic image having approximately the same resolution as that of the second template and the third mosaic image having approximately the same resolution as that of the third template.

Moreover, the mosaic-image producing unit20may produce a mosaic image of a part of the given image. For example, the mosaic-image producing unit20may produce the mosaic image by extracting a part of the given image so that the number of pixels of the template stored in the template storing unit10is approximately the same as the number of pixels of the mosaic image. In this example, the mosaic-image producing unit20produces the first mosaic image having approximately the same number of pixels as that of pixels of the first template, the second mosaic image having approximately the same number of pixels as that of pixels of the second template and the third mosaic image having approximately the same number of pixels as that of pixels of the third template.

The detecting unit60detects a desired image-element from the given image by using any of a plurality of templates stored in the template storing unit10. The detecting unit60compares the mosaic image produced by the mosaic-image producing unit20with the template stored by the template storing unit10so as to detect the desired image-element from the given image based on the comparison result. The detecting unit60includes a plurality of matching means for comparing the mosaic image produced by the mosaic-image producing unit20with the template stored in the template storing unit10. For example, the detecting unit60may compare the template with the mosaic image by a method based on the sum of differences between corresponding pixels of the template and the mosaic image, a method based on the distance of pixel-value vector between the template and the mosaic image, or the like. Moreover, the detecting unit60may compare the template with the mosaic image based on the eigenspace method, the kernel-based method, or the like.

The history storing unit70stores a used history of the template that was used by the detecting unit60. For example, the history storing unit70stores the number of times the detecting unit60used the template for each of a plurality of templates. The history storing unit70may store the used history of the template in a predetermined period. For example, the history storing unit70may store which template was used by the detecting unit60and the number of times that template was used in a predetermined period.

The history storing unit70may store a time at which each of the templates was used last by the detecting unit60. The history storing unit70may store the detection result in the detecting unit60for each of a plurality of templates. For example, the history storing unit70may store the number of times each template was used and the number of times the detecting unit60detected the desired image-element by using each template. The detecting unit60may select any of a plurality of templates stored in the template storing unit10based on the used histories stored in the history storing unit70.

The selecting unit80selects any of a plurality of matching means in the detecting unit60. For example, the selecting unit80selects any of a plurality of matching means in the detecting unit60based on the number of pixels of the given image. The selecting unit80may select any of a plurality of matching means in the detecting unit60based on whether the given image is monochrome or color.

The detecting unit60includes the first comparing unit30-1, the second comparing unit30-2, the third comparing unit30-3, a coincidence calculating unit40and an image-element detecting unit50.

The first comparing unit30-1calculates the coincidence between the first template of a plurality of templates stored in the template storing unit10and the first mosaic image produced by the mosaic-image producing unit20. The first comparing unit30-1has a plurality of matching means described before for calculating the coincidence between the first template and the first mosaic image. For example, the first comparing unit30-1may calculate the coincidence between the first template and the first mosaic image by a matching means using a method based on the sum of differences of pixel values between corresponding pixels of the first template and the first mosaic image, a method based on the distance of pixel value vector between the first template and the first mosaic image, and the like. The first comparing unit30-1may calculate the coincidence between the first template and the first mosaic image by a matching means using the eigenspace method, the kernel-based method or the like. The selecting unit80selects any of the above-mentioned matching means, and the first comparing unit30-1calculates the coincidence between the first template and the first mosaic image based on the selected matching means.

The first comparing unit30-1may calculate the coincidence between the first template that corresponds to a desired image-element and the first mosaic image. Moreover, the first comparing unit30-1selects the first template to be used, based on the used histories of a plurality of first templates stored in the history storing unit70.

The first comparing unit30-1may calculate the coincidence between each of all the first templates stored in the template storing unit10and the first mosaic image. In this case, the first comparing unit30-1may output the highest one of the calculated coincidences. Moreover, the first comparing unit30-1may calculate the coincidence(s) between one or ones of a plurality of first templates stored in the template storing unit10, that is/are specified by a user, and the first mosaic image. In addition, the template storing unit10stores a plurality of first templates in such a manner that the first templates are classified into a plurality of groups, and the first comparing unit30-1may calculate the coincidences between the first templates in one or a plurality of groups specified by the user and the first mosaic image. Also in this case, the first comparing unit30-1may output the highest one of the calculated coincidences to the coincidence calculating unit40.

The second comparing unit30-2calculates the coincidence between the second template having a higher resolution than that of the first template, that shows a part of the first template, and the second mosaic image having a higher resolution than that of the first mosaic image, that was produced by the mosaic-image producing unit20. The second comparing unit30-2calculates the coincidence between the second template that was made to correspond to the first template used by the first comparing unit30-1and the second mosaic image. Alternatively, the second comparing unit30-2selects any of the second templates that were made correspond to the first template used by the first template so as to calculate the coincidence with the selected second template and the second mosaic image. The second comparing unit30-2may select a plurality of second templates. The second comparing unit30-2may select the second template based on the used histories of the second templates stored in the history storing unit70.

The second comparing unit30-2may output all the calculated coincidences to the coincidence calculating unit40. For example, the second comparing unit30-2calculates the coincidences between the respective second templates showing an eye, nose or mouth and the second mosaic image so as to output all the calculated coincidence to the coincidence calculating unit40.

The second comparing unit30-2may calculate the coincidences between all the second templates that were made to be associated with the first template corresponding to the coincidence output from the first comparing unit30-1to the coincidence calculating unit40, and the second mosaic image. In this case, the second comparing unit30-2may output the highest one of the calculated coincidences to the coincidence calculating unit40, like the first comparing unit30-1.

The second comparing unit30-2has a plurality of matching means like the first comparing unit30-1. The second comparing unit30-2may calculate the coincidence between the second template and the second mosaic image by using the matching means selected by the selecting unit80.

Similarly, the third comparing unit30-3calculates the coincidence between the third template having a higher resolution than that of the second template, that shows the entire second template or a part of the second template, and the third mosaic image produced by the third image producing image20. The third comparing unit30-3may select the third template to be used from the third plates associated with the second template that corresponds to the coincidence output from the second comparing unit30-2to the coincidence calculating unit40, so as to calculate the coincidence between the third template thus selected and the third mosaic image. Alternatively, the third comparing unit30-3may calculate the coincidences between all the third templates associated with the second template that corresponds to the coincidence output from the second comparing unit30-2to the coincidence calculating unit40, and the third mosaic image. Moreover, the third comparing unit30-3may select the third template to be used based on the used histories of the respective third templates stored in the history storing unit70.

The third comparing unit30-3has a plurality of matching means, like the first comparing unit30-1. The third comparing unit30-3may calculate the coincidence between the third template and the third mosaic image by using the matching means selected by the selecting unit80.

The selecting unit80may select a matching means for each of a plurality of comparing units30independently of the matching means for the other comparing unit. That is, the selecting unit80may select different matching means or the same matching means for a plurality of comparing units30.

The coincidence calculating unit40calculates the coincidence between a desired image-element and a given image based on all the received coincidences. In this example, the coincidence calculating unit40calculates the coincidence between the desired image-element and the given image based on the coincidences that the first, second and third comparing units30-1,30-2and30-3calculated and output, respectively.

The image-element detecting unit50detects the desired image-element from the given image based on the coincidence calculated by the coincidence calculating unit40. In a case where the coincidence calculated by the coincidence calculating unit40is larger than a predetermined threshold value, the image-element detecting unit50determines that the desired image element was detected from the given image.

It is preferable that the coincidence calculating unit40has a means for multiplying the coincidence calculated by the comparing unit30-nby a weight coefficient. In this example, the coincidence calculating unit40includes a means for multiplying the coincidence calculated by the first comparing unit30-1by the first weight coefficient, a means for multiplying the coincidence calculated by the second comparing unit30-2by the second weight coefficient and a means for multiplying the coincidence calculated by the third comparing unit30-3by the third weight coefficient. The coincidence calculating unit40may calculate the coincidence between the desired image-element and the given image based on the sum of the coincidences which were multiplied by the weight coefficients.

The image processing apparatus100in this example can select an appropriate template efficiently because a plurality of templates are made to correspond to any of higher order templates. Moreover, the image processing apparatus100in this example can reduce leak in detection in each of a plurality of stages because the coincidence between the desired image-element and the given image is calculated based on all the coincidences calculated in a plurality of stages.

FIG. 2shows an exemplary data structure of the template stored in the template storing unit10. The template storing unit10stores a plurality of templates. The template storing unit10stores a plurality of templates as a hierarchical data structure having a plurality of layers. In this example, the template storing unit10stores a plurality of templates in form of three-layered data hierarchy, as shown inFIG. 2. In addition, the template storing unit10in this example stores templates each showing a face a person as the first templates12, templates each showing an eye or a nose of a person as the second templates14,16and templates each showing a part of the second template14,16as the third templates18,22and24.

The template storing unit10stores the lower template so as to correspond to any of upper templates, as described above. In this example, the second template14ais made to correspond to the first template12aand the first template12b. The template storing unit10may store the upper template so as to correspond to a plurality of lower templates. In other words, the template storing unit10may store at least one first template12that corresponds to a plurality of second template14. In this example, the first template12ais made to correspond to the second template14aand the second template14b.

Moreover, the template storing unit10may store a template so as to correspond to any of other templates in the same layers of hierarchy. In this example, the template storing unit10stores at least second template that corresponds to another second template. For example, the template storing unit10may store the second template14ashowing an eye so as to correspond to any of the second templates16each showing a nose. In this case, the second comparing unit30-2may calculate the coincidences between a plurality of second templates made to correspond to each other and the second mosaic images, respectively.

In addition, the template storing unit10may store a plurality of second templates having a predetermined relationship therebetween. In this example, the template storing unit10stores an interrelationship of positions shown by the respective second templates14,16as that relationship. The second comparing unit30-2may calculate the coincidence between the second template and the second mosaic image based on that relationship.

FIG. 3shows an exemplary arrangement of each of a plurality of comparing units30-n. Each comparing unit30-nincludes a similar image detecting unit30-n-aand a calculator30-n-b. The similar image detecting unit30-1-areceives the first mosaic image and the first template, and detects from the first mosaic image a similar image-element that is similar to the first template. The calculator30-1-bcalculates the coincidence between the similar image-element detected by the similar image detecting unit30-1-aand the first template. In another exemplary arrangement, the calculator30-1-bmay calculate the coincidence with the first template in each of a plurality of areas of the first mosaic image, and the similar image detecting unit30-1-amay detect the area having the coincidence that is larger than a predetermined threshold value as the similar image-element. The comparing unit30-1-aoutputs the coincidence between the similar image-element and the first template to the coincidence calculating unit40. Moreover, the similar image detecting unit30-1-aoutputs the position in the first mosaic image, at which the similar image-element was detected, to the second comparing unit30-1.

The second comparing unit30-2has the same or similar function and structure as/to those of the first comparing unit30-1. The similar image calculating unit30-2-areceives the second mosaic image and the second template, and detects from this second mosaic image the similar image-element that is similar to the second template. The calculator30-2-bcalculates the coincidence between the similar image-element detected by the similar image detecting unit30-2-aand the second template. The similar image detecting unit30-2-amay detect the similar image-element from the second mosaic image based on the position at which the first similar image detecting unit30-1-adetected the similar image-element.

In another exemplary arrangement, the calculator30-2-bmay calculate the coincidence with the second template in each of a plurality of areas of the second mosaic image and the calculator30-2-amay detect the area having the coincidence that is larger than a predetermined threshold value as the similar image-element. The comparing unit30-2outputs the coincidence between the similar image-element and the second template to the coincidence calculating unit40. Moreover, the similar image detecting unit30-2-aoutputs the position in the second mosaic image, at which the similar image was detected, to the third comparing unit30-3.

The third comparing unit30-3has the same or similar function and structure as/to those of the second comparing unit30-2. The similar image calculating unit30-3-areceives the third mosaic image and the third template, and detects from this third mosaic image a similar image-element that is similar to the third template. The calculator30-3-bcalculates the coincidence between the similar image-element detected by the similar image detecting unit30-3-aand the third template. The similar image detecting unit30-3-amay detect the similar image-element from the third mosaic image based on the position at which the first similar image detecting unit30-1-adetected the similar image-element.

In another exemplary arrangement, the calculator30-3-bmay calculate the coincidence with the third template in each of a plurality of areas of the third mosaic image and the calculator30-3-amay detect the area having the coincidence that is larger than a predetermined threshold value as the similar image-element. The comparing unit30-3outputs the coincidence between the similar image-element and the third template to the coincidence calculating unit40.

In addition, a plurality of comparing units30-nhave means for performing rotation, magnifying and/or size-reduction of the template or mosaic image that was received. The comparing unit30-nmay calculate the coincidence based on the template or mosaic image thus rotated, magnified and/or reduced in size.

FIGS. 4A and 4Bshow an operation for detecting the coincidence in the image processing apparatus100. In this example, the image processing apparatus100calculates the coincidence between a face of a person as a desired image-element and a given image.FIG. 4Ashows an image given to the mosaic-image producing unit20. The mosaic-image producing unit20produces the first mosaic image based on the given image and supplies the first mosaic image to the first comparing unit30-1described referring toFIG. 1. It is desirable that the mosaic-image producing unit20produce the first mosaic image having approximately the same resolution as that of the first template stored in the template storing unit10.

The first storing unit30-1detects a similar image-element that is similar to the face of person from the first mosaic image based on the first template, as described referring toFIG. 3. In this example, the first comparing unit30-1detects the similar image-element at a position shown inFIG. 4A.FIG. 4Bshows the similar image-element detected by the first comparing unit30-1. The first comparing unit30-1calculates the coincidence between the first mosaic image and the first template based on the thus detected similar image-element. In this example, the first comparing unit30-1calculates the coincidence between the first mosaic image and the first template, ρ1,1=0.98. Please note that the coincidence ρ in this example has a value of 1 in a case where the mosaic image and the template are fully coincident with each other and has a value of 0 in a case where they are not coincident.

FIGS. 5A,5B and5C show an operation following the coincidence detection described inFIGS. 4A and 4B. The mosaic-image producing unit20produces the second mosaic image based on the given image, and supplies the second mosaic image to the second comparing unit30-1described referring toFIG. 1. It is desirable that the mosaic-image producing unit20produce the second mosaic image having approximately the same resolution as that of the second template stored in the template storing unit10. The mosaic-image producing unit20may produce the second mosaic image based on the position in the given image at which the first comparing unit30-1detected the similar image-element. That is, the mosaic-image producing unit20may produce the second mosaic image showing a part of the first mosaic image based on the given image. Moreover, the mosaic-image producing unit20may produce the second mosaic image obtained by changing all areas in the given image to mosaic areas.

The second comparing unit30-2detects a similar image-element that is similar to a human face element, from the second mosaic image based on the second template, described referring toFIG. 3. The second comparing unit30-2may detect a plurality of similar image-elements. In this example, the second comparing unit30-2detects areas showing eyes and a nose as the similar image-elements from the second mosaic image.FIG. 5Ashows the similar image-elements detected by the second comparing unit30-2. The second comparing unit30-2calculates the coincidence between the second mosaic image and the second template based on the thus detected similar image-elements. In this example, the second comparing unit30-2calculates the coincidence between the second mosaic image and the second template, ρ2,1=0.85 and ρ2,2=0.7.

The mosaic-image producing unit20produces the third mosaic image based on the given image, and supplies the third mosaic image to the third comparing unit30-3described referring toFIG. 1. It is desirable that the mosaic-image producing unit20produce the third mosaic image having approximately the same resolution as that of the third template stored in the template storing unit10. The mosaic-image producing unit20may produce the third mosaic image based on the position in the given image at which the second comparing unit30-2detected the similar image-element. That is, the mosaic-image producing unit20may produce the third mosaic image showing a part of the second mosaic image based on the given image. Moreover, the mosaic-image producing unit20may produce the third mosaic image obtained by changing all areas in the given image to mosaic areas. Furthermore, the mosaic-image producing unit20may produce the third mosaic image having a higher resolution than that of the second mosaic image, that shows an approximately the same area as that of the second mosaic image in the given image.

The third comparing unit30-3detects a similar image-element that is similar to a human face element, from the third mosaic image based on the third template, described referring toFIG. 3. The third comparing unit30-3may detect a plurality of similar image-elements. In this example, the third comparing unit30-3detects areas showing eyes and a nose as the similar image-elements from the third mosaic image.FIG. 5Bshows the similar image-elements detected by the third comparing unit30-3. The third comparing unit30-3calculates the coincidence between the third mosaic image and the third template based on the thus detected similar image-elements. In this example, the third comparing unit30-3calculates the coincidences between the third mosaic image and the third template, ρ3,1=0.7 and ρ3,2=0.6.

Although the image processing apparatus100described referring toFIGS. 1-3includes three comparing units30, the image processing apparatus100in an alternative example includes four or more comparing units30. For example, the image processing apparatus100may further includes the fourth comparing unit. In this case, the mosaic-image producing unit20produces the fourth mosaic image having a higher resolution than that of the third mosaic image and supplies the fourth mosaic image to the fourth calculator.FIG. 5Cshows an example of the fourth mosaic image. The fourth comparing unit calculates the coincidence between the fourth mosaic image and the fourth template, like the third comparing unit30-3. In this example, the fourth comparing unit calculates the coincidences between the fourth mosaic image and the fourth template, ρ4,1=0.7 and ρ4,2=0.6.

Each of a plurality of comparing units has a plurality of matching means for calculating the coincidence, as described above. Each comparing unit uses any of the matching units so as to calculate the coincidence.

The coincidence calculating unit40described referring toFIG. 1calculates the coincidence between the desired image-element and the given image based on all the coincidences calculated by the first through third calculators30-1,30-2and30-3and the fourth calculator. For example, the coincidence calculating unit40calculates the coincidence PSOFbetween the desired image-element and the given image in accordance with the following equation.

As indicated in the above equation, by multiplying the coincidence ρ calculated in the lower layer by the coincidence ρ calculated in the upper layer, leak in the detection based on the coincidence ρ in a specific layer can be reduced. Moreover, by assigning appropriate values to the respective weight coefficients, it may be selected whether the coincidence in the upper layer, i.e., the layer of lower resolution or the coincidence in the lower layer, i.e., the layer of higher resolution is emphasized when the desired image is detected. The image processing apparatus100preferably has a means for changing a value of the weight coefficient based on the user's instruction. For example, the image processing apparatus100may make the value of the weight coefficient in the lower layer larger in a case where the user wants to detect a face of a specified person, and may make the value of the weight coefficient in the upper layer larger in a case where the user wants to detect a face of an unspecified person. Moreover, the image processing apparatus100may calculate the coincidence between the desired image and the given image in accordance with another equation based on all the coincidences calculated by the comparing units30.

The image-element detecting unit50described referring toFIG. 1may detect a desired image-element from a given image based on the coincidence PSOFcalculated by the coincidence calculating unit40. The image-element detecting unit50detects the similar image detected by the first comparing unit30-1as the desired image-element in a case where the coincidence PSOFexceeds a predetermined threshold value. According to the image processing apparatus100described in this example, leak or error in the detection can be reduced by calculating the coincidence PSOFbased on all the coincidences calculated by the comparing unit30.

FIG. 6shows the interrelationship between the templates stored in the template storing unit10described referring toFIG. 3. In this example, the template storing unit10stores the second templates14,24and26so as to be associated with the first template12. In this example, the template storing unit10stores the first template12showing a face of a person. Also, the template storing unit10stores the second templates14a,14b,24and26showing a right eye, a left eye, a nose and a mouth of a person, respectively.

Furthermore, the template storing unit10stores the interrelationship between the position of the right eye in the first template12, that is indicated by the second template14a, and the position of the left eye in the first template12, that is indicated by the second template14bas an interrelationship between positions in the first template that are respectively shown by a plurality of second templates. The template storing unit10stores X-coordinate displacement ΔX1and Y-coordinate displacement ΔY1from the right-eye position to the left-eye position as the interrelationship.

Similarly, the template storing unit10stores X-coordinate displacement ΔX2and Y-coordinate displacement ΔY2from the left-eye position to the nose position in the first template12as the interrelationship between the second template14band the second template24. The template storing unit10stores X-coordinate displacement ΔX3and Y-coordinate displacement ΔY3from the nose-eye position to the mouth position in the first template12as the interrelationship between the second template24and the second template26. The template storing unit10may calculate those coordinate displacements in a coordinate system in which a lower left corner of the first template12is an origin, and a right direction is an X-axis while an upward direction is a Y-axis. The image processing apparatus100described referring toFIG. 1compares the mosaic image obtained from the given image with the above-described template so as to detect the desired image-element from the given image. Moreover, although the right-eye template and the left-eye template are stored as the second templates in this example, it is apparent that the template storing unit10can store a template showing both eyes as the second template in an alternative example, as shown inFIG. 2. The template storing unit10may store the interrelationship between the templates in a similar manner.

FIGS. 7A-7Cexplain an exemplary operation for detecting an similar image-element in the similar image detecting unit30-1-ndescribed referring toFIG. 3. The similar image detecting unit30-1-ndetects the first similar image-element that is similar to the first template and the second similar image-element that is similar to the second template, from the given image.

FIG. 7Ashows a given image312that is an image to be processed in which the desired image-element is to be detected. The given image312may be an image has background other than an objet to be detected, for example.

FIG. 7Bshows the first mosaic image314produced by the mosaic-image producing unit20described referring toFIG. 1based on the given image and the first similar image-element412detected in the first mosaic image by the first similar image detecting unit30-1-adescribed referring toFIG. 3. In the present embodiment, the first similar image detecting unit30-1-adetects the first similar image-element412in the first mosaic image314, that is similar to the first template12shown inFIG. 6. In the present embodiment, the first mosaic image314shows the entire given image312. In an alternative example, the first mosaic image314may show a part of the given image312.

FIG. 7Cshows the second mosaic image316produced based on the given image312, and the second similar image-elements422,424,426and428detected in the second mosaic image by the second similar image detecting unit30-2-adescribed referring toFIG. 3. In the present embodiment, the second similar image detecting unit30-2-adetects in the second mosaic image316the second similar image-elements422,424,426and428that are similar to the second templates14a,14b,24and26, as shown inFIG. 6, respectively. In the present embodiment, the second mosaic image316shows the entire given image312. In an alternative example, the second mosaic image316may show a part of the given image312.

According to the image processing unit100in the present embodiment, the template storing unit10stores the interrelationship between the positions in the first template, respectively shown by a plurality of second templates. Therefore, the similar image-element can be detected in a reduced process time. Moreover, according to the image processing apparatus100in the present embodiment, the template storing unit10stores a plurality of first templates and the second template that is made to be associated with each first template. Therefore, it is possible to select an appropriate second template in accordance with the detection result of the first similar image-element. Thus, according to the image processing apparatus100in the present embodiment, it is possible to detect the similar image-element for various types of object images with high precision and high speed.

A plurality of comparing units30may calculate the aforementioned coincidences further based on the interrelationships between the similar image-elements detected by the corresponding similar image detecting units30-n-aand errors between the templates stored in the template storing unit10and the interrelationships. For example, the comparing unit30may calculate the coincidence further based on an error between a relative position of the template stored in the template storing unit10and a relative position of the similar image-element detected by the corresponding similar image detecting unit30-n-a. Moreover, the template storing unit10may store the position in the upper position, of the lower template. In this case, the comparing unit30may calculate the aforementioned coincidence further based on the detected similar image-element. In the following description, a case where the coincidence is calculated based on the position of the similar image-element is described.

FIG. 8shows the case of calculating the coincidence based on the position of the similar image-element. The first similar image-element detecting unit30-1-adescribed referring toFIG. 3detects the first similar image-element412that is similar to the first template, from a given image312. The first similar image-element detecting unit30-1-asupplies information regarding the position in the given image of the first similar image-element412thus detected to the second similar image-element detecting unit30-2-aand the calculator30-1-b.

The second similar image-element detecting unit30-2-adetects the similar image-elements422,424,426and428that are similar to the second template, from the given image312. The second similar image-element detecting unit30-2-amay detect the second similar image-elements422,424,426and428. In addition, the second similar image-element detecting unit30-2-amay detect the second similar image-elements422,424,426and428from an area near a reference position stored by the template storing unit10.

In this example, the second similar image-element detecting unit30-2-adetects image-elements showing a right eye, a left eye, a nose and a mouth as the second similar image-elements422,424,426and428, respectively. The second similar image-element detecting unit30-2-asupplies information regarding the positions in the given image of the second similar image-elements422,424,426and428thus detected to the calculator30-2-b.

The calculator30-n-bcalculates the aforementioned coincidence based on the information regarding the similar image-element detected by the similar image-element detecting unit30-n-a. For example, the calculator30-n-bmay determine, based on the distance the position of the similar image-element detected by the similar image-element detecting unit30-n-aand the reference position stored in the template storing unit10, whether or not the similar image-element is detected as a desired image-element. Moreover, the calculator30-n-amay calculate the coincidence further based on the relative positions in the given image of a plurality of similar image-elements detected by the similar image-element detecting unit30-n-a. According to the image processing apparatus100described above, the error in detection can be further reduced and it is also possible to detect the desired image-element with high precision.

As described above, a plurality of comparing units30each has a plurality of matching means for comparing a given image and a desired image-element. The comparing unit30calculates the coincidence between the given image and the desired image-element based on any of a plurality of matching means. The selecting unit80selects any of a plurality of matching means in each of the comparing units30based on information regarding the given image. The comparing units30calculate the coincidences between the given image and the desired image-element based on the respective selected matching means.

The selecting unit80may select any of a plurality of matching means in each comparing unit30based on the number of pixels of the given image. Moreover, the selecting unit80may select any of a plurality of matching means in each comparing unit30based on whether the given image is monochrome or color. In addition, the selecting unit80may select the matching means for each of a plurality of comparing units30independently of the selection for other comparing units30. In other words, the selecting unit80may select different matching means or the same matching means in a plurality of comparing units30.

In a case where the image processing apparatus100receives an image captured by an image-capturing apparatus as the given image, the selecting unit80may select any of a plurality of matching means based on an image-capturing condition under which that image was captured. For example, the selecting unit80may select any of a plurality of a plurality of matching means based on the date and time and the position at which that image was captured, the presence or absence of light emission by an electronic flash, optical and electronic magnification and the like.

Moreover, when the image processing apparatus100received digital image data of exif format as the given image, the selecting unit80may select any of a plurality of matching means based on APPI data of that digital image data. For example, the selecting unit80may select any of a plurality of a plurality of matching means based on the date and time and the position at which that image was captured, the presence or absence of light emission by an electronic flash, optical and electronic magnification, the number of pixels of that image, data indicating whether that image is monochrome or color, and the like, of APPI data.

FIG. 9shows an exemplary data structure of the exif-format digital image data. The exif-format digital image data includes SOI data, APPI data, DQT data, DHT data, SOF data, SOS data, compressed-data data and EOI data, as shown inFIG. 9. The SOI data shows the start of the compressed data; the APPI data shows an application marker segment; the DQT data shows a quantization table; the DHT date shows a Huffman table; the SOF date shows a flame header; the SOS data shows a scan header; the compressed-data data shows image data; and the EOI data shows the end of the compressed data.

When the image processing apparatus100received the exif-format digital image data, the selecting unit80may select any of a plurality of matching means based on the APPI data shown inFIG. 9. The APPI data shows the application marker segment indicating attribute information of the image data. The selecting unit80may select any of a plurality of a plurality of matching means based on the date and time and the position at which that image was captured, the presence or absence of light emission by an electronic flash, optical and electronic magnification, the number of pixels of that image, data indicating whether that image is monochrome or color, and the like, of APPI data.

According to the image processing apparatus100in this example, an appropriate matching means can be selected for each of a plurality of stages. Moreover, according to the image processing apparatus100in this example, an appropriate matching means can be selected for the given image. Furthermore, according to the image processing apparatus100in this example, it is possible to select the matching means for each of the templates used in comparison. Thus, the image processing apparatus100can detect the desired image-element from the given image efficiently with high precision. The image processing apparatus100described referring toFIGS. 1-9may be a computer that stores a program for conducting the aforementioned image process, for example.

FIG. 10shows an exemplary configuration of a computer200as the image processing apparatus100. The computer200includes a CPU700, a ROM702, a RAM704, a communication interface706, a hard disk drive710, an FD disk drive712and a CD-ROM drive716. The CPU700operates at least one program stored in the ROM702, the RAM704, the hard disk drive710, a floppy disk714and a CD-ROM718. The communication interface706communicates with external equipment via the Internet or the like. The hard disk drive710as an exemplary storage device stores setting information and the program in accordance with which the CPU700operates. The ROM702, the RAM704and/or the hard disk drive710stores an image processing program for making the computer200operate as the image processing apparatus100described referring toFIGS. 1-9.

The FD drive712reads data or a program from the floppy disk714and then supplies it to the CPU700. The CD-ROM drive716reads data or a program from the CD-ROM718and then supplies it to the CPU700. The communication interface706transmits/receives data while being connected to the Internet10.

Software executed by the CPU700is supplied to a user while being stored in a recording medium such as the floppy disk714and CD-ROM718. The software stored in the recording medium may be compressed or uncompressed. The software is installed from the recording medium into the hard disk drive710, and is then read into the RAM704so as to be executed by the CPU700.

The image processing program may make the computer200operate as the template storing unit10, mosaic-image producing unit20, comparing unit30, coincidence calculating unit40, image-element detecting unit50, detecting unit60, history storing unit70and selecting unit80described referring toFIGS. 1-9. In other words, in this example, the template storing unit10and the history storing unit70may be the RAM704or the hard disk drive710. Alternatively, the template storing unit10and the history storing unit70may be the FD drive into which the FD disk is inserted or the CD-ROM drive716into which the CD-ROM718is inserted. Moreover, the mosaic-image producing unit20, the comparing unit30, the coincidence calculating unit40, the image-element detecting unit50, the detecting unit60and the selecting unit80may be the CPU700. Furthermore, the computer200may further include another calculation device for performing the functions of the mosaic-image producing unit20, comparing unit30, coincidence calculating unit40, image-element detecting unit50, detecting unit60and selecting unit80.

The computer200may receive an image from which a desired image is to be detected from the CD-ROM718and/or FD drive716. The computer200may receive that image from the outside via the communication interface. The computer200may include a means for reading the image to be detected from a memory device that stores images in a digital camera.

The image processing program may be executed by being read from the recording medium directly to the RAM, or being read into the RAM after being installed into the hard disk drive. Moreover, the above program may be stored a single recording medium or a plurality of recording media. The image processing program may provide the respective functions by cooperation with an operating system. For example, the image processing program may ask the operating system to conduct a part of or all the functions and provide the function in accordance with a response from the operating system.

As the recording medium for storing the image processing program, an optical recording medium such as a DVD or PD, a magneto-optic recording medium such as an MD, a tape-like medium, a magnetic recording medium, a semiconductor memory such as an IC card or miniature card can be used other than the floppy disk and the CD-ROM. Moreover, a storage device such as a hard disk or RAM provided in a server system connected to an exclusive communication network or the Internet may be used as the recording medium. According to the computer of this example, it is possible to reduce the error in detection and to detect the desired image-element with high precision, like the image processing apparatus100.

As described above, according to the present invention, it is possible to detect a desired image-element from a given image with high precision.