Apparatus and method for retrieving image

An image retrieval method includes reading from a first database a first image group mapped to an object as a retrieval target, and extracting an image feature quantity of a plurality of images of the first image group. A second image group mapped to an object as a search key is read from a second database and an image feature quantity of a plurality of images of the second image group is extracted. Similarity is determined between the first image group and the second image group, based on the image feature quantity of the plurality of images of the first image group and the image feature quantity of the plurality of images of the second image group. On an output device, as retrieval results, the plurality of images of the first image group are displayed in order of similarity of the object as the retrieval target based on determined similarity.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2008-319977, filed on Dec. 16, 2008, the entire contents of which are incorporated herein by reference.

FIELD

Embodiments discussed herein are related to a method and apparatus for retrieving similar objects.

BACKGROUND

Techniques for retrieving similar objects are applied in a variety of fields such as industrial design retrieval in design examinations, product image retrieval in design and manufacture, 3-dimensional computer aided design (3D-CAD), and drawing search in component images. The industrial design retrieval in design examinations is described below.

In the design examination, existing designs similar to a design of a verification target are retrieved to determine design similarity. In the industrial design image retrieval in the design examination, an existing design and a text describing the feature of that existing design are stored on a database (DB) with the design tied to the text. The database is then searched with a keyword as a search key representing the feature of the design as the verification target. The existing design similar to the design as the verification target is thus retrieved.

Japanese Unexamined Patent Application Publication No. 2004-164503 discloses a three-dimensional model retrieval method as a typical image retrieval application. In accordance with the disclosure, a three-dimensional model similarity retrieval is performed with a two-dimensional image as a search key. Japanese Unexamined Patent Application Publication No. 2002-288690 discloses a cubical image display method displaying an image on a cube in the typical image processing.

SUMMARY

According to an aspect of the invention, an image retrieval method of a computer for retrieving similar objects includes reading from a first database an image group mapped to an object as a retrieval target, and extracting an image feature quantity of a plurality of images of the read image group, the first database storing the object as the retrieval target and the image group including the plurality of images resulting from depicting the object as the retrieval target from viewpoints in different directions, with the object as the retrieval target mapped to the image group.

According to the aspect, the image retrieval method further includes reading from a second database an image group mapped to an object as a search key and extracting an image feature quantity of a plurality of images of the read image group, the second database storing the object as the search key and the image group including the plurality of images resulting from depicting the object as the search key from viewpoints in different directions, with the object as the search key mapped to the image group.

According to the aspect, the image retrieval method further includes determining a similarity between the image group mapped to the object as the search key and the image group mapped to the object as the retrieval target, based on the image feature quantity of the plurality of images of the image group mapped to the object as the retrieval target and the image feature quantity of the plurality of images of the image group mapped to the object as the search key.

According to the aspect, the image retrieval method further includes displaying, on an output device, as retrieval results the plurality of images of the image group mapped to the object as the retrieval target in the order of similarity of the object as the retrieval target based on the determined similarity.

The object and advantages of the invention will be realized and achieved by means of the elements and combinations particularly pointed out in the claims.

DESCRIPTION OF EMBODIMENTS

An industrial design retrieval performed in a design examination is described below with reference to an embodiment and drawings. The embodiment is described below for exemplary purposes and the invention is not limited to the industrial design retrieval. The invention is applicable not only to the industrial design search but also to other image retrieval applications.

A typical image retrieval method in the design examination, such as a three-dimensional model retrieval using a two-dimensional image as a search key, may be considered. It is contemplated that six images (on six-pane images) may be used for each of a design as a verification target and an existing design (hereinafter simply referred to as an object). The design as a verification target is used as a search key, and the design (existing design) as a retrieval target is searched and retrieved according to the search key. Since each of the verification target design and the retrieval target design includes six images, some ingenuity may be used in image retrieval as described hereinbelow.

FIG. 1illustrates a hardware structure of a design image retrieval apparatus1according to an embodiment. The design image retrieval apparatus1includes input unit11, output unit12, drive unit13, auxiliary storage unit14, main storage unit15, processing unit16, and interface unit17, these elements mutually connected to each other via a bus B.

The input unit11, which may include a keyboard and/or a mouse, for example, is used to input a variety of signals. The output unit12, which may include a display device, for instance, displays a variety of windows and data. The interface unit17, which may include a modem and/or a local area network (LAN) card, is used to connect the design image retrieval apparatus1to a network such as the Internet.

A design image retrieval program of the embodiment is at least one of the variety of programs controlling the design image retrieval apparatus1. The design image retrieval program may be supplied in a recording medium18or may be downloaded via the network. The recording medium18storing the design image retrieval program may be a recording medium configured to optically or electronically record information, such as a compact disk read-only memory (CD-ROM), a flexible disk, or a magnetooptical disk. The recording medium18may alternately or additionally be a semiconductor memory configured to electrically record information, such as a ROM or a flash memory.

Recording medium18and the foregoing examples of structures and devices that can be used as the recording medium18are examples of a computer-readable storage medium that stores a program causing a computer to perform processes of retrieving similar objects discussed herein, such as the design image retrieval program.

If the recording medium18storing the design image retrieval program is loaded onto the drive unit13, the design image retrieval program can be installed onto the auxiliary storage unit14from the recording medium18via the drive unit13. Alternately, the design image retrieval program may be downloaded via the network and installed onto the auxiliary storage unit14via the interface unit17. Thus, a computer-readable storage medium that stores a program causing a computer to perform processes of retrieving similar objects can be remote with respect to design image retrieval apparatus and accessed via the network. The auxiliary storage unit14may be a hard disk drive (HDD).

The auxiliary storage unit14stores not only the design image retrieval program but also can store related files and/or data. At the startup of the computer, the main storage unit15reads the design image retrieval program from the auxiliary storage unit14and stores the design image retrieval program on the main storage unit15. The processing unit16performs a variety of processes to be discussed hereinbelow in accordance with the design image retrieval program stored on the main storage unit15.

The design image retrieval apparatus1may be used as a standalone apparatus. It is also contemplated that the design image retrieval apparatus1can be used in a network system illustrated inFIG. 2, such as a client-server system (CSS).FIG. 2illustrates a design image search system including the design image retrieval apparatus1.

In the design image search system ofFIG. 2, the design image retrieval apparatus1is connected to at least one user terminal2via a network3such as the Internet. InFIG. 2, the at least one user terminal is illustrated as including user terminal2aand user terminal2b, and will hereinafter be referred to as “user terminal2.” The user terminal2is operated by a user who searches for design images. The user terminal2transmits a request to the design image retrieval apparatus1and receives a response from the design image retrieval apparatus1. The design image retrieval apparatus1illustrated inFIG. 2may be identical in structure to the design image retrieval apparatus1illustrated inFIG. 1.

FIG. 3is a block diagram of the design image retrieval apparatus1according to an embodiment. The design image retrieval apparatus1includes image feature quantity extraction units21and22, similarity calculating unit23, retrieval result display unit24, database31storing a key image group, database32storing a retrieval target image group, storage unit33storing an image feature quantity of a key image, and storage unit34storing an image feature quantity of a retrieval target image. The database31storing the key image group and the database32storing the retrieval target image group may be external to the design image retrieval apparatus1.

FIG. 4is a data table managing key image groups stored on the database31. As illustrated inFIG. 4, the database31stores a key image group in six image files. The key image file includes a plurality of images responsive to an object as a search key. More specifically, the key image file includes a plurality of images resulting from depicting the object as the search key from viewpoints in multiple directions.

FIG. 5is a data table managing retrieval target image groups recorded on the database32. As illustrated inFIG. 5, the database32stores n retrieval target image groups in n×6 image files. The retrieval target image group includes a plurality of images responsive to the object as the retrieval target. The plurality of images result from depicting the object as the retrieval target from viewpoints in multiple directions.

Referring toFIG. 3, the image feature quantity extraction unit21extracts an image feature quantity such as color and shape from each image with respect to a key image group, and then records the image feature quantity on the storage unit33. Accordingly, the image feature quantity extraction unit21is an example of an image feature extracting means for reading from a database31an image group mapped to an object as a search key and for extracting an image feature quantity of a plurality of images of the image group. The image feature quantity extraction unit22extracts an image feature quantity such as color and shape from each image with reference to a retrieval target image group, and then records the image feature quantity onto the storage unit34. Accordingly, the image feature quantity extraction unit22is an example of an image feature extracting means for reading from the database32an image group mapped to an object as a retrieval target and for extracting an image feature quantity of a plurality of images of the image group.

An image feature quantity of color may be a “color histogram” representing the frequency of occurrence of color at each pixel. An image feature quantity of shape may be a “grid Fourier feature quantity”. In this case, an image is partitioned into concentric circular regions, and the ratio of black pixels within each concentric region is represented by the grid Fourier feature quantity.

The image feature quantities of color and shape are multi-dimensional information and are thus represented using vectors. In some embodiments, the image feature quantity extraction units21and22may trim a background portion of an image in response to a user's manual operation, or may activate an automatic background removal operation. An image feature quantity is thus extracted from an area other than the background.

An example of a “grid Fourier feature quantity” that is suitable for use with embodiments discussed herein is detailed in a paper entitled “A shape-based part retrieval method for mechanical assembly drawings,” contributed by T. Baba, R. Liu (FRDC), S. Endo, S. Shiitani, Y. Uehara, D. Masumoto, and S. Nagata, Technical Report of IEICE, PRMU 2004-225, pp. 79-84, March 2005, which is incorporated herein by reference.

The image feature quantity extraction units21and22may use a graph-based feature quantity for partial retrieval. An example of a graph-based feature quantity that is suitable for use with embodiments discussed herein is disclosed in a paper entitled “Similarity-based Partial Image Retrieval System for Engineering Drawings,” contributed by T. Baba, R. Liu (FRDC), S. Endo, S. Shiitani, Y. Uehara, D. Masumoto, and S. Nagata, Proc. of Seventh IEEE International Symposium on Multimedia (ISM2005), pp. 303-309, Dec. 14, 2005, which is incorporated herein by reference.

In graph representation, a line segment representing the outline of an object is referred to as a node, and a closeness between line segments is referred to as an edge. Objects constructed of line segments having a similar structure, regardless of the presence of slight noise or a slight difference in the lengths of the line segments, may be retrieved as similar objects in graph representation.

The similarity calculating unit23calculates a similarity between an object as a retrieval target and an object as a search key, based on image feature quantities of images responsive to the object as the retrieval target and the object as the search key. Accordingly, the similarity calculating unit23is an example of a similarity calculating means for calculating a similarity between an image group mapped to the object as a search key and an image group mapped to the object as the retrieval target. According to an embodiment, the image feature quantity extraction units21and22represent the images by the feature quantities in vector, and the similarity calculating unit23calculates the similarity between the images using Euclidean distance. The shorter the Euclidean distance, the higher the similarity.

Several different methods are available to determine similarity between objects. In one method, the mean value of the similarities between images is adopted as a similarity of the objects. More specifically, referring toFIG. 6, the image similarity is determined on all combinations of the images of the objects, and a correspondence relationship of the images providing the maximum sum of the similarities is determined.

FIG. 6illustrates the correspondence relationship of the images of the objects used in retrieval. Referring toFIG. 6, six images depicted in multiple (six) directions (plan view, bottom view, front view, rear view, right-side view, and left-side view) are used for one object. Corresponding images of the objects are line connected to each other inFIG. 6.

As illustrated inFIG. 6, for example, the right-side view of the object as the search key corresponds to the left-side view of the object as the retrieval target. The left-side view of the object as the search key corresponds to the right-side view of the object as the retrieval target. In this and other examples, the similarity calculating unit23calculates the mean value of the similarities between the six pairs of lined-connected images, and thus sets the mean value as the similarity of the objects.

In a second method of determining the similarity between the objects, the maximum one of the similarities between the corresponding images of the objects is set to be the similarity of the objects. Referring toFIG. 6, the similarities between the images in all combinations of the images are determined and the correspondence relationship of the images providing the maximum sum of similarities is determined. In this and other examples, the similarity calculating unit23sets to be the similarity between the objects the maximum one of the similarities between the line-connected images in the six pairs.

As illustrated inFIG. 7, the retrieval result display unit24arranges, as the retrieval results, the images of the object as the retrieval target similar to the object as the search key in the order of similarity, and displays the images of the object as the retrieval target together with the images of the object as the search key. Accordingly, the retrieval result display unit24is an example of a retrieval result displaying means for displaying, on an output device, as retrieval results the images of an image group mapped to the object as the retrieval target in the order of similarity of the object as the retrieval target based on the similarity calculated by the similarity calculating unit.

FIG. 7illustrates a screen image representing the retrieval results. The retrieval result display unit24displays a retrieval result display screen100as illustrated inFIG. 7. The retrieval result display screen100includes six images of an object103(such as an industrial design of a cell phone, for example) as a search key (illustrated at element101) depicted in six directions (plan view, bottom view, front view, rear view, right-side view, and left-side view), and similar image retrieval results102in which images (plan view, bottom view, front view, rear view, right-side view, and left-side view) of objects as retrieval targets similar to the object103as the search key are arranged in the order of similarity.

Referring toFIG. 7, the retrieval result display unit24displays the images of the objects as the retrieval targets that are similar to the object103as the search key in the order of the plan view, the bottom view, the front view, the rear view, the right-side view, and the left-side view on the retrieval result display screen100. In this case, the images of the objects as the retrieval targets that are similar to the object103as the search key are rearranged and displayed based on the correspondence relationship of the images maximizing the sum of the similarities.

FIG. 8illustrates another screen image of the retrieval results. In the case ofFIG. 6, the right-side view of the object103as the search key corresponds to the left-side view of the object as the retrieval target. The left-side view of the object as the search key corresponds to the right-side view of the object as the retrieval target. Referring toFIG. 8, the retrieval result display unit24rearranges a “left-side view of #1” and a “right-side view of #1” on the similar image retrieval results102and displays the re-arranged views on the retrieval result display screen100.

The retrieval result represented in a cubic diagram may be more intelligible to the user than the retrieval result represented in the six-pane views on the retrieval result display screen100ofFIGS. 7 and 8. For instance, as illustrated inFIG. 9, the retrieval result display unit24can display, in a cubic diagram on a retrieval result display screen110, images of an object113as a search key111and images of objects114-118as retrieval targets that are similar to the object113as the search key.

FIG. 9illustrates an example of screen image representing the retrieval results in cubic diagrams. The retrieval result display unit24displays the object113as a search key111in a cubic diagram while also displaying, in a cubic diagram on the retrieval result display screen110, similar image retrieval results112including objects114-118as the retrieval targets that are similar to the object113as the search key111. The retrieval result display unit24displays various planar images of the objects113-118as cubes in a virtual three-dimensional space in the example ofFIG. 9. The retrieval result display unit24produces cubic diagrams of the objects113-118by displaying the six images for each object113-118on the respective sides of each corresponding cube. In the cubic diagrams of the objects113-118, the object113as the search key111and the objects114-118as the retrieval targets that are similar to the object113as the search key111may be produced by different creators and thus depicted as images oriented in different directions.

The retrieval result display unit24may rearrange the images of the objects114-118as the retrieval targets that are similar to the object113as the search key111based on the correspondence relationship of the images maximizing the sum of the similarities illustrated inFIG. 6. The retrieval result display unit24then displays the rearranged images of the objects114-118on the retrieval result display screen110.

Referring toFIG. 9, the right-side view of the object113as the search key111corresponds to the left-side view of the object115as a retrieval target. The left-side view of the object113as the search key111corresponds to the right-side view of the object115as the retrieval target. The retrieval result display unit24thus rearranges the “left-side view of #2” and the “right-side view of #2” on the similar image retrieval results112in reverse order and then displays the rearranged images on the retrieval result display screen110as depicted inFIG. 9.

The object115on the similar image retrieval results112illustrated inFIG. 9is depicted in a laterally reversed position with respect to the object113as the search key111. The retrieval result display unit24thus interchanges the right-side view of the object115as the retrieval target with the left-side view of the object115as the retrieval target in accordance with the image similarity to comply with the object113as the search key111. The retrieval result display unit24then displays the rearranged images so that the images having a higher similarity are arranged on the same sides.

In some embodiments, the retrieval result display unit24can then change the orientation relationship of the cube on the retrieval result display screen110in response to a user's manual operation. The retrieval result display unit24can also display the image displayed on the front (such as the front view) in a front-down fashion on the objects114-118on the similar image retrieval results112.

When the user rotates the object113as the search key111on the retrieval result display screen110, the retrieval result display unit24allows the objects114-118to rotate on the similar image retrieval results112in synchronization with and in the same direction as the object113as the search key111. The user can freely rotate the object113as the search key111and the objects114-118on the similar image retrieval results112and verify the rotation results.

In order to display the retrieval result display screen110as illustrated inFIG. 9, the retrieval result display unit24represents by polyhedrons the object113as the search key111and the objects114-118as the retrieval targets in the order of similarity, and attaches and displays the images on the respective sides of the polyhedrons in correspondence to the object113as the search key111. If an image orientation label is attached as attribute information to each image, the retrieval result display unit24displays the image orientation labels of images in front (such as a front view) just below the respective object113as the search key111and respective objects114-118as the retrieval targets.

A process of the design image retrieval apparatus1is described below with reference to a flowchart ofFIG. 10and a process chart ofFIG. 11.FIG. 10is the flowchart illustrating the process of the design image retrieval apparatus1.FIG. 11is the process chart illustrating the process of the design image retrieval apparatus1.

The objects as the retrieval targets are two (X1,X2) in the example ofFIG. 10andFIG. 11. Each object X1, X2corresponds to six images in six orientation directions. Each object X1, X2is represented by a cube (hexahedron), and an image is attached to each side of the cube.

In step S1, the image feature quantity extraction unit22extracts the image feature quantities of all images of the object as a retrieval target, and stores the extracted image feature quantities onto the storage unit34.

In step S2, the image feature quantity extraction unit21extracts the image feature quantities of all images of the object as a search key, and stores the extracted image feature quantities onto the storage unit33.

In step S3, the similarity calculating unit23determines whether all the objects as the retrieval targets on the storage unit34have been selected. If objects as retrieval targets remain to be selected on the storage unit34, the similarity calculating unit23selects the object X1from the unselected objects as the retrieval targets on the storage unit34, and proceeds to step S4. If it is determined in step S3that all the objects as the retrieval targets have been selected on the storage unit34, processing proceeds to step S11.

In step S4, the similarity calculating unit23determines whether all the images of the object X1selected in step S3have been selected. If there remain images of the object X1to be selected, the similarity calculating unit23selects an unselected image of the object X1, and proceeds to step S5. If it is determined in step S4that all the images have been selected, processing proceeds to step S9.

In step S5, the similarity calculating unit23sets the image of the object X1selected in step S4to be Y1. Processing proceeds to step S6. The similarity calculating unit23determines whether all the images of the object as the search key have been selected. If any image of the object as the search key remains to be selected, the similarity calculating unit23selects the unselected image of the object as the search key, and processing proceeds to step S7. If all the images of the object as the search key have been selected, processing returns to step S4.

In step S7, the similarity calculating unit23sets the image of the object as the search key selected in step S6(selected image of the search key) to be Q1. The image Q1has an image orientation as a plan view. Image Q2-Q6to be described hereinbelow have image orientations as a bottom view, a front view, a rear view, a right-side view, and a left-side view, respectively. Similarly, images Y1-Y6to be described hereinbelow have image orientations as a plan view, a bottom view, a front view, a rear view, a right-side view, and a left-side view, respectively.

In step S8, the similarity calculating unit23calculates a similarity d(Y1,Q1) between the image Y1of the object X1and the selected image Q1of the search key, based on the image feature quantity of the image Y1extracted in step S1and the image feature quantity of the selected image Q1extracted in step S2as previously described.

The similarity calculating unit23repeats steps S6-S8until the similarity calculating unit23determines that all the images of the object as the search key have been selected. The similarity calculating unit23thus calculates similarities d(Y1,Q2)−d(Y1,Q6) between the image Y1of the object X1and each of the selected images Q2-Q6of the search key.

If it is determined that all the images of the object as the search key have been selected, the similarity calculating unit23returns from step S6to step S4. The similarity calculating unit23repeats steps S4-S8until the similarity calculating unit23determines that all the images of the object X1selected in step S3have been selected. The similarity calculating unit23thus calculates similarities d(Y2,Q1)−d(Y6,Q6) between each of the images Y2-Y6of the object X1and each of the selected images Q1-Q6of the search key.

Through the process performed heretofore, the similarity calculating unit23can produce a table200as illustrated inFIG. 11. If it is determined that all the images of the object X1selected in step S3have been selected, the similarity calculating unit23proceeds from step S4to step S9. Using the table200, the similarity calculating unit23determines a correspondence relationship T of the images Y1-Y6of the object X1and the selected images Q1-Q6of the search key in the descending order of similarity d(YQ).

The similarity calculating unit23determines the correspondence relationship T based on the table200. In the correspondence relationship T, the selected image Q1of the search key corresponds to the image Y1of the object X1, the selected image Q2of the search key corresponds to the image Y2of the object X1, the selected image Q3of the search key corresponds to the image Y3of the object X1, the selected image Q4of the search key corresponds to the image Y4of the object X1, the selected image Q5of the search key corresponds to the image Y5of the object X1, and the selected image Q6of the search key corresponds to the image Y6of the object X1.

In step S10, the similarity calculating unit23determines the sum of the similarities d(YQ) mapped by the correspondence relationship T, and sets the sum to be a similarity D(X1) between the object X1and the object of the search key. In the table200ofFIG. 11, the similarity D(X1) is 81.

Processing returns from step S10to step S3as depicted inFIG. 10. The similarity calculating unit23determines whether all the objects as the retrieval targets on the storage unit34have been selected. In the embodiment, there remains an object as a retrieval target on the storage unit34, and the similarity calculating unit23selects an unselected object X2as a retrieval target on the storage unit34.

The similarity calculating unit23performs steps S4-S8on the object X2as a retrieval target, thereby producing a table201as illustrated inFIG. 11. If it is determined that all the images of the object X2selected in S3have been selected, the similarity calculating unit23proceeds from step S4to step S9. Using the table201, the similarity calculating unit23determines a correspondence relationship T between each of the images Y1-Y6of the object X2and each of the selected images Y1-Y6of the retrieval keys in the descending order of similarity d(YQ).

The similarity calculating unit23determines the correspondence relationship T based on the table201. In the correspondence relationship T, the selected image Q1of the search key corresponds to the image Y1of the object X2, the selected image Q2of the search key corresponds to the image Y2of the object X2, the selected image Q3of the search key corresponds to the image Y3of the object X2, the selected image Q4of the search key corresponds to the image Y4of the object X2, the selected image Q5of the search key corresponds to the image Y6of the object X2, and the selected image Q6of the search key corresponds to the image Y5of the object X2.

In step S10, the similarity calculating unit23determines the sum of the similarities d(YQ) mapped by the correspondence relationship T, and sets the sum to be a similarity D(X2) between the object X2and the object of the search key. In the table201ofFIG. 11, the similarity D(X2) is 75.

Processing returns from step S10to step S3as depicted inFIG. 10. The similarity calculating unit23determines whether all the objects as the retrieval targets on the storage unit34have been selected. In the embodiment, all the objects as the retrieval targets on the storage unit34have been selected. The similarity calculating unit23proceeds to step S11and requests the retrieval result display unit24to display the retrieval results. The retrieval result display unit24sorts the similarities D(X1) and D(X2) in the descending order.

The retrieval result display unit24sorts the similarities D(X1) and D(X2) in the descending order, namely, in the order of similarity D(X1) and D(X2). In this example, the object X1is displayed first, and the object X2is displayed second since D(X1) is greater than D(X2).

Processing proceeds to step S12. The retrieval result display unit24displays the six images (images Y1-Y6) of each of the objects X1and X2on the respective sides of the cubes in accordance with the object X1and the search key, the object X2and the search key, and the respective correspondence relationships T. When the six images (images Y1-Y6) of each of the objects X1and X2are displayed on the respective sides of the cubes in accordance with the correspondence relationships T, the retrieval result display unit24determines which side of the cube each of the six images is to be displayed on based on the image orientation of the images Q1-Q6and the correspondence relationship T. For example, as illustrated inFIG. 11, the retrieval result display unit24displays the images with the image Y5of the object X2interchanged with the image Y6of the object X2.

When the images Y1-Y6of the object X1as the retrieval targets are displayed on the respective sides of the cube, one of the images Y1-Y6most similar to the images Q1-Q6as the search key is attached to the side in the same orientation as the corresponding one of the images Q1-Q6as the search key.

The cube corresponding to the object X1is rotated so that the similar image is attached to the same side on the cube as the corresponding one of the images Q1-Q6of the search key. The retrieval result display unit24then attaches the image second most similar to a corresponding one of the images Q1-Q6as the search key on the side of the cube in the same orientation as the corresponding one of the images Q1-Q6. If the image cannot be attached to the side in the same orientation as the corresponding one of the images Q1-Q6as the search key because of the limitation of the cube, the retrieval result display unit24skips that step without attaching the image on the corresponding side. Similarly, the retrieval result display unit24then attaches the images third to sixth most similar to the corresponding ones of the images Q1-Q6as the search key on the side of the cube in the same orientations as the corresponding ones of the images Q1-Q6.

For example, the most similar image may be the image on the front view of the cube, and a side similar to the images Q1-Q6as the search key may also be the front view of the cube. In such a case, the retrieval result display unit24attaches the first most similar image on the front of the cube.

The second most similar image may be the image on the rear of the cube, and a side similar to the images Q1-Q6as the search key may be the left-side view of the cube. In such a case, the retrieval result display unit24rotates the cube with the image attached first on the front of the cube, and determines whether the second most similar image can be attached on the left side of the cube. It is assumed here that the second most similar image cannot be attached on the left side. The retrieval result display unit24thus skips this step without attaching the second most similar image.

The retrieval result display unit24thus repeats the same process as the process performed on the first and second most similar images in order of the third, fourth, fifth, and sixth most similar images to the images Q1-Q6as the search key. If an image not yet attached onto any side of the cube remains subsequent to the attaching of the sixth most similar image, the retrieval result display unit24determines, for the image not yet attached onto any side of the cube, a side on which the image is to be attached within the range satisfying the rotation limitation of the cube.

FIG. 12illustrates a retrieval result display screen300based on a cubic diagram. In the example ofFIG. 12, the retrieval result display unit24determines one of the six images of each of the objects302-304as retrieval targets, which is most similar to an object301as a search key, on each of the objects302-304. The retrieval result display unit24then displays the objects302-304as the retrieval targets on the cubic diagrams on a retrieval result display screen300in the order of from greatest to least similarity.

FIG. 13illustrates another retrieval result display screen310. In the example ofFIG. 13, the retrieval result display unit24determines one of the six images of each of the objects312-314as retrieval targets, which is most similar to an object311as a search key, on each of the objects312-314. The retrieval result display unit24then displays the determined images having greater similarity in planar views on a retrieval result display screen310.

FIG. 14illustrates a retrieval result display screen320based on a representative view. The retrieval result display unit24displays representative views322-324of each object as a retrieval target (for example, images depicted at a slant angle to the object as the retrieval target) on a retrieval result display screen320in the order of similarity with an object321as a search key.

FIG. 15illustrates a retrieval result display screen with a switching operation performed to switch between a representative view and a cubic diagram. In the example ofFIG. 15, the retrieval result display unit24displays representative views332-334of an object as a retrieval target on a retrieval result display screen330in the order of similarity with an object331as a search key. The retrieval result display unit24can also display the representative views332-334of the object as the retrieval target in cubic diagrams342-344on a retrieval result display screen340in the order of similarity with an object341as a search key. The user selects the retrieval result display screen330or the retrieval result display screen340by specifying whichever retrieval result display screen330or340is desired.

FIG. 16illustrates a retrieval result display screen350displaying an image on a side specified on the search key. The side of the cube of an object351as a search key is specified when the retrieval result display unit24displays objects352-356as retrieval targets in the order of similarity with the object351as the search key. The images of the objects352-356corresponding to (i.e., similar to) the specified side are thus displayed on a retrieval result display screen350in the order of similarity.

When the objects352-356as the retrieval targets are displayed in the order of similarity with the object351as the search key, the user may manually modify the orientation of each of the objects352-356, as appropriate, or change the side on which the image is attached, as appropriate.

FIG. 17illustrates a retrieval result display screen360on which partially similar portions are marked in rectangular boxes. In this example, the similarity calculating unit23calculates as an image similarity the degree of partial similarity with an image of an object361as a search key. The retrieval result display unit24attaches and displays the images of objects362-366as retrieval targets on cubes. In this case, the retrieval result display unit24marks by a rectangular box a portion of each image of the objects362-366as the retrieval targets partially similar to the image of the object361as the search key.

In accordance with embodiments described herein, one object is displayed in a plurality of images, and retrieval results are displayed in the order of similarity. The design image retrieval apparatus1according to embodiments described herein displays the image as the search key with the image of the object as the retrieval target corresponding thereto. The images having a higher similarity are displayed on the same side in the same orientation.

The design image retrieval apparatus1of the embodiment also displays the image as the search key and the image of the object as the retrieval target so that the images having higher similarities are placed on the sides in the same orientation in order. The user can thus verify easily the retrieval results.

Embodiments discussed herein may relate to the use of a computer including various computer hardware or software units. Embodiments also include computer-readable storage media that store programs for causing computers to perform processes discussed herein. The computer-readable storage media can be any available or suitable media that can be accessed by a computer. For example, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, solid-state devices, or any other storage medium which can be used to carry or store desired program code in the form of computer-executable instructions or data structures and can be accessed by a computer.

Programs and computer-executable instructions include, for example, instructions and data which cause a general-purpose computer, special-purpose computer, or special purpose processing device to perform a function or group of functions. A computer, as used herein, can include one or more computers or processors that may be local, remote or distributed with respect to one another, or can include computer or processor subsystems.