Image output device, image output method, and computer program product

According to an embodiment, an image output device includes an extractor, a search unit, an associate unit, and a controller. The extractor is configured to extract a first parameter that varies in accordance with a movement of an object from at least one first image of the object, and extract a second parameter that varies in accordance with a movement of the object from each second image of the object. The search unit is configured to search for a second parameter similar to the first parameter. The associate unit is configured to associate the first image from which the first parameter is extracted with the second image from which the second parameter that is retrieved with respect to the first parameter is extracted. The controller is configured to instruct an output unit to output an image based on the first and second images that are associated to each other.

FIELD

Embodiments described herein relate generally to an image output device, an image output method, and a computer program product.

BACKGROUND

A technology is known in which a plurality of movie sequence is displayed next to each other so as to enable visual identification of whether or not it is the same person who is captured in the movie sequence. Moreover, in order to facilitate identification of a particular person who is to be identified, a technology is known in which that particular person is tracked in the movie sequence; a plurality of images is extracted that captures different orientations of the face of that particular person; and then the extracted images are displayed next to each other.

Regarding a mobile object, information that is beneficial in the identification of that mobile object, such as the information on the mannerisms of that mobile object, becomes easily available in the movements made by that mobile object. However, in the conventional technology described above, while visually identifying whether or not it is the same mobile object that is captured in a plurality of images, no thought is given to the movements made by that mobile object.

DETAILED DESCRIPTION

According to an embodiment, an image output device includes an acquirer, an extractor, a search unit, an associate unit, and an output controller. The acquirer is configured to acquire a plurality of first images obtained by capturing a mobile object in a first chronological order and a plurality of second images obtained by capturing the mobile object. The extractor is configured to extract a first parameter that varies in accordance with a movement of the mobile object from at least one of the first images, and extract a second parameter that varies in accordance with a movement of the mobile object from each of the second images. The search unit is configured to search the second parameters for a second parameter that is similar to the first parameter. The associate unit is configured to associate the first image from which the first parameter is extracted with the second image from which the second parameter that is retrieved with respect to the first parameter is extracted. The output controller is configured to instruct an output unit to output an image based on the first image and the second image that are associated to each other.

An exemplary embodiment of the invention is described below in detail with reference to the accompanying drawings.

FIG. 1is a configuration diagram illustrating an example of an image output device10according to the embodiment. As illustrated inFIG. 1, the image output device10includes an acquirer13, an extractor15, a search unit17, an associate unit19, an output controller21, and an output unit23.

The acquirer13, the extractor15, the search unit17, the associate unit19, and the output controller21can be implemented by executing computer programs in a processor such as a central processing unit (CPU), that is, can be implemented using software; or can be implemented using hardware such as an integrated circuit (IC); or can be implemented using a combination of software and hardware. The output unit23can be implemented using, for example, a display device such as a liquid crystal display. Alternatively, the output unit23can be implemented using a communication device such as a network interface card (NIC).

The acquirer13acquires a plurality of first images, which capture a mobile object in a first chronological order, and acquires a plurality of second images, which capture the same mobile object. In the embodiment, it is assumed that the first images and the second images are taken using monitoring cameras or security cameras that are installed throughout the town. However, that is not the only possible case. Meanwhile, the first images and the second images can either be movie sequence or be still images.

Moreover, in the embodiment, it is assumed that the first images and the second images are stored in a storage unit (not illustrated) of the image output device10, and the acquirer13acquires the images from the storage unit. However, that is not the only possible case. Alternatively, the acquirer13can acquire the first images and the second images directly from the monitoring cameras via a network. Still alternatively, an external storage unit for storing the first images and the second images can be installed on the outside of the image output device10(for example, on the cloud), and the acquirer13can acquire the images from that external storage unit. Herein, the external storage unit can be implemented using, for example, a hard disk drive (HDD) or a solid state drive (SSD).

Furthermore, in the embodiment, it is assumed that the second images capture a mobile object in a second chronological order. However, that is not the only possible case. Alternatively, the second images may not capture the mobile object in a chronological order. Herein, it is desirable that the first chronological order is different than the second chronological order. As long as the first chronological order is different than the second chronological order, the monitoring cameras capturing the first images and the monitoring cameras capturing the second images may be the same monitoring cameras or different monitoring cameras. Meanwhile, in the embodiment, although the mobile object is assumed to be a person, it is not the only possible case.

The extractor15extracts, from at least one of the first images acquired by the acquirer13, a first parameter that varies in accordance with the movements made by a person. Moreover, the extractor15extracts, from each second image acquired by the acquirer13, a second parameter that varies in accordance with the movements made by a person. In the embodiment, it is assumed that the extractor15acquires the first parameter from each first image acquired by the acquirer13.

In the embodiment, the explanation is given for an example in which the first parameters and the second parameters represent the sizes of the face of the person captured in the images. In this case, the extractor15acquires a face area (in the embodiment, assumed to be a rectangular area indicating the face) from each first image; and sets the horizontal width and the height of the face area as the first parameter. In an identical manner, the extractor15acquires a face area from each second image; and sets the horizontal width and the height of the face area as the second parameter. Herein, as long as a first parameter as well as a second parameter includes at least the horizontal width and the height of the face area, it serves the purpose. Besides, the first parameters and the second parameters can also include some other information (such as an identifier of the source image for extraction).

FIG. 2is a diagram illustrating an example of face area extraction performed according to the embodiment. In the example illustrated inFIG. 2, from an image31, the extractor15extracts an area32as the face area. Herein, the face area extraction can be performed by implementing, for example, the method disclosed in Takeshi Mita, Toshimitsu Kaneko, Bjorn Stenger, Osamu Hori: “Discriminative Feature Co-Occurrence Selection for Object Detection”. IEEE Transaction Pattern Analysis and Machine Intelligence Volume 30, Number 7, July 2008, pp. 1257-1269 (hereinafter, referred to as “reference literature 1”). Meanwhile, extraction of a body area can be performed by implementing, for example, the method disclosed in Tomoki Watanabe, Satoshi Ito, Kentaro Yoko: “Co-occurrence Histogram of Oriented Gradients for Human Detection”, IPSJ Transaction on Computer Vision and Applications Volume 2 Mar. 2010, pp. 39-47 (hereinafter, referred to as “reference literature 2”).

FIG. 3is a diagram illustrating an example of first parameters according to the embodiment.FIG. 4is a diagram illustrating an example of second parameters according to the embodiment. In the examples illustrated inFIGS. 3 and 4, each parameter represents information in which an image number, which represents the identifier of a source image for extraction of the face area, is associated to the face area extraction result (the horizontal width and the height of the face area). Herein, “w” represents the horizontal width of the face area, and “h” represents the height of the face area. Moreover, “N” as well as “M” is a positive integer equal to or greater than two.

The search unit17searches among a plurality of second parameters, which are extracted by the extractor15, for the second parameter that is similar to the first parameter extracted by the extractor15. In the embodiment, for each first parameter extracted by the extractor15, the search unit17searches among a plurality of second parameters for the second parameter that is similar to the first parameter.

More particularly, the search unit17searches among a plurality of second parameters for the second parameter for which the distance, which represents the degree of similarity, to the first parameter is the shortest. For example, for each first parameter, the search unit17calculates the distance, which represents the degree of similarity, to each of a plurality of second parameters using Equation (1) given below and searches for the second parameter for which the calculated distance is the shortest.
D=(wAx−wBz)2+(hAx−hBz)2(1)

Herein, wAx represents the horizontal width of the face area of the first parameter, hAx represents the height of the face area of the first parameter, wBz represents the horizontal width of the face area of the second parameter, and hBz represents the height of the face area of the second parameter. Moreover, “x” is an arbitrary positive integer between 1 to N, and “z” is an arbitrary positive integer between 1 to M.

The associate unit19associates the source first image for extraction of a first parameter with the source second image for extraction of the second parameter that is retrieved with respect to that first parameter by the search unit17. In the embodiment, for each first parameter, the associate unit19associates the source first image for extraction of the first parameter with the source second image for extraction of the second parameter that is retrieved with respect to the first parameter by the search unit17.

For example, a result of the search performed by the search unit17is illustrated as a graph inFIG. 5. The graph represents the second parameters that are retrieved with respect to the first parameters. That is, the graph represents the correspondence between the source first images for extraction of the first parameters and such second images that capture face areas having the closest size to the size of the face areas captured in the first images.

In this case, as illustrated inFIG. 6, the associate unit19associates the first image having a first image number A1with the second image having a second image number B11; associates the first image having a first image number A2with the second image having a second image number B12; and associates the first image having a first image number A3with the second image having a second image number B13.

The output controller21instructs the output unit23to output images based on the first images and the second images associated to each other by the associate unit19. In the embodiment, the output controller21instructs the output unit23to sequentially output the images that are based on the sets of a first image and a second image associated to each other according to the first-chronological order or the second chronological order.

Herein, regarding images based on the sets of first images and second images associated to each other, an example includes images in which first images and second images are arranged next to each other. As far as the arrangement is concerned, for example, a first image and a second image can be arranged side by side or can be arranged one above the other.

FIG. 7is a diagram illustrating an output example according to the embodiment. In the example illustrated inFIG. 7, the output controller21instructs the output unit23to sequentially output, on a screen, an image in which the first image having the first image number A1and the second image having the second image number B11that are associated to each other by the associate unit19are arranged side by side; an image in which the first image having the first image number A2and the second image having the second image number B12that are associated to each other by the associate unit19are arranged side by side; and an image in which the first image having the first image number A3and the second image having the second image number B13that are associated to each other by the associate unit19are arranged side by side, in that sequence (in the first chronological order).

As a result, it becomes possible to output images that include the movements made by a person captured in a plurality of first images and include the movements made by a person captured in a plurality of second images. Hence, it becomes possible to facilitate visual identification of whether or not the person captured in a plurality of first images is the same person who is captured in a plurality of second images.

Meanwhile, in the case when a plurality of output units23is arranged or when the output unit23performs the output on a plurality of screens (for example, performs the output on a plurality of windows); the output controller21can control the output in such a way that the sets of a first image and a second image that are associated to each other need not be arranged together, and the timing at which the first images and the second images that are associated to each other are output to the output unit23is matched. In this case, an image based on the set of a first image and a second image represents nothing but the first image and the second image.

FIG. 8is a flowchart for explaining an example of the operations performed in the image output device10according to the embodiment.

Firstly, the acquirer13acquires a plurality of first images, which capture a mobile object in a first chronological order, and acquires a plurality of second images, which capture the same mobile object (Step S101).

Then, the extractor15extracts, from each first image acquired by the acquirer13, a first parameter that varies in accordance with the movements made by the person; and extracts, from each second image acquired by the acquirer13, a second parameter that varies in accordance with the movements made by the person (Step S103).

Subsequently, for each first parameter extracted by the extractor15, the search unit17searches among the second parameters, which are extracted by the extractor15, for the second parameter that is similar to the first parameter. Then, for each first parameter, the associate unit19associates the source first image for extraction of that first parameter with the source second image for extraction of the second parameter that is retrieved with respect to the first parameter (Step S105).

Then, the output controller21instructs the output unit23to sequentially output images, each of which is based on a set of the first image and the second image that are associated to each other by the associate unit19, in the first chronological order (Step S107).

In this way, according to the embodiment, it becomes possible to output images that include the movements made by a person captured in a plurality of first images and include the movements made by a person captured in a plurality of second images. Hence, it becomes possible to facilitate visual identification of whether or not the person captured in a plurality of first images is the same person who is captured in a plurality of second images.

First Modification Example

In the embodiment, the explanation is given for an example in which the sizes of the face of a person captured in the images are considered to be the first parameters and the second parameters. In contrast, in a first modification example, the explanation is given for an example in which the orientations of the face of a person captured in the images are considered to be the first parameters and the second parameters.

In this case, the extractor15extracts the face area from each of a plurality of first images, and sets the orientation of the face in each face area as the first parameter. In an identical manner, the extractor15extracts the face area from each of a plurality of second images, and sets the orientation of the face in each face area as the second parameter.

FIG. 9is a diagram illustrating an example of face orientation extraction performed according to the first modification example. In the example illustrated inFIG. 9, the extractor15extracts the face orientation as a continuous quantity of roll/pitch/yaw. Herein, the roll represents the angle of in-screen turning with the frontward orientation as the reference; the pitch represents the angle of vertical turning with the frontward orientation as the reference; and the yaw represents the angle of sideways turning with the frontward orientation as the reference. However, the face orientation is not limited to this example. Alternatively, the face orientation can include the frontward orientation, the rightward orientation, the leftward orientation, the upward orientation, or the downward orientation.

In the example illustrated inFIG. 9, a face101indicates the face orientation in the case when the roll value is a negative value; a face102indicates the face orientation in the case when the roll value is equal to zero; and a face103indicates the face orientation in the case when the roll value is a positive value. Similarly, a face104indicates the face orientation in the case when the pitch value is a negative value; a face105indicates the face orientation in the case when the pitch value is equal to zero; and a face106indicates the face orientation in the case when the pitch value is a positive value. Moreover, a face107indicates the face orientation in the case when the yaw value is a negative value; a face108indicates the face orientation in the case when the yaw value is equal to zero; and a face109indicates the face orientation in the case when the yaw value is a positive value. Meanwhile, the orientation extraction can be performed by implementing, for example, the method disclosed in reference literature 1 or reference literature 2 mentioned above. If the faces are oriented in the same direction, then it becomes easier to identify whether the faces are of the same person by checking whether the faces match with each other or whether the appearance of a particular part of each face is identical. For example, regarding noses that do not necessarily match from the structural perspective; instead of comparing the noses from different directions, comparison of the noses from the same direction makes it easier to perform identification. Moreover, even in the case when a hidden area is present, matching the directions leads to uniformity in appearance thereby making it possible to effectively use the visible area. Meanwhile, as long as the faces can be oriented in the direction of one of the images, it serves the purpose. In the case when there is no hidden area (invisible area) or when the faces can be oriented in any direction, it is desirable that the faces are oriented close to the frontward orientation so that the face orientations match with each other and the roll/pitch/yaw values are close to zero. That makes it easier to perform identification.

FIG. 10is a diagram illustrating an example of first parameters according to the first modification example. In the example illustrated inFIG. 10, each parameter represents information in which an image number, which represents the identifier of a source image for extraction of the face area, is associated to the extraction result (roll/pitch/yaw) of the face orientation of that face area. Herein, “r” represents the roll, “p” represents the pitch, and “y” represents the yaw.

Herein, the search unit17and the associate unit19can perform operations in an identical manner to the operations explained earlier in the embodiment.

FIG. 11is a diagram illustrating an output example according to the first modification example. In the example illustrated inFIG. 11, the output controller21instructs the output unit23to sequentially output, on a screen, an image in which the first image having a first image number C1and the second image having a second image number D11that are associated to each other by the associate unit19are arranged side by side; an image in which the first image having a first image number C2and the second image having a second image number D12that are associated to each other by the associate unit19are arranged side by side; and an image in which the first image having a first image number C3and the second image having a second image number D13that are associated to each other by the associate unit19are arranged side by side, in that sequence (in the first chronological order).

As a result, it becomes possible to output images that include the face movements made by a person captured in a plurality of first images and include the face movements made by a person captured in a plurality of second images. Hence, it becomes possible to facilitate visual identification of whether or not the person captured in a plurality of first images is the same person who is captured in a plurality of second images.

Second Modification Example

In the embodiment, the explanation is given for an example in which the sizes of the face of a person captured in the images is considered to be the first parameters and the second parameters. In contrast, in a first modification example, the explanation is given for an example in which the expressions of a person captured in the images is considered to be the first parameters and the second parameters.

In this case, the extractor15extracts a face area from each of a plurality of first areas and sets the degree of smile of that face area as the first parameter. In an identical manner, the extractor15extracts a face area from each of a plurality of second areas and sets the degree of smile of that face area as the first parameter. Herein, the degree of smile represents the numerical conversion of the facial expression including expressionlessness or a smile. The degree of smile can be extracted by implementing a known method.

FIG. 12is a diagram illustrating an example of first parameters according to the second modification example. In the example illustrated inFIG. 12, each parameter represents information in which an image number, which represents the identifier of a source image for extraction of the face area, is associated to the extraction result of the degree of smile of that face area. Herein, “s” represents the degree of smile.

Herein, the search unit17and the associate unit19can perform operations in an identical manner to the operations explained earlier in the embodiment.

FIG. 13is a diagram illustrating an output example according to the second modification example. In the example illustrated inFIG. 13, the output controller21instructs the output unit23to sequentially output, on a screen, an image in which the first image having a first image number E1and the second image having a second image number F11that are associated to each other by the associate unit19are arranged side by side; an image in which the first image having a first image number E2and the second image having a second image number F12that are associated to each other by the associate unit19are arranged side by side; and an image in which the first image having a first image number E3and the second image having a second image number F13that are associated to each other by the associate unit19are arranged side by side, in that sequence (in the first chronological order).

As a result, it becomes possible to output images that include the changes in the facial expression of a person captured in a plurality of first images and include the changes in the facial expression of a person captured in a plurality of second images. Hence, it becomes possible to facilitate visual identification of whether or not the person captured in a plurality of first images is the same person who is captured in a plurality of second images.

Third Modification Example

In the embodiment, the explanation is given for an example in which the sizes of the face of a person captured in the images is considered to be the first parameters and the second parameters. In contrast, in a third modification example, the explanation is given for an example in which the relative positions of the feet of a person captured in the images are considered to be the first parameters and the second parameters.

In this case, the extractor15extracts a feet area from each of a plurality of first areas and sets the relative position of the feet area with respect to the ground as the first parameter. In an identical manner, the extractor15extracts a feet area from each of a plurality of second areas and sets the relative position of the feet area with respect to the ground as the second parameter. Meanwhile, the relative position of the feet area with respect to the ground can be extracted by implementing, for example, the method disclosed in reference literature 1 or reference literature 2 mentioned above.

FIG. 14is a diagram illustrating an example of first parameters according to the third modification example. In the example illustrated inFIG. 14, each parameter represents information in which an image number, which represents the identifier of a source image for extraction of the feet area, is associated to the extraction result of the relative position of the feet area with respect to the ground. Herein, “d” represents the relative position of the feet area with respect to the ground.

Meanwhile, the search unit17and the associate unit19can perform operations in an identical manner to the operations explained earlier in the embodiment.

FIG. 15is a diagram illustrating an output example according to the third modification example. In the example illustrated inFIG. 15, the output controller21instructs the output unit23to sequentially output, on a screen, an image in which the first image having a first image number G1and the second image having a second image number H11that are associated to each other by the associate unit19are arranged side by side; an image in which the first image having a first image number G2and the second image having a second image number H12that are associated to each other by the associate unit19are arranged side by side; and an image in which the first image having a first image number G3and the second image having a second image number H13that are associated to each other by the associate unit19are arranged side by side, in that sequence (in the first chronological order).

As a result, it becomes possible to output images that include the walking of a person captured in a plurality of first images and include the walking of a person captured in a plurality of second images. Hence, it becomes possible to facilitate visual identification of whether or not the person captured in a plurality of first images is the same person who is captured in a plurality of second images.

Meanwhile, in the third modification example, the explanation is given for a case in which the relative positions of the feet of a person captured in the images are considered to be the first parameters and the second parameters. Alternatively, even in the case when the relative positions of the hands of a person captured in the images are considered to be the first parameters and the second parameters, images including the walking of the person can be output in an identical manner.

Fourth Modification Example

In the description according to the embodiment and the description according to the first to third modification examples, a second parameter for which the distance, which represents the degree of similarity, to a first parameter is the shortest is considered to be the second parameter similar to that first parameter. However, alternatively, a second parameter for which the distance, which represents the degree of similarity, to a first parameter is not only the shortest but is also is equal to or smaller than a threshold value is considered to be the second parameter similar to that first parameter.

Accordingly, the search unit17can be configured to search among a plurality of second parameters for the second parameter for which the distance, which represents the degree of similarity, to a first parameter is not only the shortest but is also is equal to or smaller than a threshold value.

However, in this case, if there is no second parameter for which the distance, which represents the degree of similarity, to the first parameter is equal to or smaller than the threshold value; then the search unit17cannot retrieve the second parameter.

For that reason, the associate unit19generates a supplemental image with the use of the source second images for extraction of such second parameters which are retrieved with respect to first parameters that are extracted from other first images which, in the first chronological order, are present previous and subsequent to the source first image for extraction of a first parameter with respect to which no second parameter was retrieved that has the distance, which represents the degree of similarity, to the first parameter equal to or smaller than the threshold value. Then, the associate unit19associates the first image with the supplemental image.

FIG. 16is a diagram illustrating an example of the result of associating the first images and the second images according to the fourth modification example. In the example illustrated inFIG. 16, it is assumed that, with respect to the first parameters extracted from the first images having first image numbers G1, G2, G3, and G5; the search unit17retrieves the second parameters extracted from the second images having second image numbers H11, H12, H13, and H15, respectively. However, it is assumed that, with respect to the first parameter extracted from the first image having a first image number G4, the search unit17does not retrieve any second parameter.

In this case, the associate unit19associates the first images having the first image numbers G1, G2, G3, and G5with the second images having the second image numbers H11, H12, H13, and H15, respectively. Moreover, the associate unit19makes use of the second image having the second image number H13that is the source second image for extraction of the second parameter retrieved with respect to the first parameter that was extracted from the first image which has the first image number G3and which is previous or subsequent in the first chronological order to the first image having the first image number G4; makes use of the second image having the second image number H15that is the source second image for extraction of the second parameter retrieved with respect to the first parameter that was extracted from the first image which has the first image number G5and which is previous or subsequent in the first chronological order to the first image having the first image number G4; and generates a supplemental image having a second image number H14. Then, the associate unit19associates the first image having the first image number G4with the supplemental image having the second image number H14.

Subsequently, the output controller21instructs the output unit23to sequentially output, in the first chronological order or in the second chronological order, images based on the sets of a first image and a second image that are associated to each other and images based on the sets of the first images and the supplemental images that are associated to each other.

Fifth Modification Example

In the description according to the embodiment and the description according to the first to fourth modification examples, each of a plurality of first images is associated with a second image that includes a similar parameter. However, alternatively, from among a plurality of first images, only a predetermined first image can be associated with a second image that includes a similar parameter. Then, each other first image other than the predetermined first image can be associated to a second image that is shifted from the abovementioned associated second image by the number of frames equal to the number of frames between the other first image and the predetermined first image.

In this case, from among a plurality of first images, the extractor15can extract the first parameter from the predetermined first image.

The associate unit19can associate the predetermined first image, from which the first parameter was extracted, with the source second image for extraction of the second parameter retrieved with respect to the abovementioned first parameter. Moreover, the associate unit19can associate each other first image other than the predetermined first image with a second image that is shifted from the abovementioned associated second image by the number of frames equal to the number of frames between the other first image and the predetermined first image.

FIG. 17is a diagram illustrating an example of the result of association between the first images and the second images according to the fifth modification example. In the example illustrated inFIG. 17, the first image having the first image number G1, which is the predetermined first image from which the first parameter was extracted, is associated with the second image having a second image number I11, which is the source second image for extraction of the second parameter retrieved with respect to the first parameter. Moreover, to the first image having the first image number G2is associated the second image having a second image number I12that is shifted from the second image having the second image number I11by the number of frames equal to the number of frames (a single frame) between the first image having the first image number G1and the first image having the first image number G2. In an identical manner, to the first image having the first image number G3is associated the second image having a second image number I13; to the first image having the first image number G4is associated the second image having a second image number I14; and to the first image having the first image number G5is associated the second image having a second image number I15.

Then, the output controller21instructs the output unit23to sequentially output, in the first-chronological order or in the second chronological order, the images based on the sets of a first image and a second image that are associated to each other.

As a result, it becomes possible to output images that include a particular point of time during the movements (for example, the start of a movement or the end of a movement) captured in a plurality of first images and includes a particular point of time during the movements captured in a plurality of second images. Hence, it becomes possible to facilitate visual identification of whether or not the person captured in a plurality of first images is the same person who is captured in a plurality of second images.

Hardware Configuration

FIG. 18is a diagram illustrating an exemplary hardware configuration of the image output device10according to the embodiment and the modification examples described above. As illustrated inFIG. 18, the image output device10according to the embodiment and the modification examples described above has the hardware configuration of a general-purpose computer that includes a central processing unit (CPU)902; a random access memory (RAM)906; a read only memory (ROM)904that is used to store computer programs; a hard disk drive (HDD)908; an I/F910that serves as the interface with the HDD908; an I/F912that serves as the interface for the purpose of image input; an input device914such as a mouse or a keyboard; an I/F916that serves as the interface with the input device914; a display device918such as a display; an I/F920that serves as the interface with the display device918; and a bus922. Herein, the CPU902, the ROM904, the RAM906, the I/F910, the I/F912, the I/F916, and the I/F920are connected to each other via the bus922.

In the image output device10according to the embodiment and the modification examples described above, the CPU902reads computer programs from the ROM904, and runs the computer programs by loading them in the RAM906. As a result, the abovementioned constituent elements are implemented in the computer. Then, in the image output device10according to the embodiment and the modification examples described above, the CPU902makes use of the information stored in the HDD908and associates the first images and the second images that are input via the I/F912.

Meanwhile, the computer programs can also be stored in the HDD908. Alternatively, the computer programs can be recorded in the form of installable or executable files in a computer-readable recording medium such as a compact disk read only memory (CD-ROM), a compact disk readable (CD-R), a memory card, a digital versatile disk (DVD), or a flexible disk (FD); and can be provided as a computer program product. Still alternatively, the computer programs can be saved as downloadable files on a computer connected to the Internet or can be made available for distribution through a network such as the Internet.

For example, unless contrary to the nature thereof, the steps of the flowcharts according to the embodiments described above can have a different execution sequence, can be executed in plurality at the same time, or can be executed in a different sequence every time.

As described above, according to the embodiment and the modification examples described above, it becomes possible to facilitate visual identification of whether or not it is the same mobile object that is captured in a plurality of images.