Information processing device, information processing method and computer readable medium

An information processing device that is connected to a projecting device that projects an annotation image input from an external terminal onto a projection region including an object, and is connected to an image capturing device that captures an image of the projection region. The information processing device obtains image data to be input to and output from the projecting device and the image capturing device through a communication with the external terminal. The information processing device includes: a memory that stores a database including CAD data to be used for designing the object, and the attribute information of the CAD data; an acquiring unit that acquires the identification information of the object; an identifying unit that identifies the CAD data of the object included in the database, based on the acquired identification information of the object; and a controller that associates the data of an image captured by the image capturing device with the CAD data of the object identified by the identifying unit, and causes the memory to store the data of the captured image associated with the CAD data, so that the data of the captured image can be viewed based on the attribute information of the CAD data of the object identified by the identifying unit.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2008-008565 filed Jan. 17, 2008.

BACKGROUND

1. Technical Field

The present invention relates to an information processing device, an information processing method and a computer readable medium.

2. Related Art

There have been known remote indication systems each including a server (a computer, for example) connected to a camera and a projector, and a remote client (a computer, for example) connected to the server via a network.

SUMMARY

According to an aspect of the invention, there is provided an information processing device that is connected to a projecting device that projects an annotation image input from an external terminal onto a projection region including an object, and is connected to an image capturing device that captures an image of the projection region. The information processing device obtains image data to be input to and output from the projecting device and the image capturing device through a communication with the external terminal. This information processing device includes: a memory that stores a database including CAD data to be used for designing the object, and the attribute information of the CAD data; an acquiring unit that acquires the identification information of the object; an identifying unit that identifies the CAD data of the object included in the database, based on the acquired identification information of the object; and a controller that associates the data of an image captured by the image capturing device with the CAD data of the object identified by the identifying unit, and causes the memory to store the data of the captured image associated with the CAD data, so that the data of the captured image can be viewed based on the attribute information of the CAD data of the object identified by the identifying unit.

DETAILED DESCRIPTION

The following is a description of exemplary embodiments of the present invention, with reference to the accompanying drawings.

First Exemplary Embodiment

FIG. 1is a block diagram illustrating the structure of a remote indication system in accordance with an exemplary embodiment of the present invention.

The remote indication system ofFIG. 1includes a personal computer (PC)1that functions as a server, and a PC2(an external terminal) that functions as a client. The PC1and the PC2are connected to each other via a network3. A projector4(a projecting device) and a camera5(an image capturing device) are connected to the PC1.

In accordance with a control command from the PC1, the projector4emits light beams or projects an annotation image or the like onto an object8and a screen12via a half mirror6. Annotation images may include various forms of images such as lines, characters, symbols, figures, colors, and fonts.

The camera5is formed with a PTZ (Pan-Tilt-Zoom) camera, a network camera, or a video camera, and is placed on a table. The camera5captures a reflected image of the screen12as well as the object8via the half mirror6, and outputs the captured image to the PC1. The camera5captures an entire image of the object8. The optic axis and field angle of the projector4are the same as the optic axis and field angle of the camera5. The camera5captures a still image of the object8in accordance with an instruction that is input from the PC2or an instruction that is input directly to the PC1, or on a regular basis (at 0.1 seconds intervals, for example). A QR code7including the ID (identification information) of the object8is attached to the surface of the object8.

The PC1outputs the image captured by the camera5to the PC2via the network3. The PC2is connected to a display205, and the display205has a display region15and a user interface (UI)14. The captured image transmitted from the PC1is displayed in the display region15. The PC2may be a personal computer that is integrally formed with the display205.

A microphone10(the (second) voice input/output device) receives a voice instruction or the like of the user of the PC1, and outputs the voice to the speaker19connected to the PC2. A speaker11(the (second) voice input/output device) outputs a voice instruction or the like of the user of the PC2that is input through a microphone18connected to the PC2.

The UI14includes a group of buttons such as a pen button, a text button, and an erase button, and icons defined by lines and colors. The image captured by the camera5is displayed in the display region15. InFIG. 1, the image of the screen12including the object8that is captured by the camera5is displayed in the display region15.

For example, when the pen button of the UI14is pressed to draw a figure or the like on the object8in the display region15, the information about the figure (more specifically, coordinates information) is output from the PC2to the PC1. The PC1then decodes and converts the figure information (the coordinate information) to an image to be projected by the projector4and outputs the image to the projector4. The projector4then projects the image onto the object8.

Meanwhile, the PC2outputs control commands to the PC1, so as to control operations of the projector4and the camera5(such as the image capturing angles and the brightness of images captured by the camera5, and the brightness of images projected by the projector4).

The microphone18(the first voice input/output device) receives a voice instruction or the like of the user of the PC2, and outputs the voice to the speaker11connected to the PC1. The speaker19(the first voice input/output device) outputs a voice instruction or the like of the user of the PC1that is input through the microphone10connected to the PC1.

InFIG. 1, the number of clients is only one (the PC2), but the remote indication system may include two or more clients (PCs).

In the remote indication system having the above structure, an image captured by the camera5is displayed on the display205of the PC2, and an annotation image written on the PC2is projected onto the object8through the projector4. Accordingly, the user of the PC1and the user of the PC2can communicate with each other with the use of the object8that is the real thing. The user of the PC1and the user of the PC2can also have voice communications with each other through the microphone10, the speaker11, the microphone18, and the speaker19.

FIG. 2is a block diagram showing the functional structures of the PCs1and2.

The PC1includes: a controller101(the acquiring unit, the identifying unit, and the controller) that controls operations of the projector4and the camera5, as well as the entire device; a transmitting and receiving unit102that exchanges information with the PC2via the network3; a memory103(the memory) that stores control programs, data, information, and the likes; and an interface unit104that is connected to the projector4, the camera5, the microphone10, and the speaker11. The PC1may further include a display105that displays a captured image, and an operating unit106(the input unit) that is formed with a mouse, a keyboard, and the likes. The controller101is connected to the transmitting and receiving unit102, the memory103, the interface unit104, the display105, and the operating unit106. The controller101is further connected to the projector4, the camera5, the microphone10, and the speaker11via the interface unit104.

The PC2includes: a controller201(the acquiring unit, the identifying unit, and the controller) that controls the entire device; a transmitting and receiving unit202that exchanges data and information with the PC1via the network3; a memory203(the memory) that stores control programs, data, information, and the likes; an operating unit204(the input unit) that is formed with a mouse, a keyboard, and the likes; a display205; and an interface unit206. The controller201is connected to the transmitting and receiving unit202, the memory203, the operating unit204, the display205, and the interface unit206. The controller201is further connected to the microphone18and the speaker19via the interface unit206.

FIG. 3Ais a block diagram showing the hardware structure of the PC1.FIG. 3Bis a block diagram showing the hardware structure of the PC2.

The PC1includes: a CPU21that controls the entire device; a ROM22that stores control programs; a RAM23that functions as a working area; a hard disk drive (HDD)24that stores various kinds of information and programs; a network interface26that connects to another computer; and a USB (universal serial bus) interface28that connects to a USB device (not shown). The CPU21is connected to the ROM22, the RAM23, the hard disk drive (HDD)24, the network interface26, and the USB interface28via a system bus29. The PC1is connected to the projector4, the camera5, the microphone10, and the speaker11via the USB interface28. The PC1may further include a mouse and keyboard25and a display27.

The controller101is equivalent to the CPU21that performs various kinds of operations in accordance with control programs. The transmitting and receiving unit102is equivalent to the network interface26, and the memory103is equivalent to the hard disk drive (HDD)24. The interface unit104is equivalent to the USB interface28. The display105is equivalent to the display27, and the operating unit106is equivalent to the mouse and keyboard25.

The PC2includes: a CPU31that controls the entire device; a ROM32that stores control programs; a RAM33that functions as a working area; a hard disk drive (HDD)34that stores various kinds of information and programs; a mouse and keyboard35; a network interface36that connects to another computer; a display37formed with a liquid crystal monitor or a CRT; and a USB (universal serial bus) interface38that connects to a USB device (not shown). The CPU31is connected to the ROM32, the RAM33, the hard disk drive (HDD)34, the mouse and keyboard35, the network interface36, the display37, and the USB interface38via a system bus39. The PC2is connected to the microphone18and the speaker19via the USB interface38.

The controller201is equivalent to the CPU31that performs various kinds of operations in accordance with control programs. The transmitting and receiving unit202is equivalent to the network interface36, and the memory203is equivalent to the hard disk drive (HDD)34. The operating unit204is equivalent to the mouse and keyboard35, and the display205is equivalent to the display37.

FIG. 4shows an example of a CAD database that is stored in at least one of the memory103and the memory203.

The CAD database stores object IDs that are the identification information for identifying objects such as the object8, CAD data (the names of files), and attribute information. The attribute information includes the titles, the user names indicating the creators of CAD data, and date data about the creation time and date of the CAD data.

FIG. 5is a flowchart showing an operation to be performed by the PC1.

The controller101of the PC1outputs an initial projection image to the projector4(step S1). The initial projection image is a black image.

The controller101then outputs an image captured by the camera5to the PC2via the transmitting and receiving unit102and the network3(step S2), and starts recording the sound that is input through the microphone10and the sound that is output from the speaker11(step S3).

The controller101then determines whether the image captured by the camera5includes the QR code7(step S4).

If the determination result of step S4is “YES”, the controller101decodes the QR code7to obtain the object ID of the object8(step S5), and searches the CAD database stored in the memory103for the CAD data corresponding to the object ID (step S6). In this manner, the controller101identifies the CAD data corresponding to the object ID (step S7). If the determination result of step S4is “NO”, the operation moves on to step S8.

Upon receipt of the coordinate data of an annotation image from the PC2through the transmitting and receiving unit102(step S8), the controller101performs a process for projecting an image in accordance with the coordinate data (step S9). More specifically, the controller101decodes and converts the coordinate data of the annotation image to a projection image for the projector4. The controller101outputs the projection image to the projector4. The projector4then projects the projection image, or the annotation image, onto the object8and the screen12.

The controller101then determines whether there is a received instruction to store the image captured by the camera5, the sound that is input through the microphone10, and the sound that is output through the speaker11(step S10). An instruction to store a captured image and sound may be input through the operating unit106or the operating unit204, or may be input by the camera5capturing a QR code including an instruction to store a captured image and sound.

If the determination result of step S10is “NO”, the operation moves on to step S12. If the determination result of step S10is “YES”, the controller101associates the image captured by the camera5, the sound input through the microphone10, and the sound output through the speaker11with the CAD data identified by the object ID of the object8in step S7, and stores the image and the sound associated with the CAD data into the memory103(step S11). If the determination result of step S4is “NO”, the object ID of the object8is not found. Therefore, the image captured by the camera5, the sound input through the microphone10, and the sound output through the speaker11are stored into the memory103, separately from the CAD data identified in step S7.

To reduce the size of the image captured by the camera5to be stored in step S11, the controller101may compare the image (still image) captured by the camera5with the previous captured image, and determine whether there is a difference greater than a predetermined threshold value. In this case, the controller101stores the image captured by the camera5into the memory103, only if the captured image has a difference greater than the threshold value with respect to the previous captured image. For example, in a case where the number of dots forming the image captured by the camera5is larger or smaller than the number of dots forming the previous captured image by 100 dots or more, or where the total number of dots forming the image captured by the camera5is larger or smaller than the total number of dots forming the previous captured image by 3% or more, the controller101determines that there is a difference greater than the predetermined threshold value, and stores the captured image into the memory103.

FIG. 6shows examples of associations between the CAD data and the captured images and audio data. InFIG. 6, the CAD data having the object ID “object001” and the captured image and audio data having the object ID “object001” are associated with each other, and are stored in the memory103. The attribute information about the CAD data having the object ID “object001” inFIG. 6has a link column as well as the items included in the attribution information about the captured image and audio data having the object ID “object001” inFIG. 4. The file names of the associated captured images and audio data are written in the link column. As will be described later, the controller101reads and outputs the captured image and audio data associated with the CAD data from the memory103, when the link to the captured image and audio data included in the attribute information about the CAD data is clicked.

The attribute information added to the captured images and audio data shown inFIG. 6are attribute information about meetings, and includes the titles, the user names of the meeting participants, and the dates of the meetings.

After step S11, the controller101determines whether all the procedures such as the projecting procedure and the image and sound storing procedure have been carried out (step S12). If the determination result of step S12is “NO”, the operation returns to step S2. If the determination result of step S12is “YES”, the operation comes to an end.

Although the controller101obtains an object ID through the QR code7in steps S4and S5, the process of obtaining an object ID is not limited to that. For example, the controller101may obtain an object ID that is input through the operating unit106or the operating unit204. Also, a RFID tag reader or a barcode reader (not shown) connected to the PC1may read a RFID tag or a barcode attached to the object8, so that the controller101obtains the object ID.

FIG. 7Ashows the captured image having the object ID “object001” and its attribute information.FIG. 7Bshows the transitions of the screen of the display105or205.

Through the operation shown inFIG. 5, the captured image shown inFIG. 7Aand audio data are stored in the memory103.

While the application for producing CAD data stored in the memory103or203is activated, the controller101causes the display105or205to display the CAD data corresponding to the object ID “object001” shown on the left side inFIG. 7B, when the object ID “object001” is input through the operating unit106or204. Further, when an instruction to display the attribute information about the CAD data, the controller101causes the display105or205to display the attribute information about the CAD data, as shown in the center ofFIG. 7B. If the link to “image001-025” and “sound001” is clicked through the operating unit106or204in this situation, the controller101causes the display105or205to display the window showing the captured image on the screen, as shown on the right side inFIG. 7B, and outputs the sound of “sound001” through the speaker11or19. In this manner, when the link to the captured image and audio data included in the attribute information about the CAD data is clicked, the controller101reads and outputs the captured image and audio data associated with the CAD data from the memory103.

As described above in detail, in accordance with this exemplary embodiment, the memory103stores the database containing the CAD data used to design the object8and their attribute information, and the controller101obtains the object ID (identification information) of the object8. Based on the object ID of the object8, the controller101identifies the CAD data about the object8contained in the database. The controller101associates the captured image data and the audio data with the CAD data about the object8, and stores the captured image data and the audio data associated with the CAD data into the memory103, so that the data of the image captured by the camera5and the audio data input and output through the microphone10and the speaker11can be viewed and listened to, based on the attribute information of the identified CAD data about the object8.

In this manner, the captured image and audio data about the object8obtained through a communication with the PC2can be readily viewed and listened to, based on the CAD data of the object8.

In the first exemplary embodiment, the controller101associates captured image data and audio data with identified CAD data of the object, and causes the memory103to store the captured image data and audio data with the CAD data. However, it is also possible to associate only the captured image data with the identified CAD data of the object, and cause the memory103to store the captured image associated with the CAD data. In such a case, the captured image data about the object8obtained through a communication with the PC2can be readily viewed, based on the CAD data of the object8. Also, the controller101(the data converter) may convert the audio data into text data by a known technique. The controller101then associates the captured image data and text data with the identified CAD data of the object, and causes the memory103to store the captured image data and the text data associated with the CAD data. In this manner, the captured image and the text data corresponding to the audio data about the object8obtained through a communication with the PC2can be readily viewed, based on the CAD data of the object8.

Furthermore, in a case where the object8is measured with the use of various measurement devices (such as a size measuring machine, a thermometer, a hydrometer, and an ammeter), the controller101may associate the data obtained through those measurement devices with the CAD data of the object, and cause the memory103to store the measurement data and the CAD data.

Second Exemplary Embodiment

In the first exemplary embodiment, the CAD data is associated with a captured image and audio data, and the associated image and data are stored in the memory103. In this exemplary embodiment, on the other hand, CAD data is associated with a captured image and audio data, and the associated image and data are stored in the memory203.

In the first exemplary embodiment, the CAD database is stored beforehand in the memory103. In this exemplary embodiment, on the other hand, the CAD database shown inFIG. 4is stored beforehand in the memory203.

FIG. 8is a flowchart of an operation to be performed by the PC2.

The controller201obtains a captured image from the PC1(step S21), and starts recording the sound that is input through the microphone18and the sound that is output from the speaker19(step S22).

The controller201then determines whether the captured image includes the QR code7(step S23).

If the determination result of step S23is “YES”, the controller201decodes the QR code7to obtain the object ID of the object8(step S24), and searches the CAD database stored in the memory203for the CAD data corresponding to the object ID (step S25). In this manner, the controller101identifies the CAD data corresponding to the object ID (step S26). If the determination result of step S23is “NO”, the operation moves on to step S27.

The controller201then determines whether a screen process command to draw an annotation image has been received through the operating unit204(step S27).

If the determination result of step S27is “NO”, the operation moves on to step S30. If the determination result of step S27is “YES”, the controller201draws the annotation image on the display region15in accordance with the screen process command (step S28), and transmits the coordinate data of the annotation image to the PC1through the transmitting and receiving unit202(step S29) The coordinate data transmitted to the PC1is received by the controller101through the transmitting and receiving unit102in step S8ofFIG. 5.

The controller201then determines whether an instruction to store the captured image, the sound input through the microphone18, and the sound output through the speaker19has been received (step S30). The instruction to store the captured image and the sound may be input through the operating unit106or the operating unit204, or may be input by the camera5capturing the QR code including the instruction to store the captured image and the sound.

If the determination result of step S30is “NO”, the operation moves on to step S32. If the determination result of step S30is “YES”, the associates the captured image, the sound input through the microphone18, and the sound output through the speaker19with the CAD data identified by the object ID of the object8in step S26, and stores the image and the sound associated with the CAD data into the memory203(step S31). If the determination result of step S23is “NO”, the object ID of the object8is not found. Therefore, the captured image, the sound input through the microphone18, and the sound output through the speaker19are stored into the memory203, separately from the CAD data identified in step S26.

To reduce the size of the image captured by the camera5to be stored in step S31, the controller201may compare the captured image (still image) with the previous captured image, and determine whether there is a difference greater than a predetermined threshold value. In this case, the controller201stores the captured image into the memory203, only if the captured image has a difference greater than the threshold value with respect to the previous captured image. For example, in a case where the number of dots forming the captured image is larger or smaller than the number of dots forming the previous captured image by 100 dots or more, or where the total number of dots forming the captured image is larger or smaller than the total number of dots forming the previous captured image by 3% or more, the controller201determines that there is a difference greater than the predetermined threshold value, and stores the captured image into the memory203.

After step S31, the controller201determines whether all the procedures such as the transmission of the coordinate data of the annotation image and the image and sound storing procedure have been carried out (step S32). If the determination result of step S32is “NO”, the operation returns to step S12. If the determination result of step S32is “YES”, the operation comes to an end.

Although the controller201obtains an object ID through the QR code7in steps S23and S24, the process of obtaining an object ID is not limited to that. For example, the controller201may obtain an object ID that is input through the operating unit106or the operating unit204. Also, the controller201may obtain an object ID that is input directly from the PC1.

Like the controller101explained with reference toFIGS. 7A and 7B, when the link to the captured image and audio data included in the attribute information about the CAD data is clicked, the controller201reads and outputs the captured image and audio data associated with the CAD data from the memory203.

As described above in detail, in accordance with this exemplary embodiment, the memory203stores the database containing the CAD data used to design the object8and their attribute information, and the controller201obtains the object ID (identification information) of the object8. Based on the object ID of the object8, the controller201identifies the CAD data about the object8contained in the database. The controller201associates the captured image data and the audio data with the CAD data about the object8, and stores the captured image data and the audio data associated with the CAD data into the memory203, so that the data of the image captured by the camera5and the audio data input and output through the microphone18and the speaker19can be viewed and listened to, based on the attribute information of the identified CAD data about the object8.

In this manner, the captured image and audio data about the object8obtained through a communication with the PC1can be readily viewed and listened to, based on the CAD data of the object8.

In the second exemplary embodiment, the controller201associates captured image data and audio data with identified CAD data of the object, and causes the memory203to store the captured image data and audio data with the CAD data. However, it is also possible to associate only the captured image data with the identified CAD data of the object, and cause the memory203to store the captured image associated with the CAD data. In such a case, the captured image data about the object8obtained through a communication with the PC1can be readily viewed, based on the CAD data of the object8. Also, the controller201(the data converter) may convert the audio data into text data by a known technique. The controller201then associates the captured image data and text data with the identified CAD data of the object, and causes the memory203to store the captured image data and the text data associated with the CAD data. In this manner, the captured image and the text data corresponding to the audio data about the object8obtained through a communication with the PC1can be readily viewed, based on the CAD data of the object8.

Furthermore, in a case where the object8is measured with the use of various measurement devices (such as a size measuring machine, a thermometer, a hydrometer, and an ammeter), the controller201may associate the data obtained through those measurement devices with the CAD data of the object, and cause the memory203to store the measurement data and the CAD data.

Third Exemplary Embodiment

In the first exemplary embodiment, the controller101associates the CAD data corresponding to the object ID of the object8with the captured image and audio data of the object8, and stores the CAD data associated with the captured image and audio data into the memory103. In cases where the object8is a component of a completed product and the object8is redesigned with the use of CAD data, operators might wish to refer not only to the captured image and audio data about the object8but also to the captured images and audio data about other components related to the object8. With such cases being taken into consideration in this exemplary embodiment, the controller101associates the CAD data corresponding to the object ID of the object8with the captured images and audio data of the object8and the other objected related to the object8, and stores the CAD data associated with the captured images and audio data into the memory103.

FIG. 9is a flowchart of an operation to be performed by the PC1. InFIG. 9, the same procedures as those ofFIG. 5are denoted by the same step numbers as those ofFIG. 5, and explanation of them is omitted here.

In the following description, an image of the object8is first captured, and after that, images of other components related to the object8are captured.

After step S9, the controller101displays a storage instruction dialog on the display105or205(step S41).

FIG. 10Ashows an example of the display105displaying a storage instruction dialog61.FIG. 10Bshows an example of the display205displaying the storage instruction dialog61.

The storage instruction dialog61includes: a temporary storage button62for temporarily storing the captured image and audio data of each object into the memory103; a final storage button63for storing finally the captured images and audio data of the object8and the other objects related to the object8into the memory103; and a cancel button64.

Referring back toFIG. 9, the controller101determines whether the temporary storage button62is pressed through the operating unit106or the operating unit204(step S42).

If the determination result of step S42is “YES”, the controller101temporarily stores the captured images and audio data into the memory103(step S32), and returns to step S2. If the controller101returns to step S2from step S43, the controller101switches to another sound recording file, and starts recording sound. If the controller101returns to step S2from step S43, images of objects other than the object8can be captured.

If the determination result of step S42is “NO”, the controller101determines whether the final storage button63is pressed through the operating unit106or the operating unit204(step S44).

If the determination result of step S44is “NO”, or when the cancel button64is pressed, the operation moves on to step S46.

If the determination result of step S44is “YES”, the controller101associates the image being currently captured and the audio data, and the captured images and the audio data temporarily stored in the memory103with the CAD data of the object8first identified by the object ID of the object8in step S7, and stores the images and the audio data associated with the CAD data into the memory103(step S45). If the determination result of step S4is “NO”, the object ID of the object8is not found. Therefore, the captured images and the audio data are stored into the memory103, separately from the CAD data of the object8identified in step S7.

To reduce the amount of the captured image data temporarily to be stored in step S43and the amount of the captured image data to be finally stored in step S45, the controller101may compare the image (still image) captured by the camera5with the previous captured image, and determine whether there is a difference greater than a predetermined threshold value. In this case, the controller101stores the captured image into the memory103, only if the captured image has a difference greater than the threshold value with respect to the previous captured image. For example, in a case where the number of dots forming the image captured by the camera5is larger or smaller than the number of dots forming the previous captured image by 100 dots or more, or where the total number of dots forming the image captured by the camera5is larger or smaller than the total number of dots forming the previous captured image by 3% or more, the controller101determines that there is a difference greater than the predetermined threshold value, and stores the captured image into the memory103.

FIG. 11shows an example case where one set of CAD data is associated with two or more captured images and two or more sets of audio data. InFIG. 11, the CAD data having the object ID “object001” is associated with two or more captured images and two or more sets of audio data, and are stored in the memory103. The attribute information about the CAD data having the object ID “object001” inFIG. 11has a link column as well as the items included in the attribution information about the CAD data having the object ID “object001” inFIG. 4. The file names of the associated captured images and audio data are written in the link column. The controller101reads and outputs the captured images and audio data associated with the CAD data from the memory103, when the link to the captured images and audio data included in the attribute information about the CAD data is clicked.

The attribute information added to the captured images and audio data shown inFIG. 11are attribute information about meetings, and includes the titles, the user names of the meeting participants, and the dates of the meetings.

After step S45, the controller101determines whether all the procedures such as the projecting procedure and the image and sound storing procedure have been carried out (step S46). If the determination result of step S46is “NO”, the operation returns to step S2. If the determination result of step S46is “YES”, the operation comes to an end.

In steps S41through S45, the controller101displays the storage instruction dialog on the display105or the display205, and determines whether the temporary storage button62or the final storage button63is pressed. However, the controller101may determine whether the temporary button62or the final storage button63is pressed, based on whether a predetermined key input (Alt+F1, for example) corresponding to the temporary storage button62or a predetermined key input (Alt+F2, for example) corresponding to the final storage button63is made.

In step S45, the controller101associates the image being currently captured and the audio data, and the captured images and the audio data temporarily stored in the memory103with the CAD data of the object8first identified by the object ID of the object8in step S7, and stores the images and the audio data associated with the CAD data into the memory103. However, the controller101may associate the image being currently captured and the audio data, and the captured images and the audio data temporarily stored in the memory103with the CAD data of the object8last identified by the object ID of the object8in step S7, and stores the images and the audio data associated with the CAD data into the memory103.

As shown inFIG. 12, the procedures of steps S41through S46may be added to the procedures of steps S21through S29. By doing so, the controller201of the PC2can associate the CAD data corresponding to the object ID of the object8with the captured images and audio data of the object8and the other objects related to the object8, and store the captured images and audio data associated with the CAD data into the memory203.

As described above in detail, in accordance with this exemplary embodiment, in a case where an image of each of the objects is captured and the audio data files are switched when the objects are switched, the controller101associates the captured image data and the audio data of the objects with the first or last identified CAD data about the object8, and stores the captured image data and the audio data associated with the CAD data into the memory103, so that the images of the objects captured by the camera5and the two or more sets of audio data input and output through the microphone10and the speaker11can be viewed and listened to, based on the attribute information of the first or last identified CAD data of the object8.

In this manner, based on the first or last identified CAD data about the object8, the captured images and audio data about objects including the object8obtained through a communication with the PC2can be readily viewed and listened to.

Likewise, in a case where an image of each of the objects is captured and the audio data files are switched when the objects are switched, the controller201associates the captured image data and the audio data of the objects with the first or last identified CAD data about the object8, and stores the captured image data and the audio data associated with the CAD data into the memory203, so that the images of the objects captured by the camera5and the two or more sets of audio data input and output through the microphone18and the speaker19can be viewed and listened to, based on the attribute information of the first or last identified CAD data of the object8.

In this manner, based on the first or last identified CAD data about the object8, the captured images and audio data about objects including the object8obtained through a communication with the PC1can be readily viewed and listened to.

When the temporary storage button62is pressed, the controller101or the controller201causes the memory103or the memory203to temporarily store the captured image data about each object and the audio data about each object. When the final storage button63is pressed, the controller101or the controller201associates the captured image data of the objects and the two or more sets of audio data with the first or last identified CAD data of the object8, and causes the memory103or the memory203to store the captured image data and the audio data of the objects associated with the CAD data, so that the images of the objects captured by the camera5and the two or more sets of audio data input and output through the microphone10and the speaker11or the microphone18and the speaker19can be viewed and listened to, based on the attribute information of the first or last identified CAD data of the object8.

Accordingly, when the temporary storage button62is pressed, the captured image and audio data of each object are temporarily stored into the memory103or the memory203. When the final storage button63is pressed, the captured images and audio data of the objects including the temporarily stored captured images and audio data are associated with the first or last identified CAD data of the object8, and can be stored into the memory103or the memory203.

In the third exemplary embodiment, the controller101or the controller201associates captured image data and audio data with identified CAD data of the object8, and causes the memory103or the memory203to store the captured image data and audio data associated with the CAD data. However, it is also possible to associate only the captured image data with the identified CAD data of the object8, and cause the memory103or the memory203to store the captured images associated with the CAD data. In such a case, the captured image data about the object8obtained through a communication with the PC2or the PC1can be readily viewed, based on the CAD data of the object8. Also, the controller101or the controller201(the data converter) may convert the audio data into text data by a known technique. The controller101or the controller201then associates the captured image data and text data with the identified CAD data of the object8, and causes the memory103or the memory203to store the captured image data and the text data associated with the CAD data. In this manner, the captured images and the text data corresponding to the audio data about the object8obtained through a communication with the PC2or the PC1can be readily viewed, based on the CAD data of the object8.

Furthermore, in a case where the object8is measured with the use of various measurement devices (such as a size measuring machine, a thermometer, a hydrometer, and an ammeter), the controller101or the controller201may associate the data obtained through those measurement devices with the CAD data of the object, and cause the memory103or the memory203to store the measurement data and the CAD data.

Alternatively, a recording medium having the software program for realizing the functions of the PC1and the PC2recorded thereon may be provided to each PC, and the CPU of each PC may read and execute the program recorded on the recording medium. In this manner, the same effects as those of the above-described exemplary embodiments can be achieved. The recording medium for supplying the program may be a CD-ROM, a DVD, a SD card, or the like.

Also, the CPU of each PC may execute the software program for realizing the functions of each PC. In this manner, the same effects as those of the above-described exemplary embodiments can also be achieved.

It should be understood that the present invention is not limited to the above-described exemplary embodiments, and various modifications may be made to them without departing from the scope of the invention.