Abstract:
A guidance device which performs measurement by placing an object without contact, comprising: a virtual image display unit displaying a virtual image; a sample display generation unit making the virtual image display unit display a sample virtual image based on a shape of the object at a position where the object is to be finally arranged; a status detection unit detecting a position and attitude of the object; a correction amount calculation unit calculating a difference between the position and attitude of the object and the position and attitude of the virtual image; and a guidance display generation device making the virtual image display unit display a guidance display indicating the calculated difference in the same position as the virtual image.

Description:
CROSS REFERENCE TO RELATED APPLICATION 
       [0001]    This application is a continuation of PCT application PCT/JP2006/311531 filed on Jun. 8, 2006. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The present invention relates to a guidance device and method for guiding an object to the correct position and attitude relative to a capture device in a device such as an inspection device, an authentication device, etc. for capturing a subject without contact. 
         [0004]    2. Description of the Related Art 
         [0005]    There is a device for capturing a subject without contact to perform an inspection and authentication. To obtain correct inspection and authentication results using this device, it is necessary to capture a image of a subject such as a hand in a predetermined position, direction, and shape. A user does not capture a image of a subject while watching through the finder of the device held by user hands as with a camera, but the subject is often captured by placing the subject over a capture device fixed to a device. Furthermore, since a subject is not fixed, the position, direction, shape, etc. of the subject are not determined, and it has been difficult to capture an appropriate image. 
         [0006]    Therefore, conventional techniques have adopted a method of showing a sample of placing a subject over a capture device, and notifying a user whether or not the current position is a correct position by voice etc. 
         [0007]    For example, the patent document 1 discloses the technology of applying a model of a hand as a sample to be placed over the surface of a captureing device of an authentication device for capturing an image of a vein without contact, detecting the position and direction of the hand, and providing a notification by voice, a bell, the light of a lamp, etc. when the hand is too close to the surface of the device. 
         [0008]    The patent document 2 discloses the technology of displaying a 3-dimensional image of a finger by a stereoscope, detecting the amount of overlap between the 3-dimensional image and the finger, and capturing an image when they match each other. 
         [0009]    However, in the conventional technology, it is hard to determine how the position and direction of a subject are to be corrected. Therefore, an appropriate position can be obtained only after much trial and error, and the positioning efficiency is very low. 
         [0010]    Patent Document 1: International Publication No. 04/021884 Pamphlet 
         [0011]    Patent Document 2: Japanese Published Patent Application No. H5-108808 
       SUMMARY 
       [0012]    A guidance device according to one aspect performs measurement by placing an object over the device without contact, and includes: a virtual image display unit for displaying a virtual image; a sample display generation unit for making the virtual image display unit display a sample virtual image based on the shape of the object at a position where the object is to be finally arranged; a status detection unit for detecting the position and attitude of the object; an unit for calculating the amount of correction on the difference between the position and attitude of the object and the position and attitude of the virtual image; and a guidance display generation unit for making the virtual image display unit display a guidance display indicating the calculated difference in the same position as the virtual image. 
         [0013]    It is desired that, in the guidance device according to the present invention, the virtual image has the same size and shape as the object. 
         [0014]    It is also desires that, in the guidance device according to the present invention, the size and shape of the virtual image are determined by the data recorded on a storage medium. 
         [0015]    In addition, it is desired that, in the guidance device according to the present invention, the guidance display is an arrow display indicating the shift of the position, the change of the attitude, or the change of the shape from the current object to the virtual image. 
         [0016]    In addition, it is desired that, in the guidance device according to the present invention, the guidance is an animation display indicating the shift of the position, the change of the attitude, or the change of the shape from the current object to the virtual image. 
         [0017]    Furthermore, it is desired that, in the guidance device according to the present invention, the display of the shift of the position, the change of the attitude, or the change of the shape is separately performed. 
         [0018]    It is also desired that, in the guidance device according to the present invention, auxiliary lines indicating the position and attitude are displayed around each of the object and the virtual images. 
         [0019]    In addition, it is desired that, in the guidance device according to the present invention, auxiliary planes indicating the position and attitude are displayed around each of the object and the virtual images. 
         [0020]    According to configuration above, since a user can recognize a practical amount of correction to obtain the correct position and attitude without moving the line of sight from a subject, the subject can be efficiently positioned without wasteful operations. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS  
         [0021]      FIG. 1  shows the configuration of the guidance device according to an embodiment of the present invention; 
           [0022]      FIG. 2  shows the process flow of the processing unit of the guidance device according to an embodiment of the present invention; 
           [0023]      FIG. 3  shows the configuration of the display device of the guidance device according to an embodiment of the present invention; 
           [0024]      FIG. 4  shows the principle of displaying a virtual image using a concave mirror; 
           [0025]      FIG. 5  shows the display device using a stereoscope; 
           [0026]      FIG. 6  shows the display device using polarizing glasses; 
           [0027]      FIG. 7  is an explanatory view of the change of the position of a subject; 
           [0028]      FIG. 8  is an explanatory view of changes of the attitude of a subject; 
           [0029]      FIG. 9  shows a change of the shape of a subject; 
           [0030]      FIG. 10  shows the guidance display according to an embodiment of the present invention when a displacement occurs; 
           [0031]      FIG. 11  shows the guidance display according to an embodiment of the present invention when a change of attitude occurs; 
           [0032]      FIG. 12  shows the guidance display according to an embodiment of the present invention when a deformation occurs; 
           [0033]      FIG. 13  is a flowchart of the guidance process by the guidance device according to an embodiment of the present invention; 
           [0034]      FIG. 14  is a flowchart when correction animation is used in guidance display; 
           [0035]      FIG. 15  is an explanatory view of the detecting method according to an embodiment of the present invention when a sample is hidden; 
           [0036]      FIG. 16  shows auxiliary display by an auxiliary line according to an embodiment of the present invention; and 
           [0037]      FIG. 17  shows auxiliary display by an auxiliary plane according to an embodiment of the present invention. 
       
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
       [0038]    The embodiments of the present invention are described below with reference to the attached drawings. 
         [0039]      FIG. 1  shows the configuration of the guidance device according to an embodiment of the present invention. 
         [0040]    The guidance device according to the present invention includes a display device  101  for displaying a guidance display  107  indicating a sample virtual image  106  and the amount of correction, a position/attitude/shape detection device (status detection device)  102  for detecting the status such as the position, attitude, shape, etc. of a subject  108 , a storage medium  103  storing the shape and the size of the subject  108 , a processing unit  104  for processing each signal, and a camera  105  for capturing the subject  108 . 
         [0041]    The storage medium  103  is a portable storage medium such as an IC card, a magnetic card, etc., and records the size and the shape of a subject such as the hand of a user etc. Since the size and the shape of a subject such as a hand etc. depend on each user, it is desired that the data of the size and the shape of a subject recorded on the storage medium  103  is the data indicating the same shape and size as the subject such as the hand of a user etc. During the operation, the user sets the storage medium  103  on the read unit of the guidance device. The storage medium  103  can be a storage device built in the guidance device or a storage device connected over a network. 
         [0042]    The guidance device according to the present invention displays the sample virtual image  106  to be seen in front of the camera on the basis of the shape and the size of the subject  108  and the correct position relative to the camera  105 . When the user places the subject  108  closer to the device, the image of the subject  108  is captured by the camera  105 , and the position, attitude, and shape of the subject  108  are detected by the position/attitude/shape detection device  102 . 
         [0043]    The position, attitude, and shape of the subject  108  are compared with the sample information recorded on the storage medium  103 , the amount to be corrected is calculated by the processing unit  104 , and the display indicating the amount to be corrected is shown close to the sample virtual image  106 . The user moves the subject to the correct position while simultaneously watching the sample virtual image  106 , the guidance display  107 , and the subject  108 . 
         [0044]    Described below are the details of each unit. 
         [0045]      FIG. 2  shows the process flow of the processing unit of the guidance device according to an embodiment of the present invention. 
         [0046]    In step S 201 , the sample data generation unit in the processing unit of the guidance device calculates correct sample data from the shape/size information about the subject read from the storage medium  103  and the correct position information (not shown in the attached drawings) on the camera  105  stored in the guidance device, and the sample data is transmitted to the sample display generation unit and the correction amount calculation unit in the processing unit of the guidance device. 
         [0047]    In step S 202 , the sample display generation unit in the processing unit of the guidance device generates a sample display signal on the basis of the calculated sample data, and outputs the signal to the display device  101 . 
         [0048]    On the other hand, in step S 203 , the position, attitude, and shape of the subject detected by the position/attitude/shape detection device  102  are compared with the sample data by the correction amount calculation unit in the processing unit of the guidance device, and the amount of correction is calculated and output to the guidance display generation unit in the process unit of the guidance device. 
         [0049]    In step S 204 , the guidance display generation unit generates a guidance display signal on the basis of the calculated amount of correction, outputs the signal to the display device  101 , and the guidance display  107  is displayed. 
         [0050]      FIG. 3  shows the configuration of the display device of the guidance device according to an embodiment of the present invention. 
         [0051]    In  FIG. 3 , the display device  101  displays a virtual image using a concave mirror  301 . 
         [0052]      FIG. 4  shows the principle of displaying a virtual image using a concave mirror. 
         [0053]    As shown in  FIG. 4 , two concave mirrors  401  and  402  are arranged opposite each other with the focus of one of the mirror arranged at the center of the other mirror. Then, the virtual image of an object placed at the center of one concave mirror  402  appears at the center of the other concave mirror  401 . Since the concave mirror can sufficiently function with a part of the mirror cut out, only a part of the concave mirror  301  can be used as shown in  FIG. 3  so far as the direction of a user can be limited. As shown in  FIG. 3 , when a display panel  302  such as a liquid crystal display etc. is arranged at the center of the concave mirror  301 , and a sample virtual image generated by the processing unit shown in  FIG. 2  and an image for performing guidance display are displayed, the virtual images  106  and  107  appear forward the lens of the camera  105  by the reflection of the concave mirror  301 . 
         [0054]    According to the present invention, the device for realizing virtual display is not limited to the above-mentioned device, but another device can be used. 
         [0055]      FIG. 5  shows the display device using a stereoscope. 
         [0056]    In  FIG. 5 , the display device using a stereoscope separately shows parallax images to the left and right eyes of a user, has display panels  501  and  502  in the scope, displays parallax images on the display panels  501  and  502 , and shows the images reflected by half mirrors  503  and  504  to the left and right eyes, respectively, thereby showing a virtual image forward. Since the image is projected by the half mirror, the outside real world can be simultaneously observed. 
         [0057]      FIG. 6  shows the display device using polarizing glasses. 
         [0058]    In  FIG. 6 , the display device using polarizing glasses  601  displays parallax images on a display panel  602  capable of displaying two polarization states, or projects images on two projectors in the separate polarization states so that the images can be observed by the glasses with the polarizing filters for the right and left polarization, thereby showing a 3-dimensional image. 
         [0059]    Other special glass systems can be a system displaying parallax images with a time difference and having right and left shutters, a system displaying parallax images in different colors such as red and blue, etc. and viewed using right and left color filters, etc. 
         [0060]    In addition, a virtual image can also be realized by providing a prism on the surface of a display panel (although not shown in the attached drawings) so that separate images can be shown to the left and right eyes of the user. 
         [0061]    Described next is the position/attitude/shape detection device according to an embodiment of the present invention. Before describing the position/attitude/shape detection device according to an embodiment of the present invention, items relevant to the position, attitude, and shape are described. 
         [0062]      FIG. 7  is an explanatory view of the change in position of a subject. 
         [0063]    A subject has the freedom of moving parallel to the x, y, and X axes orthogonal to one another. Detecting a position refers to detecting the amount of parallel movement of the subject relative to the correct position. The positions in the x and y directions can be easily detected from the images obtained by the camera  105 . Although it is difficult to accurately detect the position in the z direction, it can be easily detected by a distance sensor in, for example, an infrared reflection system. 
         [0064]      FIG. 8  is an explanatory view of a change of the attitude of a subject. 
         [0065]    As shown in  FIG. 8 , the subject has the freedom of rotating on the current position. The rotation is performed on the three axes, that is, the x, y, and z axes. Detecting the attitude refers to detecting the amount of rotation from the correct status (in which direction the subject is placed). A change in attitude on the z axis can be easily detected from the image captured by the camera  105 . The technique of detecting the changes in attitude on the x and y axes is disclosed by the International Publication No. 04/021884 Pamphlet “Personal Recognition Device” disclosed by the Applicant. Also disclosed is the technique of detecting the attitude by a capturing device provided with a plurality of distance sensors described in International Publication No. 04/084140 Pamphlet “Shooting Device”. 
         [0066]      FIG. 9  shows a change in shape of a subject. 
         [0067]    Normally, the freedom of an object placed in space can be 6 types of freedom shown in  FIGS. 7 and 8 . However, there can be a subject whose shape can be changed by a connection point (joint) where a plurality of portions are coupled such as a folding mobile telephone, a human body, etc. 
         [0068]    Therefore, a change in shape as shown in  FIG. 9  is also considered.  FIG. 9  shows a change in opening fingers, for example. On the right in  FIG. 9 , the finger enclosed by the circle is opened while it is closed on the left in  FIG. 9 . The change in shape can be detected by comparing the shape obtained from the image of the capturing camera  105  with the shape of the sample image. 
         [0069]    Described next is the guidance display according to an embodiment of the present invention. 
         [0070]      FIG. 10  shows the guidance display according to an embodiment of the present invention when a displacement occurs. 
         [0071]    The sample virtual image  106  is displayed at the position where the subject  108  is to be placed. Therefore, when the position of the subject  108  is not appropriate, the sample virtual image  106  and the subject  108  are observed separately. The graphics indicating the amount of movement to be made in a given direction are overlapped and displayed on the sample virtual image  106 . That is, the graphics are displayed at the same position as the subject  108  and the sample virtual image  106 , especially the graphics are displayed at between the subject  108  and the sample virtual image  106 . For example, as shown in  FIG. 10 , animation display of the movement of the sample virtual image  106  is performed from the current position of the subject  108  to the correct position, or an arrow is displayed from the current position to the correct position with the length depending on the moving distance. 
         [0072]      FIG. 11  shows the guidance display according to an embodiment of the present invention when a change of attitude occurs. 
         [0073]    When the attitude (angle) of the sample virtual image  106  is different from the attitude of the subject  108 , the sample virtual image  106  and the subject  108  are observed respectively as with the case of the displacement described above. The graphics indicating the amount of rotation to be made in a given direction are overlapped and displayed on the sample virtual image  106 . For example, as shown in  FIG. 11 , animation display of the rotation of the sample virtual image  106  is performed from the current attitude of the subject  108  to the correct attitude, or an arrow is displayed depending on the rotation angle. 
         [0074]      FIG. 12  shows the guidance display according to an embodiment of the present invention when a deformation occurs. 
         [0075]      FIG. 12  practically shows an example when the thumb is crooked. The portion without deformation corresponding to the sample virtual image  106  is hidden behind the subject  108 , but each of the deformed portion corresponding to the sample virtual image  106  and the subject  108  visually recognized. The graphics indicating the amount of movement to be made to the deformed portion are overlapped and displayed on the sample virtual image  106 . For example, as shown in  FIG. 12 , animation display of the movement or arrow display is performed from the current position to the correct position. 
         [0076]    Thus, when the guidance display is performed, there is the possibility of incomprehensible display if all correction elements including the movement, attitude, and deformation are simultaneously displayed. Therefore, each element is assigned a priority and then guided. The priority can be assigned by: 
         [0077]    (1) repeating the guidance until the convergence is attained in the order of movement, attitude, and deformation; and 
         [0078]    (2) repeating the guidance in the order from the largest correction rate until the convergence is attained. 
         [0079]    The correction rate is defined as the rate of an amount of correction to the reference determined for each of the position, attitude, and deformation. For example, the change in position can be defined as an amount of change in position with 5 cm set as 100%, the change in attitude can be defined as an amount of change in angle with 30° as 100%, and the deformation can be defined as a rate of an amount of corrected area to the area of a sample projected on the Z axis, thereby comparing the amount of correction between different dimensions. 
         [0080]    Since the subject  108  completely overlaps the sample virtual image  106  when the position, attitude, and shape are correct as a result of the guidance, the display of the sample virtual image cannot be seen. Thus, the completion of the correction can be informed, but a further clear notification can be made by performing auxiliary display by changing the color around the sample virtual image etc. 
         [0081]    In addition, voice, a bell, a lamp, etc. can be used as auxiliary means from the start to the end of the guidance above as with the conventional technology. 
         [0082]      FIG. 13  is a flowchart of the guidance process by the guidance device according to an embodiment of the present invention.  FIG. 13  practically shows the case where the guidance is performed in order from the highest correction rate. 
         [0083]    In step S 1301 , a record medium such as an IC card etc. is inserted into a read unit. 
         [0084]    In step S 1302 , the information about the shape and size of a subject is read from the storage medium, and a sample image is generated. 
         [0085]    In step S 1303 , the position of the subject is detected. 
         [0086]    In step S 1304 , it is determined whether or not the subject is in a guidance range. If it is in the guidance range, control is passed to step S 1305 . If not, control is returned to step S 1303 . The guidance range is set as a range in which the display device can display a virtual image. 
         [0087]    In step S 1305 , a sample generated in step S 1302  is displayed. 
         [0088]    In step S 1306 , the position, attitude, and shape of the subject are detected. 
         [0089]    In step S 1307 , the correction rate of each of the position, attitude, and shape is calculated. 
         [0090]    In step S 1308 , it is determined whether or not all correction rates are within the respective references. If yes, control is passed to step S 1309 . If not, control is passed to step S 1311 . 
         [0091]    In step S 1309 , the subject is shot. 
         [0092]    In step S 1310 , the user is notified of the completion of the process above, and the display of the sample is cleared. 
         [0093]    In step S 1311 , the respective correction rates of the position, attitude, and shape are compared, and the highest rate is selected. 
         [0094]    In step S 1312 , the guidance display is performed. Then, control is returned to step S 1306 . 
         [0095]      FIG. 14  is a flowchart when correction animation is used in guidance display. 
         [0096]    In step S 1401 , the variable i is set to 1. 
         [0097]    In step S 1402 , the image showing the subject being shifted toward the sample by {(total amount of correction/n)*i} is generated. 
         [0098]    In step S 1403 , displayed correction image is cleared if any, and the image generated in step S 1402  is displayed, thereby updating the display image. 
         [0099]    In step S 1404 , i is incremented. 
         [0100]    In step S 1405 , it is determined whether i equals n (i=n) If i=n, control is passed to step S 1406 , and control is returned to S 1402  when i≠n. It is also possible to return control to step S 1402  to terminate the process after entering the loop several times. 
         [0101]    For example, consider the animation of displaying the correction of the position. Assume n is 10 (n=10). When the position of the subject is displaced by 5 cm relative to the sample virtual image, the total amount of correction is 5 cm. Since 0.5 cm is obtained by dividing 5 cm by 10, a virtual image 0.5 cm closer to the sample virtual image from the subject is displayed, and a virtual image further closer by 0.5 cm, that is, a total of 1 cm closer to the sample virtual image is displayed. Similarly, by sequentially displaying the virtual images 0.5 cm sequentially closer to the sample, the animation of displaying the correction can be configured. 
         [0102]      FIG. 15  is an explanatory view of the detecting method according to an embodiment of the present invention when a sample is hidden. 
         [0103]    Exceptionally, when the subject  108  covers the sample virtual image  106  when viewed from a user, the sample virtual image  106  can be unseen in the incomplete correction state. In this case, it is desired to use auxiliary display such as auxiliary lines, auxiliary planes, etc. 
         [0104]    Since the guidance display is performed only when the subject is in a detectable range, the auxiliary display can be performed when the subject  108  enters the range indicated by the diagonal lines where the detectable area overlaps the area from the line of sight to the sample virtual image  106  as shown in  FIG. 15 . A margin is required for the area in which auxiliary display is necessary because the area depends on the height of a user. However, since a user uses the device from the direction predetermined to a certain extent, the position of the line of sight of the user can be limited to a certain extent. The area requiring auxiliary display is measured and determined in advance. 
         [0105]      FIG. 16  shows auxiliary display by an auxiliary line according to an embodiment of the present invention. 
         [0106]      FIG. 16  practically shows an example of displaying auxiliary lines indicating the position and attitude as enclosing the outside of the subject and the sample virtual image. The auxiliary lines are drawn with the subject  108  and the sample virtual image  106  set at the center, and when the auxiliary lines are displayed, the guidance display such as arrows etc. is performed as overlapping the auxiliary lines, not the subject  108  itself. 
         [0107]      FIG. 17  shows auxiliary display by an auxiliary plane according to an embodiment of the present invention. 
         [0108]      FIG. 17  practically shows an example of displaying auxiliary planes indicating the position and attitude around the subject and the sample virtual image. The guidance is performed by overlapping the animation display and arrow display in which the auxiliary plane indicating the subject  108  moves to the position of the sample virtual image. 
         [0109]    The auxiliary lines and auxiliary planes can be displayed only when the subject exists in a position where there is the possibility that a sample virtual image is hidden, or can be constantly displayed. However, it is appropriate only when they are required. 
         [0110]    The embodiments of the present invention are described with reference to the attached drawings, but the present invention is not limited to the embodiments described above, and various configurations and shapes can be realized within the scope of the gist of the present invention.