Abstract:
An image processing method includes acquiring a three-dimensional model obtained by modeling a plurality of objects included in a work space, acquiring, from a camera which is hold by a user, an image captured by the camera, the user existing in the work space, acquiring, from a sensor which is hold by the user, distance information indicating distances between the sensor and each of the plurality of objects, determining a position of the user in the work space based on the three-dimensional model and the distance information, identifying a predetermined region closest to the position of the user among at least one of predetermined regions defined in the three-dimensional model, generating a display screen displaying the predetermined region and the image, and outputting the display screen to another computer.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2015-167814, filed on Aug. 27, 2015, the entire contents of which are incorporated herein by reference. 
       FIELD 
       [0002]    Techniques disclosed in the present embodiments are related to an image processing device and so forth. 
       BACKGROUND 
       [0003]    In recent years, an augmented reality (AR) technique has existed in which additional information is displayed in a superimposed manner by computer graphics and so forth on a screen of a terminal device used by a worker at a work site and assistance for work is carried out. 
         [0004]      FIG. 23  is a diagram illustrating one example of the AR technique. As illustrated in  FIG. 23 , for example, when a user photographs a marker  11  and a check target  12  by using a camera incorporated in a portable terminal  10 , object information  13  with respect to the marker  11  is superimposed on a display screen  10   a  of the portable terminal  10 . 
         [0005]    There is conventional technique 1 in which, by applying this AR technique, a worker transmits a photographed image photographed by a camera to a remote assistance person who is not present at the site and the remote assistance person makes an instruction of work to the worker while viewing the transmitted photographed image. For example, in conventional technique 1, the remote instruction person assists the worker with work by giving a marker to a work target included in the photographed image and causing the photographed image given the marker to be displayed on the terminal device of the worker at the work site. 
         [0006]    Furthermore, conventional technique 2 exists in which, in order to support a doctor who carries out an operation, information on a dangerous range is set in advance in a three-dimensional model of an affected area of a patient and the distance between a hand of the doctor and the dangerous range is displayed on a photographed image by a camera in a superimposed manner. 
         [0007]    The techniques related to the conventional technique are disclosed in Japanese Laid-open Patent Publication No. 2009-43242, Japanese Laid-open Patent Publication No. 2000-279425, Japanese Laid-open Patent Publication No. 2014-174507, and Japanese Laid-open Patent Publication No. 11-309269, for example. 
       SUMMARY 
       [0008]    According to an aspect of the embodiments, an image processing method executed by a computer, the image processing method comprising: acquiring a three-dimensional model obtained by modeling a plurality of objects included in a work space; acquiring, from a camera which is hold by a user, an image captured by the camera, the user existing in the work space; acquiring, from a sensor which is hold by the user, distance information indicating distances between the sensor and each of the plurality of objects; determining a position of the user in the work space based on the three-dimensional model and the distance information; identifying a predetermined region closest to the position of the user among at least one of predetermined regions defined in the three-dimensional model; generating a display screen displaying the predetermined region and the image; and outputting the display screen to another computer. 
         [0009]    The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. 
         [0010]    It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0011]      FIG. 1  is a diagram illustrating a configuration of a remote work assistance system according to present embodiment 1; 
           [0012]      FIG. 2  is a functional block diagram illustrating a configuration of an image processing device according to the present embodiment 1; 
           [0013]      FIG. 3  is a diagram illustrating one example of a photographed image and an image based on three-dimensional distance information; 
           [0014]      FIG. 4  is a diagram for explaining three-dimensional model information, work range information, and dangerous range information; 
           [0015]      FIG. 5  is diagram (1) illustrating one example of a display screen generated by a generating unit according to the present embodiment 1; 
           [0016]      FIG. 6  is diagram (2) illustrating one example of the display screen generated by the generating unit according to the present embodiment 1; 
           [0017]      FIG. 7  is a flowchart illustrating a procedure of processing of an image processing device according to the present embodiment 1; 
           [0018]      FIG. 8  is a flowchart illustrating a procedure of processing of generating information on a display screen; 
           [0019]      FIG. 9  is a flowchart illustrating a procedure of processing of determining a work target; 
           [0020]      FIG. 10  is a flowchart illustrating a procedure of processing of determining a dangerous range; 
           [0021]      FIG. 11  is a diagram for explaining another kind of processing (1) of the generating unit; 
           [0022]      FIG. 12  is a diagram for explaining processing of adjusting a placement position of a photographed image; 
           [0023]      FIG. 13  is a flowchart illustrating a processing procedure of another kind of processing (1) of the generating unit; 
           [0024]      FIG. 14  is a diagram for explaining another kind of processing (2) of the generating unit; 
           [0025]      FIG. 15  is diagram (1) illustrating one example of a display screen generated by a generating unit according to present embodiment 2; 
           [0026]      FIG. 16  is a flowchart illustrating a processing procedure of another kind of processing (2) of the generating unit; 
           [0027]      FIG. 17  is a diagram for explaining another kind of processing (3) of the generating unit; 
           [0028]      FIG. 18  is a diagram illustrating one example of a data structure of a rotation amount table; 
           [0029]      FIG. 19  is a flowchart illustrating a processing procedure of another kind of processing (3) of the generating unit; 
           [0030]      FIG. 20  is diagram (2) illustrating one example of the display screen generated by the generating unit according to the present embodiment 2; 
           [0031]      FIG. 21  is a diagram for explaining one example of another rotation axis; 
           [0032]      FIG. 22  is a diagram illustrating one example of a computer that executes an image processing program; and 
           [0033]      FIG. 23  is a diagram illustrating one example of an AR technique. 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0034]    However, in the above-described conventional techniques, there is a problem that it is difficult for the work assistance person to easily grasp the situation of the worker at the work site. 
         [0035]    For example, in conventional technique 1, the worker and the remote assistance person share a two-dimensional photographed image and therefore it is difficult for the work assistance person to grasp a sense of distance between the worker and the work target and so forth and grasp the situation of the work. 
         [0036]    Furthermore, in conventional technique 2, it is difficult to grasp the situation of the worker in the whole work space although the local distance between the dangerous range and the worker can be grasped. 
         [0037]    In one aspect, the embodiments intend to provide an image processing device, an image processing method, and an image processing program that allow a work assistance person to easily grasp the situation of a worker at a work site. 
         [0038]    Embodiments of an image processing device, an image processing method, and an image processing program disclosed by the present application will be described in detail below with reference to the drawings. This invention is not limited by the embodiments. 
       Embodiment 1 
       [0039]      FIG. 1  is a diagram illustrating a configuration of a remote work assistance system according to the present embodiment 1. As illustrated in  FIG. 1 , the remote work assistance system includes an image processing device  100  and a remote assistance person terminal  200 . For example, the image processing device  100  and the remote assistance person terminal  200  are mutually coupled via a network  50 . 
         [0040]    The image processing device  100  is a device used by a worker at a work site. The image processing device  100  notifies the remote assistance person terminal  200  of information on a photographed image photographed by a camera. Furthermore, when transmitting the information on the photographed image to the remote assistance person terminal  200 , the image processing device  100  determines the work position of the worker from a three-dimensional model of the work space and three-dimensional distance information of a distance sensor. The image processing device  100  generates a display screen including information on the direction and distance of a dangerous region or a work region from the work position of the worker and notifies the remote assistance person terminal  200  of the display screen in conjunction with the information on the photographed image. 
         [0041]    The remote assistance person terminal  200  is a device used by an assistance person who assists the worker with work. For example, the remote assistance person terminal  200  displays the display screen notified from the image processing device  100 . The remote assistance person thereby grasps the work situation of the worker and carries out various kinds of assistance. 
         [0042]      FIG. 2  is a functional block diagram illustrating a configuration of the image processing device according to the present embodiment 1. As illustrated in  FIG. 2 , this image processing device  100  includes a communication unit  110 , a distance sensor  120 , a camera  130 , an input unit  140 , a display unit  150 , a storing unit  160 , and a control unit  170 . For example, the distance sensor  120  and the camera  130  are set on a helmet or the like worn by a worker at the time of work. 
         [0043]    The communication unit  110  is a communication device that carries out data communications with the remote assistance person terminal  200  via the network  50 . The control unit  170  to be described later transmits and receives data via the communication unit  110 . 
         [0044]    The distance sensor  120  is a sensor that measures the three-dimensional distance from the distance sensor  120  to an object included in the measurement range. For example, the distance sensor  120  measures the three-dimensional distance on the basis of a triangulation system, time-of-flight, or the like. In the following description, information on the three-dimensional distance measured by the distance sensor  120  will be represented as three-dimensional distance information. The distance sensor  120  outputs the three-dimensional distance information to the control unit  170 . 
         [0045]    The camera  130  is a device that photographs an image of a photographic range. The image photographed by the camera  130  will be represented as the photographed image. The camera  130  outputs information on the photographed image to the control unit  170 . For example, the camera  130  may be mounted on a head mounted display (HMD) the worker wears at the head. 
         [0046]      FIG. 3  illustrates one example of a photographed image and an image based on three-dimensional distance information. In  FIG. 3 , an image  14  is the photographed image and an image  15  is the image based on three-dimensional distance information. This image based on three-dimensional distance information is expressed in such a manner that a closer part is brighter (hatching is coarser or white) and a farther part is darker (hatching is finer or black). When attention is paid to a partial image  14   a  of the image  14 , it is difficult to grasp the distance. In contrast, when attention is paid to a partial image  15   a  of the image  15 , the distance can be grasped on the basis of the thickness of the color. 
         [0047]    The input unit  140  is an input device that inputs various kinds of information to the image processing device  100 . The input unit  140  corresponds to an input device such as a touch panel and input buttons. 
         [0048]    The display unit  150  is a display device that displays information output from the control unit  170 . The display unit  150  corresponds to a liquid crystal display, a touch panel, or the like. 
         [0049]    The storing unit  160  includes three-dimensional model information  160   a , work range information  160   b , and dangerous range information  160   c . The storing unit  160  corresponds to a storing device such as a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), and a flash memory for example. 
         [0050]    The three-dimensional model information  160   a  is information obtained by modeling the shapes of plural objects included in the work space. For example, the three-dimensional model information  160   a  disposes plural objects on the basis of the origin of a world coordinate system set in advance and defines the three-dimensional coordinates at which the objects are disposed and the shapes of the objects. 
         [0051]    The work range information  160   b  is information that defines a work target on which a worker carries out work in the work space. For example, in the work range information  160   b , the work target is defined by three-dimensional coordinates on the basis of the origin of the world coordinate system set in advance. 
         [0052]    The dangerous range information  160   c  is information that defines a dangerous region in the work space. For example, in the dangerous range information  160   c , the dangerous range is defined by three-dimensional coordinates on the basis of the origin of the world coordinate system set in advance. 
         [0053]      FIG. 4  is a diagram for explaining the three-dimensional model information, the work range information, and the dangerous range information. In  FIG. 4 , appearance when the three-dimensional model information  160   a  is viewed from the front side is illustrated. As illustrated in  FIG. 4 , the three-dimensional coordinates and shapes of various kinds of objects are defined in the three-dimensional model information  160   a . When the work target corresponding to the work range information  160   b  is indicated by using the three-dimensional model information  160   a , a work target  61  is obtained for example. When the dangerous range corresponding to the dangerous range information  160   c  is indicated by using the three-dimensional model information  160   a , dangerous ranges  62   a  and  62   b  are obtained for example. 
         [0054]    Referring back to  FIG. 2 , the control unit  170  includes an acquiring unit  170   a , a determining unit  170   b , a generating unit  170   c , and a notifying unit  170   d . The generating unit  170   c  is one example of a screen generating unit. The control unit  170  corresponds to an integrated device such as an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA) for example. Furthermore, the control unit  170  corresponds to an electronic circuit such as a central processing unit (CPU) and a micro processing unit (MPU) for example. 
         [0055]    The acquiring unit  170   a  acquires the three-dimensional model information  160   a , the work range information  160   b , and the dangerous range information  160   c  from the communication unit  110  or the input unit  140 . The acquiring unit  170   a  stores, in the storing unit  160 , the three-dimensional model information  160   a , the work range information  160   b , and the dangerous range information  160   c  that are acquired. 
         [0056]    Furthermore, the acquiring unit  170   a  acquires three-dimensional distance information from the distance sensor  120  and outputs the acquired three-dimensional distance information to the determining unit  170   b . The acquiring unit  170   a  acquires a photographed image from the camera  130  and outputs the acquired photographed image to the generating unit  170   c.    
         [0057]    The determining unit  170   b  is a processing unit that determines the work position of the worker on the basis of the three-dimensional distance information and the three-dimensional model information  160   a . The determining unit  170   b  determines a work target close to the work position on the basis of the work position of the worker and the work range information  160   b . The determining unit  170   b  determines a dangerous range close to the work position on the basis of the work position of the worker and the dangerous range information  160   c . The determining unit  170   b  outputs the determination result to the generating unit  170   c . In the following, one example of the processing of the determining unit  170   b  will be described. 
         [0058]    Processing of determining the work position by the determining unit  170   b  will be described. Regarding the three-dimensional distance information and the three-dimensional model information  160   a , the determining unit  170   b  carries out alignment between the respective points of the three-dimensional distance information and the respective points of the three-dimensional model information  160   a  on the basis of the iterative closest point (ICP). The determining unit  170   b  thereby estimates the three-dimensional coordinates of the camera  130 . If the camera  130  is set on a helmet of the worker, the three-dimensional coordinates of the camera  130  correspond to the position of the head of the worker. 
         [0059]    Furthermore, the determining unit  170   b  extracts a region of a hand of the worker from the difference between the three-dimensional distance information and the three-dimensional model information  160   a . The determining unit  170   b  combines the three-dimensional coordinates of the head of the worker and the three-dimensional coordinates of the extracted region of the hand to thereby determine the work position of the worker. 
         [0060]    Subsequently, processing of determining a work target close to the work position by the determining unit  170   b  will be described. The determining unit  170   b  calculates the traveling direction of the hand by repeatedly executing the processing of extracting the above-described hand region every time the three-dimensional distance information is accepted. The determining unit  170   b  compares a straight line extending along the traveling direction of the hand and the three-dimensional coordinates of the work target defined in the work range information  160   b  to determine the work target whose distance from the straight line is the shortest. 
         [0061]    If the determining unit  170   b  fails at the extraction of the hand region or if the distance between the straight line and the work target is not within a prescribed distance, the determining unit  170   b  determines the work target on the basis of the central position of the field of view of the worker. For example, the determining unit  170   b  assumes the position of an object included in the central part of the three-dimensional distance information to be the central position of the field of view. The determining unit  170   b  determines the work target closest to the central position of the field of view. 
         [0062]    Subsequently, processing of determining a dangerous range close to the work position by the determining unit  170   b  will be described. The determining unit  170   b  calculates the traveling direction of the hand similarly to the above-described processing. The determining unit  170   b  compares a straight line extending along the traveling direction of the hand and the three-dimensional coordinates of the dangerous range defined in the dangerous range information  160   c  to determine a dangerous range whose distance from the straight line is shorter than a prescribed distance D1. 
         [0063]    Furthermore, the determining unit  170   b  determines a dangerous range whose distance from the present three-dimensional position of the hand is shorter than a prescribed distance D2. The determining unit  170   b  determines a dangerous range whose distance from the three-dimensional coordinates of the head of the worker is shorter than a prescribed distance D3. The above-described prescribed distances D1 to D3 are set in advance by an administrator. For example, the value of the prescribed distance D1 is set to a value larger than the value of the prescribed distance D2. Furthermore, the determining unit  170   b  may adjust the value of the prescribed distance D1 according to the movement speed of the hand. For example, the determining unit  170   b  decreases the value of the prescribed distance D1 as the speed of the hand increases. 
         [0064]    The generating unit  170   c  is a processing unit that generates a display screen to be displayed by the remote assistance person terminal  200  on the basis of the determination result of the determining unit  170   b . The generating unit  170   c  outputs information on the generated display screen to the notifying unit  170   d.    
         [0065]      FIG. 5  is diagram (1) illustrating one example of the display screen generated by the generating unit according to the present embodiment 1. As illustrated in  FIG. 5 , a photographed image  20   a  and a three-dimensional model  20   b  are disposed in a display screen  20 . The photographed image  20   a  corresponds to a photographed image by the camera  130  and the three-dimensional model  20   b  corresponds to the three-dimensional model information  160   a.    
         [0066]    The generating unit  170   c  causes displayed content  21  indicating the distance to a work target and displayed content  22  indicating the distance to a dangerous range to be displayed in the photographed image  20   a . The generating unit  170   c  may change the color of the displayed content  22  indicating the distance to a dangerous range according to the distance between a hand of the worker and the dangerous range. For example, the generating unit  170   c  displays red as the color of the displayed content  22  if the distance between the hand of the worker and the dangerous range is shorter than a threshold. The generating unit  170   c  displays yellow if the distance between the hand of the worker and the dangerous range is equal to or longer than the threshold. 
         [0067]    The generating unit  170   c  adjusts the placement positions of the pieces of displayed content  21  and  22  so that the pieces of displayed content  21  and  22  may be kept from overlapping with the position of the hand of the worker. 
         [0068]    The generating unit  170   c  causes pieces of displayed content  23  and  24  indicating dangerous regions, displayed content  25  indicating the position of the hand of the worker, and displayed content  26  indicating the worker to be displayed in the three-dimensional model  20   b . Furthermore, the generating unit  170   c  may display the dangerous ranges with a given color. Moreover, the generating unit  170   c  may explicitly indicate the position of the three-dimensional model  20   b  corresponding to the photographed image  20   a . In the example illustrated in  FIG. 5 , it is indicated that the photographed image  20   a  is a photographed image corresponding to a region  27  in the three-dimensional model  20   b.    
         [0069]    The display screen generated by the generating unit  170   c  is not limited to the display screen  20  illustrated in  FIG. 5 .  FIG. 6  is diagram (2) illustrating one example of the display screen generated by the generating unit according to the present embodiment 1. For example, the display unit  150  may display only a range corresponding to the photographed image  20   a  in the three-dimensional model  20   b.    
         [0070]    The notifying unit  170   d  is a processing unit that transmits information on the display screen generated by the generating unit  170   c  to the remote assistance person terminal  200 . 
         [0071]    Next, the processing procedure of the image processing device  100  according to the present embodiment 1 will be described.  FIG. 7  is a flowchart illustrating a procedure of the processing of the image processing device according to the present embodiment 1. As illustrated in  FIG. 7 , the acquiring unit  170   a  of the image processing device  100  acquires the three-dimensional model information  160   a , the work range information  160   b , and the dangerous range information  160   c  (step S 101 ). 
         [0072]    The acquiring unit  170   a  acquires a photographed image from the camera  130  (step S 102 ) and acquires three-dimensional distance information from the distance sensor  120  (step S 103 ). The determining unit  170   b  of the image processing device  100  determines the work position of a worker on the basis of the three-dimensional distance information and the three-dimensional model information  160   a  (step S 104 ). 
         [0073]    The determining unit  170   b  determines the distances between a work range and a dangerous range and a hand of the worker (step S 105 ). The generating unit  170   c  of the image processing device  100  generates information on a display screen (step S 106 ) and the notifying unit  170   d  of the image processing device  100  notifies the remote assistance person terminal  200  of the information on the display screen (step S 107 ). 
         [0074]    If the processing is ended (step S 108 , Yes), the image processing device  100  ends the processing. On the other hand, if the processing is not ended (step S 108 , No), the image processing device  100  makes transition to the step S 102 . 
         [0075]    Next, one example of the processing of generating the information on the display screen, represented in the step S 106  in  FIG. 7 , will be described.  FIG. 8  is a flowchart illustrating a procedure of the processing of generating the information on the display screen. As illustrated in  FIG. 8 , the determining unit  170   b  of the image processing device  100  determines a work target from the distance between the work range and the hand of the worker (step S 201 ). 
         [0076]    The determining unit  170   b  determines a dangerous range close to the worker from the distance between the dangerous range and the hand of the worker (step S 202 ). The generating unit  170   c  of the image processing device  100  decides displayed content on the basis of the work target and the dangerous range (step S 203 ). 
         [0077]    The generating unit  170   c  decides the display position of the displayed content (step S 204 ). The generating unit  170   c  generates the information on the display screen by disposing the displayed content at the display position in the display screen (step S 205 ). 
         [0078]    Next, one example of the processing of determining the work target, represented in the step S 201  in  FIG. 8 , will be described.  FIG. 9  is a flowchart illustrating a procedure of the processing of determining the work target. As illustrated in  FIG. 9 , the determining unit  170   b  of the image processing device  100  determines whether or not a hand exists in a photographed image (step S 301 ). If a hand does not exist in the photographed image (step S 301 , No), the determining unit  170   b  makes transition to a step S 304 . 
         [0079]    On the other hand, if a hand exists in the photographed image (step S 301 , Yes), the determining unit  170   b  extracts the work target closest to the straight line extending along the traveling direction of the hand (step S 302 ). The determining unit  170   b  determines whether or not the distance between the hand and the work target is equal to or shorter than a given distance (step S 303 ). 
         [0080]    If the distance between the hand and the work target is equal to or shorter than the given distance (step S 303 , Yes), the determining unit  170   b  ends the processing of determining the work target. For example, if the step S 303  results in Yes, the work target extracted in the step S 302  is the work target of the determination result. 
         [0081]    On the other hand, if the distance between the hand and the work target is not equal to or shorter than the given distance (step S 303 , No), the determining unit  170   b  extracts the work target closest to the center of the field of view (step S 304 ). For example, if the step S 303  results in No, the work target extracted in the step S 304  is the work target of the determination result. 
         [0082]    Next, one example of the processing of determining the dangerous range, represented in the step S 202  in  FIG. 8 , will be described.  FIG. 10  is a flowchart illustrating a procedure of the processing of determining the dangerous range. As illustrated in  FIG. 10 , the determining unit  170   b  of the image processing device  100  determines whether or not a hand exists in a photographed image (step S 401 ). If a hand does not exist in the photographed image (step S 401 , No), the determining unit  170   b  makes transition to a step S 404 . 
         [0083]    If a hand exists in the photographed image (step S 401 , Yes), the determining unit  170   b  extracts a dangerous range within a prescribed distance from the straight line extending along the traveling direction of the hand (step S 402 ). 
         [0084]    The determining unit  170   b  extracts a dangerous range within a prescribed distance from the hand (step S 403 ). The determining unit  170   b  extracts a dangerous range within a prescribed distance from the photographing position (S 404 ). 
         [0085]    Next, effects of the image processing device  100  according to the present embodiment 1 will be described. When transmitting information on a photographed image to the remote assistance person terminal  200 , the image processing device  100  determines the work position of the worker from the three-dimensional model information  160   a  and three-dimensional distance information. The image processing device  100  generates a display screen including information on the direction and distance of a dangerous region or a work region from the work position of the worker and notifies the remote assistance person terminal  200  of the display screen in conjunction with the information on the photographed image. Thus, not only the position of the worker but also the distance to the dangerous range or the work range that has an influence on the worker can be displayed and the remote assistance person can easily grasp the situation of the worker at the work site. 
       Embodiment 2 
       [0086]    The processing of the image processing device  100  represented in embodiment 1 is one example. In the present embodiment 2, another kind of processing executed by the image processing device  100  will be described. 
         [0087]    Another kind of processing (1) of the generating unit  170   c  will be described. In the case of generating a display screen by disposing a photographed image and a three-dimensional model, the generating unit  170   c  adjusts the placement position of the photographed image according to the position of a worker on the three-dimensional model. For example, the generating unit  170   c  adjusts the position of the photographed image in such a manner that the position of the worker in the horizontal direction on the three-dimensional model  20   b  overlaps with the position of the photographed image in the horizontal direction. The generating unit  170   c  may use the position of the point of view of the worker instead of the position of the worker. 
         [0088]      FIG. 11  is a diagram for explaining another kind of processing (1) of the generating unit. In  FIG. 11 , display screens  20 A and  20 B are illustrated as one example. A description will be made about the display screen  20 A. In the display screen  20 A, the worker  26  is located on the left side of the three-dimensional model  20   b  and the photographed image  20   a  corresponds to a region  27   a . In this case, the generating unit  170   c  disposes the photographed image  20   a  on the left side of the display screen  20 A so that the position of the photographed image  20   a  in the horizontal direction may overlap with the position of the worker  26  in the horizontal direction. 
         [0089]    A description will be made about the display screen  20 B. In the display screen  20 B, the worker  26  is located on the right side of the three-dimensional model  20   b  and the photographed image  20   a  corresponds to a region  27   b . In this case, the generating unit  170   c  disposes the photographed image  20   a  on the right side of the display screen  20 B so that the position of the photographed image  20   a  in the horizontal direction may overlap with the position of the worker  26  in the horizontal direction. 
         [0090]    Here, an example of the processing of adjusting the placement position of the photographed image by the generating unit  170   c  will be described.  FIG. 12  is a diagram for explaining the processing of adjusting the placement position of the photographed image. As illustrated in  FIG. 12 , the generating unit  170   c  divides the three-dimensional model  20   b  into a left region  1 A, a central region  1 B, and a right region  1 C. If the position of the worker is included in the left region  1 A, the generating unit  170   c  disposes the photographed image  20   a  on the left side as illustrated in the display screen  20 A in  FIG. 11 . 
         [0091]    If the position of the worker is included in the central region  1 B, the generating unit  170   c  does not adjust the position of the photographed image  20   a  and keeps the unchanged placement position. If the position of the worker is included in the right region  1 C, the generating unit  170   c  disposes the photographed image  20   a  on the right side as illustrated in the display screen  20 B in  FIG. 11 . 
         [0092]      FIG. 13  is a flowchart illustrating a processing procedure of another kind of processing (1) of the generating unit. With  FIG. 13 , a description will be made by using the position of the worker as one example. However, the position of the point of view of the worker may be used instead of the position of the worker. 
         [0093]    As illustrated in  FIG. 13 , the generating unit  170   c  determines whether or not the position of the worker is in the central region of a three-dimensional model (step S 501 ). If the position of the worker is in the central region of the three-dimensional model (step S 501 , Yes), the generating unit  170   c  ends the processing. 
         [0094]    If the position of the worker is not in the central region of the three-dimensional model (step S 501 , No), the generating unit  170   c  determines whether or not the position of the worker is in the left region of the three-dimensional model (step S 502 ). If the position of the worker is in the left region of the three-dimensional model (step S 502 , Yes), the generating unit  170   c  disposes a photographed image on the left side of a display screen (step S 503 ). 
         [0095]    On the other hand, if the position of the worker is in the right region of the three-dimensional model (step S 502 , No), the generating unit  170   c  disposes the photographed image on the right side of the display screen (step S 504 ). 
         [0096]    The generation of the display screen in the above-described manner by the generating unit  170   c  can suppress the amount of movement of the line of sight when a remote assistance person views the worker on the three-dimensional model  20   b  and the photographed image  20   a . Furthermore, setting the central region  1 B in the three-dimensional model  20   b  can inhibit the placement position of the photographed image  20   a  from being frequently changed. 
         [0097]    Next, another kind of processing (2) of the generating unit  170   c  will be described. In the case of disposing a three-dimensional model on a display screen, the generating unit  170   c  may display only a peripheral range based on the position of the worker instead of displaying the whole range of the three-dimensional model. In the following description, the region of the peripheral range based on the position of the worker will be represented as the peripheral region. 
         [0098]    The generating unit  170   c  sets a peripheral region having a size set in advance in a three-dimensional model and determines whether or not a dangerous range or a work range is included in the peripheral region. If a dangerous range or a work range is not included in the peripheral region, the generating unit  170   c  reduces the peripheral region to the minimum display range defined in advance. The generating unit  170   c  clips the three-dimensional model by the minimum display range and disposes the clipped three-dimensional model in a display screen. 
         [0099]    On the other hand, if a dangerous range or a work range is included in the peripheral region, the generating unit  170   c  reduces and adjusts the range of the peripheral region within a range in which the dangerous range or the work range is not erased. The generating unit  170   c  clips the three-dimensional model by the adjusted peripheral region and disposes the clipped three-dimensional model in the display screen. 
         [0100]      FIG. 14  is a diagram for explaining another kind of processing (2) of the generating unit. In  FIG. 14 , the generating unit  170   c  sets a peripheral region based on the position of the worker  26  in the three-dimensional model  20   b  and defines the range of the peripheral region as a range  2 A. The dangerous regions  62   a  and  62   b  and the work region  61  are included in the range  2 A. Therefore, the generating unit  170   c  reduces and adjusts the range of the peripheral region within a range in which the dangerous ranges  62   a  and  62   b  and the work range  61  are not erased. The range of the peripheral region after the adjustment is a range  2 C. The generating unit  170   c  clips the peripheral region of the range  2 C in the three-dimensional model  20   b  and disposes the clipped peripheral region in the display screen. 
         [0101]    If a dangerous region and a work range are not included in the range  2 A, the generating unit  170   c  reduces the range of the peripheral region to a minimum display range  2 B and clips the three-dimensional model by the minimum display range  2 B to dispose the clipped three-dimensional model in the display screen. 
         [0102]      FIG. 15  is diagram (1) illustrating one example of a display screen generated by the generating unit according to the present embodiment 2. In  FIG. 15 , display screens  20 C and  20 D are illustrated as one example. In the display screen  20 C, the photographed image  20   a  and the three-dimensional model  20   b  are disposed. Furthermore, the range of a peripheral region based on the position of the worker  26  is clipped as the three-dimensional model  20   b . The work target  61  and the dangerous ranges  62   a  and  62   b  are included in the three-dimensional model  20   b  of the display screen  20 C. 
         [0103]    In the display screen  20 D, the photographed image  20   a  and the three-dimensional model  20   b  are disposed. Furthermore, the range of a peripheral region based on the position of the worker  26  is clipped as the three-dimensional model  20   b . The dangerous range  62   b  is included in the three-dimensional model  20   b  of the display screen  20 D. 
         [0104]      FIG. 16  is a flowchart illustrating a processing procedure of another kind of processing (2) of the generating unit. As illustrated in  FIG. 16 , the generating unit  170   c  sets the range of the peripheral region to an initial value (step S 601 ). The generating unit  170   c  determines whether or not a work range or a dangerous range is included in the range of the peripheral region (step S 602 ). 
         [0105]    If a work range or a dangerous range is not included in the range of the peripheral region (step S 602 , No), the generating unit  170   c  reduces and adjusts the range of the peripheral region to the minimum display range (step S 603 ). 
         [0106]    On the other hand, if a work range or a dangerous range is included in the range of the peripheral region (step S 602 , Yes), the generating unit  170   c  reduces and adjusts the range of the peripheral region within a range in which the dangerous range or the work range is not erased (step S 604 ). 
         [0107]    In the above-described manner, the generating unit  170   c  clips the range of the peripheral region from the three-dimensional model and displays the clipped range. This can narrow down the range to which a remote assistance person may pay attention and can alleviate the burden of the remote assistance person. 
         [0108]    Incidentally, the generating unit  170   c  may adjust the range of the peripheral region in any manner. For example, the generating unit  170   c  employs the minimum display range as the initial value of the peripheral region and determines whether or not the boundary of the peripheral region overlaps with a dangerous region or a work region. If the boundary of the peripheral region does not overlap with a dangerous region or a work region, the generating unit  170   c  employs the minimum display range as the peripheral region. On the other hand, if the boundary of the peripheral region overlaps with a dangerous region or a work region, the generating unit  170   c  extends and adjusts the range of the peripheral region until the peripheral region contains the dangerous region or the work region. 
         [0109]    Next, another kind of processing (3) of the generating unit  170   c  will be described. If the distance between a hand of the worker and a work target becomes shorter than a threshold, the generating unit  170   c  may generate a virtual image of the case in which the work target is viewed from a different point of view from the point of view of the worker by utilizing the three-dimensional model information  160   a , and display the image on a display screen. In the following description, the virtual image of the case in which a work target is viewed from a different point of view from the point of view of the worker will be represented as the another-point-of-view image. 
         [0110]      FIG. 17  is a diagram for explaining another kind of processing (3) of the generating unit. The generating unit  170   c  virtually rotates the position of the point of view of the distance sensor  120  as the distance between the position of a hand  26   a  of the worker and a work target  30  becomes shorter, and the generating unit  170   c  generates an another-point-of-view image of the case in which the work target  30  is viewed from the virtual position of the point of view resulting from the rotation. 
         [0111]    For example, if the distance between the hand  26   a  of the worker and the work target  30  becomes shorter than a first threshold, the generating unit  170   c  rotates the camera by θ1, with the straight line linking the hand  26   a  of the worker and the work target  30  being the axis, and generates an another-point-of-view image of the case in which the work target  30  is viewed from a position  120   b  of the point of view. Furthermore, if the distance between the hand  26   a  of the worker and the work target  30  becomes shorter than a second threshold, the generating unit  170   c  further rotates the camera by θ2 around the axis and generates an another-point-of-view image of the case in which the work target  30  is viewed from a position  120   c  of the point of view. 
         [0112]    The generating unit  170   c  may determine the angle of the rotation by using not only the distance between a hand of the user and a work target but also the distance between the hand of the worker and a dangerous range. For example, the generating unit  170   c  determines the angle of the rotation by using a rotation amount table. 
         [0113]      FIG. 18  is a diagram illustrating one example of a data structure of the rotation amount table. As illustrated in  FIG. 18 , this rotation amount table associates a distance d1, a distance d2, and an angle θ. The distance d1 indicates the distance between a hand of the worker and a dangerous range. The distance d2 indicates the distance between the hand of the worker and a work target. The angle θ indicates the rotation angle around the axis linking the hand of the worker and the work target. 
         [0114]    For example, the generating unit  170   c  determines the angle of the rotation on the basis of a processing procedure illustrated in  FIG. 19  and the rotation amount table.  FIG. 19  is a flowchart illustrating a processing procedure of another kind of processing (3) of the generating unit. As illustrated in  FIG. 19 , the generating unit  170   c  determines whether or not the distance d1 between the dangerous range and the hand is shorter than a given threshold (step S 701 ). If the distance d1 between the dangerous range and the hand is shorter than the given threshold (step S 701 , Yes), the generating unit  170   c  determines the rotation angle of the point of view on the basis of the rotation amount table and the distance d1 (step S 702 ). For example, if the distance d1 is equal to or shorter than 1 cm in the step S 702 , the rotation angle is “90 deg.” 
         [0115]    On the other hand, if the distance d1 between the dangerous range and the hand is not shorter than the given threshold (step S 701 , No), the generating unit  170   c  determines whether or not the distance d2 between the work range and the hand is shorter than a given threshold (step S 703 ). If the distance d2 between the work range and the hand is shorter than the given threshold (step S 703 , Yes), the generating unit  170   c  determines the rotation angle of the point of view on the basis of the rotation amount table and the distance d2 (step S 704 ). For example, if the distance d2 is 30 cm in the step S 704 , the rotation angle is “45 deg.” 
         [0116]    On the other hand, if the distance d2 between the work range and the hand is not shorter than the given threshold (step S 703 , No), the generating unit  170   c  sets the rotation amount of the point of view to 0 (step S 705 ). 
         [0117]    The generating unit  170   c  may generate the another-point-of-view image in any manner. For example, the generating unit  170   c  converts three-dimensional distance information from the distance sensor  120  to three-dimensional distance information of the virtual position of the point of view based on the rotation angle decided by the processing based on  FIG. 19 . Then, the generating unit  170   c  generates an another-point-of-view image on the basis of the three-dimensional distance information of the virtual position of the point of view and the three-dimensional model information  160   a . If the rotation angle around the axis is equal to or larger than a threshold, the generating unit  170   c  may synthesize the three-dimensional model information  160   a  and an image of the hand to carry out conversion to an image as viewed from the virtual position of the point of view. 
         [0118]      FIG. 20  is diagram (2) illustrating one example of the display screen generated by the generating unit according to the present embodiment 2. In  FIG. 20 , display screens  20 E and  20 F are illustrated as one example. In the display screen  20 E, the photographed image  20   a , the three-dimensional model  20   b , and an another-point-of-view image  20   c  are disposed. For example, the another-point-of-view image  20   c  of the display screen  20 E is an another-point-of-view image when the point of view is rotated by θ1 around the axis. 
         [0119]    In the display screen  20 F, the photographed image  20   a , the three-dimensional model  20   b , and the another-point-of-view image  20   c  are disposed. For example, the another-point-of-view image  20   c  of the display screen  20 F is an another-point-of-view image when the point of view is rotated by θ1+θ2 around the axis. 
         [0120]    Due to the execution of the above-described processing, even when it is difficult to grasp a sense of distance from a photographed image by the camera  130 , the remote assistance person can easily grasp the distance between the worker and the dangerous range or the distance between the worker and the work range through generation of an another-point-of-view image and notification of the another-point-of-view image to the remote assistance person. 
         [0121]    Incidentally, the rotation axis of the point of view may be other than the straight line linking the hand  26   a  of the worker and the work target  30  like that illustrated in  FIG. 17 .  FIG. 21  is a diagram for explaining one example of another rotation axis. In  FIG. 21 , the straight line linking a hand and a work target is defined as a straight line  31  and the straight line corresponding to the line of sight of the camera  130  is defined as a straight line  32 . A tangent plane to the work target  30  at the intersection of the straight line  31  and the work target  30  is defined as a plane  40 . A straight line that intersects the straight line  32  at a right angle on the plane  40  is defined as a straight line  33 . Here, a straight line  34  that intersects the straight line  32  and the straight line  33  at a right angle on the plane  40  may be employed as the rotation axis. The plane  40  may be not a tangent plane but a plane that intersects the straight line  31  at a right angle. 
         [0122]    Furthermore, although the generating unit  170   c  determines the rotation angle on the basis of the rotation amount table illustrated in  FIG. 18 , the configuration is not limited thereto. For example, the generating unit  170   c  may pay attention to the point at which the straight line linking a hand and a work target intersects the work target and determine an angle with which the point to which the attention is paid is not hidden by the hand as the rotation angle. Furthermore, if the distance between a hand and a work target becomes shorter than a threshold, the generating unit  170   c  may set the rotation angle to 90 degrees across the board. 
         [0123]    Next, a description will be made about one example of a computer that executes an image processing program that implements functions similar to those of the image processing device  100  represented in the above-described embodiments 1 and 2.  FIG. 22  is a diagram illustrating one example of the computer that executes the image processing program. 
         [0124]    As illustrated in  FIG. 22 , a computer  300  includes a CPU  301  that executes various kinds of arithmetic processing, an input device  302  that accepts input of data from a user, and a display  303 . Furthermore, the computer  300  includes a reading device  304  that reads a program and so forth from a storage medium, an interface device  305  that gives and receives data to and from another computer via a network, a camera  306 , and a distance sensor  307 . Moreover, the computer  300  includes a RAM  308  that temporarily stores various kinds of information and a hard disk device  309 . In addition, the respective devices  301  to  309  are coupled to a bus  310 . 
         [0125]    The hard disk device  309  includes an acquisition program  309   a , a determination program  309   b , and a generation program  309   c . The CPU  301  reads out the acquisition program  309   a , the determination program  309   b , and the generation program  309   c  and loads the programs into the RAM  308 . 
         [0126]    The acquisition program  309   a  functions as an acquisition process  308   a . The determination program  309   b  functions as a determination process  308   b . The generation program  309   c  functions as a generation process  308   c.    
         [0127]    Processing of the acquisition process  308   a  corresponds to the processing of the acquiring unit  170   a . Processing of the determination process  308   b  corresponds to the processing of the determining unit  170   b . Processing of the generation process  308   c  corresponds to the processing of the generating unit  170   c.    
         [0128]    The acquisition program  309   a , the determination program  309   b , and the generation program  309   c  do not necessarily need to be stored in the hard disk device  309  from the beginning. For example, the respective programs are stored in a “portable physical medium” such as flexible disc (FD), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), magneto-optical disc, or integrated circuit (IC) card inserted into the computer  300 . Then, the computer  300  may read out and execute the respective programs  309   a  to  309   c.    
         [0129]    Incidentally, in the above-described embodiments 1 and 2, the case in which the image processing device  100  includes the acquiring unit  170   a , the determining unit  170   b , the generating unit  170   c , and the notifying unit  170   d  is described. However, the configuration is not limited thereto. For example, the remote assistance person terminal  200  may execute part of the processing of the image processing device  100 , and the image processing device  100  and the remote assistance person terminal  200  may cooperate to generate the above-described display screen and display the display screen on the remote assistance person terminal  200 . For example, the image processing device  100  may include the acquiring unit  170   a  and the remote assistance person terminal  200  may include the determining unit  170   b  and the generating unit  170   c . In this case, the image processing device  100  notifies the remote assistance person terminal  200  of information acquired from the distance sensor  120  and the camera  130  and the remote assistance person terminal  200  generates the display screen by using the notified information. 
         [0130]    All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.