Danger presentation device, danger presentation system, danger presentation method and program

A danger presentation device includes a worker position acquisition unit configured to acquire a worker position which is a position of a worker; a worker view range determination unit configured to determine a view range of the worker depending on the worker position acquired by the worker position acquisition unit; a position/posture determination unit configured to determine a position/posture which contains at least one of a position of a robot and a posture of the robot at a specific time in which at least a part of the robot which operates in accordance with a motion planning is included in the view range; and an image generation unit configured to generate image data for illustrating the position/posture determined by the position/posture determination unit.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to a danger presentation device which generates data for presenting a possible danger caused by a robot which operates in accordance with a motion planning.

(2) Description of the Related Art

Conventionally, robots have been used in production sites and the like. When a robot is used, a worker may come into contact with the robot and get injured. Thus, a safety measure may be taken by surrounding the robot with a safety fence. However, it may be difficult for a worker do some work when a safety fence is installed between the robot and the worker. In this case, for example, it is difficult for the worker and the robot to carry a table by holding both ends of the table, respectively, in a cooperative manner. In the above case, it is also difficult for the robot to hand over a part, a tool, or the like to a worker who does assembly work.

In such a work situation, it is not practical to install a safety fence between a worker and a robot that works cooperatively with the worker, or a robot that works near the worker. Thus, when such a robot is used, a user of the robot needs to pay more attention to safety than is normally needed.

Japanese Unexamined Patent Application Publication No. 2009-123045 discloses a movable robot which visually displays a dangerous area around the movable robot. The dangerous area around the movable robot described in Japanese Unexamined Patent Application Publication No. 2009-123045 varies every moment depending on the direction and the speed of the movement of the robot. The movable robot then determines a dangerous area based on a motion planning such as the movement speed of the robot.

FIG. 38is a diagram illustrating a dangerous area when the movable robot is stopped. A movable robot9001illustrated inFIG. 38includes a front-side projector9101, a rear-side projector9102, a right-side projector9103, and a left-side projector9104. A dangerous area9060illustrated inFIG. 38is the dangerous area when the movable robot9001is stopped. The dangerous area9060is projected to the four projectors9101,9102,9103, and9104, and is presented to a worker.

FIG. 39is a diagram illustrating the dangerous area9060when the movable robot9001illustrated inFIG. 38is in motion. The dangerous area9060illustrated inFIG. 39is the dangerous area when the movable robot9001moves to the left side at a speed V1. The dangerous area9060has a left side area of the movable robot9001in the moving direction, which is larger than the left side area of the movable robot9001when stopped.

FIG. 40is a diagram illustrating the dangerous area9060when the movable robot9001illustrated inFIG. 38moves faster than the speed V1. The dangerous area9060becomes even greater than the dangerous area when moving at the speed V1.

FIG. 41is a diagram illustrating the dangerous area9060when the movable robot9001illustrated inFIG. 38operates a left arm9012. The dangerous area9060has an expanded area near the left arm9012.

However, even when a robot presents a dangerous area around itself, the dangerous area may not be appropriately recognized by a worker. For example, when a robot presents a dangerous area at a place which cannot be viewed by a worker, the dangerous area cannot be appropriately recognized by the worker. Thus the robot and the worker may collide with each other, thereby preventing smooth operation of the robot.

In addition, it is difficult for the worker to precisely recognize a dangerous situation with the dangerous area shown around the robot. For example, it is difficult for the worker to recognize a danger related situation such as in which direction movement would avoid the dangerous situation, in which direction movement would be more dangerous, which area is less dangerous, or how dangerous the current state is. Hereinafter, specific examples are described.

FIG. 42is a diagram illustrating a problem in a conventional technology. A worker9061in the dangerous area9060cannot directly view either of a right arm9011and a left arm9012of the movable robot9001from the position of the worker9061illustrated inFIG. 42. Thus, even though the worker9061recognizes that he is in the dangerous area9060, he cannot tell which one of the right arm9011and the left arm9012is moving toward him. Therefore, the worker9061cannot tell in which direction an arm of the movable robot9001is moving toward him, thus the worker9061cannot determine in which direction he should move to avoid the dangerous situation.

SUMMARY OF THE INVENTION

Now, it is an object of the present invention to provide a danger presentation device which generates data for appropriately presenting a dangerous situation accompanied by the motion of a robot.

In order to solve the aforementioned problem, a danger presentation device in the present invention includes a worker position acquisition unit configured to acquire a worker position which is a position of a worker; a worker view range determination unit configured to determine a view range of the worker depending on the worker position acquired by the worker position acquisition unit; a position/posture determination unit configured to identify a specific time in which at least a part of a robot which operates according to a motion planning is included in the view range, and to determine a position/posture which contains at least one of a position of the robot at the specific time and a posture of the robot at the specific time; and an image generation unit configured to generate image data for illustrating the position/posture determined by the position/posture determination unit.

Accordingly, the danger presentation device can generate data for presenting the posture of the robot at a time when the robot appears in the worker's view. Therefore, the danger presentation device can generate data for appropriately presenting a dangerous situation accompanied by the motion of the robot.

In addition, the danger presentation device may further include a dangerous area generation unit configured to generate a dangerous area according to the motion planning, the dangerous area being covered by movement of the robot, wherein the position/posture determination unit is configured to determine the position/posture at the specific time in a time period including a predicted time of a collision between the worker and the robot, based on the worker position and the motion planning, and the image generation unit is configured to generate the image data for illustrating the dangerous area and the position/posture determined by the position/posture determination unit.

Accordingly, the danger presentation device can generate data for illustrating the position and/or posture of the robot around the time when a collision is expected. That is to say, the danger presentation device can generate data for appropriately presenting a dangerous situation around the robot.

In addition, the position/posture determination unit may be configured to determine the position/posture at the specific time which is a time before the predicted time of the collision.

Accordingly, the danger presentation device can generate data for illustrating the position and/or posture of the robot at a time before the time when a collision is expected.

In addition, the position/posture determination unit may be configured to determine the position/posture at the specific time in which a distance between the worker position and a moving position of the robot is greater than or equal to a predetermined distance, the specific time being before the predicted time of the collision and closest to the predicted time of the collision.

Accordingly, a sufficient distance is provided between the position/posture of the robot and the worker. Consequently, the danger presentation device can generate data for presenting the position and posture of the robot more appropriately.

In addition, the position/posture determination unit may be configured to predict a position of the worker after an elapse of a predetermined time, as the worker position, based on the worker position acquired by the worker position acquisition unit, to predict the collision between the worker and the robot based on the predicted worker position and the motion planning, and to determine the position/posture at the specific time in a time period including the predicted time of the collision.

Accordingly, a collision between the robot and the worker is predicted more precisely. Therefore, data with higher precision is generated.

In addition, the position/posture determination unit may be configured to determine the position/posture at the specific time which is the predicted time of the collision.

Accordingly, the position and posture of the robot at the collision due to a movement of the worker is determined. Consequently, the situation of the collision is expressed more precisely.

In addition, the position/posture determination unit may be configured to determine the position/posture at the specific time in which a motion range covered by movement of the robot according to the motion planning during a predetermined time has an overlapping intersection with the view range with the overlapping intersection greater than or equal to a predetermined size, the specific time being after a predicted time of a first collision between the worker and the robot, and the specific time being closest to the predicted time of the first collision.

Accordingly, the robot has an overlapping intersection with the worker's view at a specific time with the overlapping intersection being greater than or equal to a predetermined size. Consequently, the danger presentation device can generate more appropriate data.

In addition, in the case where a collision between the worker and the robot is predicted, and there is no time applicable to the specific time, the position/posture determination unit may be configured to cause a warning device to output a warning.

Accordingly, in the case where the robot is outside the worker's view in spite of a possibility of a collision, the danger presentation device can inform the worker of a danger by presenting a warning such as a warning sound or a warning display.

In addition, in the case where the specific time is at or after a time at which the robot passes through the worker position which is acquired by the worker position acquisition unit, the position/posture determination unit may be configured to cause a warning device to output a warning.

Accordingly, when there is a possibility that the worker may not appropriately recognize a dangerous situation due to the timing of appearing of the robot in the worker's view, the danger presentation device can inform the worker of a danger by presenting a warning such as a warning sound or a warning display.

In addition, in the case where the specific time is before a time at which the robot passes through the worker position which is acquired by the worker position acquisition unit, the image generation unit may be configured to generate the image data.

Accordingly, when there is a possibility that the worker may not appropriately recognize a dangerous situation due to the timing of appearing of the robot in the worker's view, the danger presentation device can inform the worker of a danger by presenting a warning such as a warning sound or a warning display. When there is little possibility that the worker erroneously recognizes a dangerous situation, the danger presentation device generates data for illustrating the position and posture of the robot.

In addition, the danger presentation device may further include a worker orientation acquisition unit configured to acquire a worker orientation which is an orientation of a worker; and an obstruction information acquisition unit configured to acquire obstruction information which is information of an obstruction that obstructs the view range, wherein the worker view range determination unit is configured to determine the view range depending on the worker position acquired by the worker position acquisition unit, the worker orientation acquired by the worker orientation acquisition unit, and the obstruction information acquired by the obstruction information acquisition unit.

Accordingly, a detailed view range is determined based on the position, orientation of the worker, and information of an object blocking the view range. Therefore, more appropriate data is generated.

In addition, the image generation unit may be configured to generate the image data for illustrating the position/posture and a reaching time between a current time and the specific time.

Accordingly, the danger presentation device can generate data which indicates when the robot approaches the worker. That is to say, the danger presentation device can generate data which indicates a degree of danger.

In addition, the image generation unit may be configured to determine a display format for the position/posture depending on a reaching time between a current time and the specific time, and to generate the image data for illustrating the position/posture in the determined display format.

Accordingly, the danger presentation device can generate data according to a degree of danger.

In addition, the image generation unit may be configured to determine the display format so that a brightness when the reaching time is a first time is higher than a brightness when the reaching time is a second time which is longer than the first time, and to generate the image data for illustrating the position/posture in the determined display format.

Accordingly, the danger presentation device generates data which indicates the robot with a highlight image when a degree of danger is high. Therefore, the danger presentation device can indicate a degree of danger.

The danger presentation system according to the present invention may also include the danger presentation device.

Accordingly, a danger presentation system including the danger presentation device is achieved.

In addition, the danger presentation system may further include the robot which operates in accordance with the motion planning

Accordingly, the danger presentation system can generate data for appropriately presenting a dangerous situation accompanied by the motion of the robot.

In addition, the danger presentation system may further include a display device configured to as an image present the image data generated by the image generation unit.

Accordingly, the display device can present data as an image, the data being generated by the danger presentation device. Consequently, a danger situation is presented appropriately.

In addition, the danger presentation system may further include a warning sound output device configured to output a warning sound.

Accordingly, the danger presentation system can output a warning sound for informing the worker of a danger when the position and/or posture of the robot cannot be presented in the view range of the worker.

In addition, a danger presentation method in the present invention may include acquiring a worker position which is a position of a worker; determining a view range of the worker depending on the worker position acquired in the acquiring; identifying a specific time in which at least a part of a robot which operates according to a motion planning is included in the view range and determining a position/posture which contains at least one of a position of the robot at the specific time and a posture of the robot at the specific time; and generating image data for illustrating the position/posture determined in the determining of the position/posture.

Accordingly, a danger presentation method is achieved.

In addition, a program in the present invention may cause a computer to execute: acquiring a worker position which is a position of a worker; determining a view range of the worker depending on the worker position acquired in the acquiring; identifying a specific time in which at least a part of a robot which operates according to a motion planning is included in the view range, and determining a position/posture which contains at least one of a position of the robot at the specific time and a posture of the robot at the specific time; and generating image data for illustrating the position/posture determined in the determining of the position/posture.

Accordingly, the danger presentation method is achieved as a program.

The present invention can generate data for appropriately presenting a dangerous situation accompanied by the motion of a robot.

DETAILED DESCRIPTION OF THE INVENTION

A danger presentation system in Embodiment 1 presents a dangerous area to a worker so that the worker in the dangerous area can recognize the direction in which a robot is approaching the worker.

The danger presentation system in Embodiment 1 is for a rotary robot with one degree of freedom. A robot with one degree of freedom is an example of the robots to which the present invention is directed. The robots to which the present invention is directed include a robot with multiple degrees of freedom, a linear movement robot, a mobile robot, and a robot in combination thereof.

FIG. 1is a conceptual diagram of the danger presentation system in Embodiment 1. The danger presentation system1000illustrated inFIG. 1includes a danger presentation device1100, a robot1200, and a display device1300. The danger presentation system1000projects a dangerous area31to the floor around a robot1200. The danger presentation system1000projects the posture of the robot1200at a specific time. The danger presentation system1000may project a point illustrating the position of the robot1200at a specific time.

Accordingly, the danger presentation system1000presents a worker51a direction in which the robot1200is approaching. Hereinafter, the danger presentation system1000which achieves the above-described presentation is described.

FIG. 2is a general diagram of the danger presentation system1000illustrated inFIG. 1. The danger presentation device1100includes a dangerous area generation unit1101, a worker position acquisition unit1102, a position/posture determination unit1103, and an image generation unit1104. The robot1200includes a motion planning unit1201, a motion control unit1202, and a movable unit1203. Hereinafter, each component is described.

The movable unit1203is a unit which is physically movable in the robot1200. The movable unit1203moves in a space in which a worker may come into contact with the movable unit1203. For example, when the robot1200has an arm for moving an object, the arm is the movable unit1203. When the main body of the robot1200moves, the main body is the movable unit1203.

FIG. 3is an outside view of the robot1200illustrated inFIG. 1. The robot1200includes a movable unit1203, and a base unit2001which is not physically movable. The movable unit1203moves when carrying an object2002. The motion planning unit1201and the motion control unit1202which are not illustrated inFIG. 3are physically contained in the movable unit1203or the base unit2001. The motion planning unit1201and the motion control unit1202may be contained in either one of the movable unit1203and the base unit2001.

The motion planning unit1201outputs a motion planning which indicates how the movable unit1203moves. For example, the motion planning is a set of angles each indicating a rotation angle for which the movable unit1203is rotated, or a set of rotation angles which are ordered. In Embodiment 1, the motion planning is {R—0, R—1, R—2, . . . , R_n} indicating a set of rotation angles of the movable unit1203at time 0, time 1, time 2, . . . , time n, respectively where time 0 corresponds to an initial state.

FIG. 4is a diagram illustrating a rotation angle. The XY-plane illustrated inFIG. 4is the horizontal plane containing the robot1200which is horizontally placed. A point A(AX, AY) indicates the center position of rotation of the movable unit1203. A point B (X—0, Y—0) indicates the position of one end of the movable unit1203in an initial state. Here, the position of the one end of the movable unit1203is the position of the end of the movable unit1203, which is opposite to the center position of rotation of the movable unit1203. A point C (X_n, Y_n) indicates the position after the end of the movable unit1203is rotated for 270 degrees.

Here, the rotation angle is the angle formed by a first directed line segment AB and a second directed line segment AC, the first directed line segment AB having a starting point A(AX, AY) and an end point B(X—0, Y—0), and the second directed line segment AC having a starting point A(AX, AY) and an end point C(X_n, Y_n). The dangerous area31is the area covered by the first directed line segment AB when the first directed line segment AB is moved to the position of the second directed line segment AC based on the motion planning of the robot1200. In the case where the worker51is in the dangerous area31, there is a risk that the worker51could collide with the robot1200.

A rotation angle with a positive value indicates a motion planning which rotates the movable unit1203counter clockwise. A rotation angle with a negative value indicates a motion planning which rotates the movable unit1203clockwise.

FIG. 5is a diagram illustrating an example of relationship between time and rotation angle. The rotation angle at time 1 is the rotation angle R—1 illustrated inFIG. 5, and is 10 degrees from the initial position of the robot1200. The rotation angle at time 2 is the rotation angle R—2 illustrated inFIG. 5, and is 20 degrees from the initial position of the robot1200. The rotation angle at time 27 is the rotation angle R—27 illustrated inFIG. 5, and is 270 degrees from the initial position of the robot1200.

In the following description, the rotation angles contained in a set of ordered rotation angles are referred to as the 0th rotation angle, the 1st rotation angle, the 2nd rotation angle, . . . , the n-th rotation angle starting form the first angle. Here, the ordinal number starts with 0th rather than the 1st. Because the time starts with 0, so does the rotation angle. Accordingly the rotation angle at time 0 is R—0, thus the indexes of angle and time correspond to each other.

For example, time 0, time 1, . . . , time 27 with a time interval of 1 second are given, and the movable unit1203rotates for 270 degrees counter clockwise at 10 degrees per second, the motion planning is a set of rotation angles, {0, 10, 20, . . . , 270}.

Here, the above set of rotation angles is previously stored in the motion planning unit1201. The motion planning unit1201outputs the rotation angle at a designated time in accordance with a request from the outside. That is to say, when time 0, time 1, time 2, . . . , time 27 are designated, the motion planning unit1201outputs 0, 10, 20, . . . , 270 as the rotation angles. Or when the 0th, the 1st, the 2nd, . . . , the 27th are designated, the motion planning unit1201outputs 0, 10, 20, . . . , 270 as the rotation angles.

The motion planning is expressed by angles with a predetermined time interval in the above, but may be a set of combination of time and rotation angle {(t0, Θ0), (t1, Θ1), (t2, Θ2), . . . (tn, Θn)}.

The motion planning unit1201may plan the movement of the robot1200based on an environmental map information previously held or ambient information acquired by a visual sensor. The motion planning unit1201does not need to directly plan the position of a movement destination of the movable unit1203, but may plan a target movement speed, a target acceleration, and a target angle trajectory of a joint of the movable unit1203.

The motion control unit1202moves the movable unit1203in accordance with the motion planning determined by the motion planning unit1201.

The dangerous area generation unit1101forms the dangerous area31based on the motion planning determined by the motion planning unit1201. Specifically, the dangerous area generation unit1101generates the dangerous area31by determining the area that is covered by the movable unit1203when the movable unit1203is moved in accordance with the motion planning of the motion planning unit1201.

More specifically, the dangerous area31inFIG. 4is described. First, the dangerous area generation unit1101previously holds the point A(AX, AY) as the position of the shaft of the movable unit1203, and the point B(X—0, Y—0) as the position of the movable unit1203at time 0. Next, the dangerous area generation unit1101acquires the last rotation angle “270” held by the motion planning unit1201.

Next, the dangerous area generation unit1101defines the dangerous area31as a set of point (X, Y) on the XY plane that satisfies the following two conditions.

The first condition is that the distance between the point (X, Y) and point A(AX, AY) is shorter than the distance between the point B(X—0, Y—0) and the point A(AX, AY). The second condition is that the angle formed by a first vector and a second vector is greater than or equal to 0 and less than or equal to “270” which is the last rotation angle in the motion planning, the first vector having the starting point A(AX, AY) and the end point B(X—0, Y—0), and the second vector having the starting point A(AX, AY) and the end the point (X, Y).

The dangerous area31determined in this manner is displayed by the display device1300illustrated inFIG. 1. The display device1300is a projector which is installed on the ceiling of a room in which the robot1200is disposed, and projects the dangerous area31and the later-described characteristic posture in a direction to the worker51, i.e., the floor.

The dangerous area generation unit1101determines a sector only by the last rotation angle held by the motion planning unit1201in the above, but may determine, as the dangerous area31, the sector area defined by the point A(AX, AY) and the two positions of the distal end of the movable unit1203corresponding to the k-th and (k−1)th rotation angles held by the motion planning unit1201(where k=1, 2, . . . , n). Alternatively, the dangerous area generation unit1101may determine, as the dangerous area31, the triangle that is defined, as an approximation of the sector, by the point A(AX, AY) and the two positions of the distal end of the movable unit1203corresponding to the k-th and (k−1)th rotation angles held by the motion planning unit1201(where k=1, 2, . . . , n).

The worker position acquisition unit1102acquires the position of the worker51. For example, an image-capturing unit (not shown) installed on the ceiling captures the worker51, and the worker position acquisition unit1102acquires the worker51position using image processing technology. Alternatively, the position acquisition unit1102may acquire the worker51position by a position sensor (not shown) previously held by the worker51.

The position/posture determination unit1103determines at least one (hereinafter also referred to as a position/posture) of the position (hereinafter also referred to as a characteristic position) of the robot1200to be presented to the worker51, and a posture (hereinafter also referred to as a characteristic posture) of the robot1200to be presented to the worker51. The position/posture determination unit1103then determines the position/posture so that the worker51can appropriately recognize a dangerous situation around the robot1200. The characteristic position and/or the characteristic posture that are determined by the position/posture determination unit1103is presented to the worker51by the later-described display device1300, the worker51being in a dangerous situation.

In Embodiment 1, the characteristic posture is the posture of the movable unit1203at the time a predetermined time before the predicted time of collision between the worker51and the movable unit1203. The characteristic position is the point which is closest to the worker, the point among the points contained in the movable unit1203. Here, a target time at which the characteristic posture is determined is not the time of collision, but a time before the collision. The danger presentation system1000appropriately informs the worker51of the direction in which the movable unit1203is approaching the worker51by presenting the position of the movable unit1203at a time before the collision.

The characteristic position may be the position which indicates the center of the robot1200or the movable unit1203at the target time. For example, when the entire robot1200moves, by the danger presentation system1000presenting the center position as the characteristic position, the worker51can recognize the direction in which the robot1200is approaching.

FIG. 6is a flowchart illustrating the flow of the process performed by the position/posture determination unit1103illustrated inFIG. 2. In the description, k is a variable, and is held by a storage area such as a register of CPU, a ROM, a RAM, or a HDD.

FIG. 7is a diagram illustrating the characteristic posture61determined by the position/posture determination unit1103. InFIG. 7, the point (X—0, Y—0), the point (X—1, Y—1), . . . , the point (X—27, Y—27) indicate the positions of the distal end of the movable unit1203at time 0, time 1, time 2, . . . , time 27, respectively. The point (PX, PY) indicates the worker position which is the worker51position acquired by the worker position acquisition unit1102.

Hereinafter, the flow of the process performed by the position/posture determination unit1103is described with reference toFIGS. 6 and 7.

First, the position/posture determination unit1103acquires the worker51position by the worker position acquisition unit1102(S101). In the case of the example indicated inFIG. 7, the position/posture determination unit1103acquires the worker position (PX, PY).

Next, the position/posture determination unit1103set that k=(S102). Next, the position/posture determination unit1103acquires the motion planning at time (k−1) from the motion planning unit1201(S103). Because k=1 in this step, the position/posture determination unit1103acquires the 0th rotation angle “0.” In the case of the example illustrated inFIG. 7, when k=1, the position/posture determination unit1103acquires the rotation angle “0” which corresponds to the position (X—0, Y—0).

Next, the position/posture determination unit1103acquires the motion planning at time k from the motion planning unit1201(S104). In the case of the example illustrated inFIG. 7, when k=1, the position/posture determination unit1103acquires the rotation angle “10” which corresponds to the position (X—1, Y—1); and when k=2, the position/posture determination unit1103acquires the rotation angle “20” which corresponds to the position (X—2, Y—2).

In the case where the k-th motion planning cannot be acquired from the motion planning unit1201, the position/posture determination unit1103terminates the process. Specifically, when k=28, the motion planning unit1201does not hold the 28th motion planning in the example illustrated inFIG. 7. In this case, the position/posture determination unit1103terminates the process.

Here, as long as the motion planning unit1201has a motion planning, the position/posture determination unit1103continues to acquire a motion planning. However, in order to reduce processing amount, when the value of k reaches a certain value or more, the position/posture determination unit1103preferably terminates the process even when the motion planning unit1201still has a motion planning.

In the case where the k-th motion planning can be acquired from the motion planning unit1201, the position/posture determination unit1103generates a range (also referred to as a movement range or a motion range) covered by the movable unit1203that moves during a time period from time (k−1) to time k (S105).

Here, the position/posture determination unit1103generates a sector containing the following three points. First, the first point which is the sector center is the rotation center point A(AX, AY) of the movable unit1203. The second point is the point (X_k−1, Y_k−1) which is the position of the distal end of the movable unit1203at time (k−1). That is to say, the second point is the position of the distal end of the movable unit1203in the (k−1)th rotation angle of the motion planning unit1201. The third point is the point (X_k, Y_k) which is the position of the distal end of the movable unit1203at time k. That is to say, the third point is the position of the distal end of the movable unit1203in the k-th rotation angle of the motion planning unit1201.

Specifically, when k=1 in the example illustrated inFIG. 7, the position/posture determination unit1103generates the sector defined by (AX, AY), (X—0, Y—0), and (X—1, Y—1). When k=18, the position/posture determination unit1103generates the sector defined by (AX, AY), (X—17, Y—17), and (X—18, Y—18).

Next, the position/posture determination unit1103determines whether or not the worker51position acquired from the worker position acquisition unit1102is included in the movement range of the movable unit1203(S106).

Specifically, when k=1 in the example illustrated inFIG. 7, the worker position is not included in the sector. Then k is incremented sequentially from 1, and when k=18, the worker position is included in the movement range (a sector) of the movable unit1203defined by the point A(AX, AY), the point (X—17, Y—17), and the point (X—18, X—18). Thus, when k=0, 1, 2, . . . , 17, the position/posture determination unit1103determines that the worker51position is not included in the movement range of the movable unit1203, and when k=18, the position/posture determination unit1103determines that the worker51position is included in the movement range of the movable unit1203.

When the worker position is not included in the movement range (NO in S106), the position/posture determination unit1103stores the value k incremented by 1 to k (S108). The position/posture determination unit1103acquires the motion planning at time k from the motion planning unit1201(S104), and subsequent process is repeated.

When the worker position is included in the movement range (YES in S106), the position/posture determination unit1103determines the posture of the movable unit1203at time (k−1) as the characteristic posture (S107). That is to say, when k=18 in the example illustrated inFIG. 7, the characteristic posture is defined by the posture of the movable unit1203when k=17 because the worker position is included in the sector. More specifically, the position/posture determination unit1103defines the characteristic posture as line segment joining the point A(AX, AY) and the point (X—17, Y—17).

Here, a target time at which the characteristic posture is determined is not the time of collision, but a time before the collision. Accordingly, the danger presentation system1000can appropriately inform the worker51of the direction in which the movable unit1203is approaching the worker51by presenting the position of the movable unit1203at a time before the collision. Time (k−1) is just an example, and the time at which the characteristic posture is determined may be a time before time (k−1).

To set a target time, the position/posture determination unit1103may measure the distance between the line segment of the characteristic posture and the worker51position at time k−1, time k−2, . . . , 0, and then sets the target time as the measurement time for which the measured distance reaches a predetermined distance or more for the first time in the above order of the measurement times. It is easier for the worker51to recognize a dangerous situation with a characteristic posture away from the worker51by a predetermined distance or more.

Next, the position/posture determination unit1103determines a characteristic position (S109). That is to say, the position/posture determination unit1103determines a point on the line segment joining the point A(AX, AY) and the point (X—17, Y—17) as the characteristic position, the point being closest to the worker position (PX, PY).

The image generation unit1104generates an image which illustrates the dangerous area31generated by the dangerous area generation unit1101, and the characteristic determined by the position/posture determination unit1103. That is to say, the image generation unit1104superimposes the characteristic posture and position determined by the position/posture determination unit1103on the dangerous area31generated by the dangerous area generation unit1101.

The display device1300is a device that displays an image generated by the image generation unit1104.

FIG. 8is a diagram illustrating how the display device1300displays an image generated by the image generation unit1104. The display device1300installed at the upper portion of a room in which robot1200is disposed presents the dangerous area31around the robot1200. In addition, the display device1300presents the images of a characteristic posture71and a characteristic position72.

In the above manner, the dangerous presentation system1000in Embodiment 1 presents the dangerous area31around the robot1200. In addition, the danger presentation system1000presents the image of the characteristic posture71.

The worker51can recognize the dangerous area31where the movable unit1203of the robot1200moves. However, the direction in which the robot is approaching the worker51cannot be recognized only with the dangerous area31. Thus, when the worker51tries to escape from the dangerous area31, the worker51may accidentally approach the movable unit1203.

The worker51can recognize the direction in which the movable unit1203is approaching with the characteristic posture. However, the worker51cannot recognize the direction in which the movable unit1203moves after approaching the worker51. Consequently, the worker51may keep dodging to avoid the movable unit1203within the motion space of the movable unit1203.

However, the worker51can determine in which direction the worker51should move to secure safety by recognizing the dangerous area31and the characteristic posture that are displayed by the danger presentation system1000in Embodiment 1.

The danger presentation system in Embodiment 1 presents a dangerous area to a worker in the dangerous area. The dangerous presentation system in Embodiment 2 further presents a dangerous area to a worker who is currently not in the dangerous area, but may move to the dangerous area later.

FIG. 9is a general diagram of the dangerous presentation system in Embodiment 2. A dangerous presentation system1010in Embodiment 2 includes a danger presentation device1110, a robot1200, and a display device1300. The dangerous presentation device1110includes a position/posture determination unit1113whose operation is different from the operation of the position/posture determination unit1103in Embodiment 1. Other components are the same as those of Embodiment 1, and description is omitted.

FIG. 10is a flowchart illustrating the process of determination of a collision position for presentation, the determination being made by the position/posture determination unit1113illustrated inFIG. 9.

FIG. 11is a graph illustrating the relationship between the movement range of the movable unit1203and predicted worker position. Hereinafter, the flow of the process performed by the position/posture determination unit1113is described with reference toFIGS. 10 and 11.

First, the position/posture determination unit1113acquires a worker position (S101). Next, the position/posture determination unit1113sets variable k to 1 (S102). Next, the position/posture determination unit1113acquires the (k−1)th motion planning (S103). Next, the position/posture determination unit1113tries to acquire the k-th motion planning and determines whether or not the k-th motion planning can be acquired (S104).

The position/posture determination unit1113, when being unable to acquire the motion planning, terminates the process, and when being able to acquire the motion planning, generates the movement range of the movable unit1203(S105). So far, the position/posture determination unit1113performs the same process as that in Embodiment 1 illustrated inFIG. 6.

The position/posture determination unit1113generates the movement range of the movable unit1203(S105), and then predicts the position of the worker at the k-th time (S201). For example, the position/posture determination unit1113previously holds an average moving speed V of the worker, and predicts that the worker is in a range which is a set of points whose distance from the initial position is less than or equal to kxV at time k. InFIG. 11, P—1, P—2, . . . , P_k indicate predicted positions of the worker at time 1, time 2, . . . , time k, respectively, with time 0 corresponding to the initial state. The circles for indicating corresponding predicted positions have the center at the worker position (PX, PY), and radii of 1×V, 2×V, . . . , k×V, respectively.

The position/posture determination unit1113then calculates whether or not part of the predicted position of the worker is included in the movement region of the movable unit1203. Specifically, the position/posture determination unit1113determines whether or not there is a point which is included in the sector illustrating the movement region of the movable unit1203as well as in the circle illustrating the predicted position of the worker (S202).

When the predicted position of the worker is not in the movement range (NO in S202), the position/posture determination unit1113stores the value of k incremented by 1 into k similarly to Embodiment 1 illustrated inFIG. 6(S108).

When the predicted position of the worker is included in the movement range (YES in S202), the position/posture determination unit1113determines the position of the movable unit1203at time k as the characteristic posture (S203). That is to say, when k=18, the predicted position of the worker is included in the sector, the position/posture determination unit1113determines the position of the movable unit1203at time 18 as the characteristic posture. More specifically, the position/posture determination unit1113determines the line segment joining the point A(AX, AY) and the point (X—18, Y—18) as the characteristic posture.

When the characteristic posture is determined, the position/posture determination unit1103in Embodiment 1 uses a time before the predicted time of collision to determine the characteristic posture, however, the position/posture determination unit1113in Embodiment 2 may use the predicted time of collision to determine the characteristic posture. Even when the predicted time of collision is used, the position/posture determination unit1113can present in which direction of movement, the worker comes into contact with the movable unit1203, by predicting the position of the worker in motion.

The position/posture determination unit1113may determine the characteristic posture on the basis of a time before the predicted time of a collision similarly to Embodiment 1. Accordingly, the danger presentation system1010can present the direction in which the movable unit1203is approaching more reliably.

Next, the position/posture determination unit1113determines a predicted position a collision at time k as the characteristic position (S204). For example, the position/posture determination unit1113determines, as the characteristic position, the overlapping intersection between the circle illustrating the predicted position of the worker and the line segment illustrating the characteristic posture. In the example illustrated inFIG. 11two characteristic positions73exist. The position/posture determination unit1113may determine both or either one of the two characteristic positions73as the characteristic position.

The position/posture determination unit1113may determine the center of the two characteristic positions73as the characteristic position. The center of the two characteristic positions73is the closest to the position (PX, PY) of the worker, thus has high possibility of a collision with the worker.

The characteristic posture and characteristic position determined in this manner are presented by the danger presentation system1010.

As described above, the danger presentation system1010in Embodiment 2 presents the dangerous area31around the robot1200. The danger presentation system1010further presents an image of the movable unit in the characteristic posture71. By recognizing the dangerous area and the characteristic posture displayed by the dangerous presentation system1010, the worker51can determine in which direction the worker51should move to secure safety.

The worker51when moving near the robot1200, may move into more dangerous area than before. However, the danger presentation system1010can present to the worker51which direction of movement would create a risk of collision with the robot1200by predicting the movement of the worker51.

When there is possibility of collision between a worker and a robot, the danger presentation system in Embodiment 1 and Embodiment 2 displays, as a characteristic position, a certain place where the worker may collide with the robot at an earliest possible time, or presents the posture of the robot at the time of collision as a characteristic posture. However, the worker may not be able to view the presented characteristic position or characteristic posture depending on the direction in which the worker faces. The danger presentation system in Embodiment 3 presents a characteristic position or a characteristic posture in the area which can be viewed by a worker in consideration of the direction in which the worker faces.

FIG. 12is a general diagram of a danger presentation system1020in Embodiment 3. The danger presentation system1020in Embodiment 3 includes a danger presentation device1120, a robot1200, a display device1300, and a warning sound output device1400. The danger presentation device1120includes a position/posture determination unit1123whose operation is different from the operation of the position/posture determination unit1103in Embodiment 1. The danger presentation device1120further includes a worker view range determination unit1125. Other components are the same as those of Embodiment 1, and description is omitted.

The worker view range determination unit1125determines the view range of the worker (also referred to as a view, a view area, or a worker view range) based on the orientation of the posture of the worker. The worker view range determination unit1125determines the view range of the worker under the assumption that, for example, a worker wears a helmet equipped with a position sensor each in the front and in the back, and looks in the direction of the line joining the positions acquired by the two sensors.

FIG. 13is a diagram illustrating the view area determined based on the values acquired from the position sensor. The worker front position (FX, FY) indicates the position of the position sensor installed in front of the helmet, and the worker back position (BX, BY) indicates the position of the position sensor installed on the back of the helmet.

The worker view range determination unit1125determines, as the view range, a sector (view74illustrated inFIG. 13) which is part of the circle with the center at the worker front position (FX, FY) and the radius of D, the sector being formed by rotating the radius in the direction through the worker back position (BX, BY) and the worker front position (FX, FY) for a rotation angle α clockwise and counter clockwise.

The worker view range determination unit1125may determine the radius D based on the concept of personal space. “Personal space is an area around the body of a human or an animal, and if other human or animal of the same species comes into the area, a negative response is normally caused. (Psychology Dictionary, edited by Tamotsu Fujinaga, and Mariko Naka, Maruzen, 2004)”

Now, the danger presentation system1020can make a worker have a desire to escape from the area by presenting the characteristic posture and characteristic position of the robot within the range of personal space.

Edward Hall has shown that personal space changes based on the intimacy of the relationship between you and other human. Here, it is generally expected that an intimate feeling of a worker to a robot increases as an increase in the number of times the robot is used, the number of times the robot is touched, and the time period during which the robot is observed by the worker. Then, the worker view range determination unit1125may narrow the personal space and the view74based on the number of times of the use, touch, and the observation time of the robot.

That is to say, the danger presentation system1020includes a unit to acquire the number of times of use, touch, and the observation time of the robot, and the worker view range determination unit1125may reduce the radius D in accordance with an increase in the number of times of use, touch, and the observation time of the robot. For example, for a worker who has an intimate feeling to the robot, the radius D is set to 45 cm, and for a worker who has no intimate feeling to the robot, the radius D is set to 120 cm or 350 cm.

In addition, the distance of personal space changes depending on the race and the character of a target worker. Thus, the danger presentation system1020may include a unit to acquire the race and the character of a target worker, and the worker view range determination unit1125may determine based on the race and the character of a target worker.

The worker view range determination unit1125may determine a view area by estimating the position of a camera installed in front of the helmet based on the image obtained from the camera. Here, instead of a helmet, the camera may be installed on an eyeglass, a bag carried by the worker, or a wheelchair in the case where the worker is in the wheelchair. Alternatively, the worker view range determination unit1125may estimate the view direction of the worker based on the position of the eyes of the worker by capturing the worker's face with a camera installed indoors.

The danger presentation system1020in Embodiment 3 presents a characteristic position or a characteristic posture to the view74, however, in some cases, the characteristic position or the characteristic posture cannot be presented to the view74, or it is better not to present the characteristic position or the characteristic posture to the view74. Hereinafter, specific description is given.

FIG. 14is a diagram illustrating a view range of a worker who faces away from a robot in Embodiment 3.

The worker51is outside the dangerous area31. The view74of the worker51is not overlapped with the dangerous area31. However, because of the movement of the worker51, a collision at time k is predicted.

In this case, even when the danger presentation system1020presents an image of the characteristic posture61and the characteristic position73, the image does not appear in the view74of the worker51. Thus, in this case, the danger presentation system1020causes the output device1400to output a warning sound in order to inform the worker51of a danger. A configuration may be made such that when a danger is notified to the worker51, a message such as “a robot is approaching from the back” is presented in the view range of the worker51by using the display device1300instead of using the warning sound output device1400. Alternatively, a configuration may be made so as to inform the worker51of an approach of a robot from the back by blinking the output image of the display device1300.

When a warning sound is outputted, a message is presented, or a blinking display is shown constantly without adopting the above-mentioned configuration, attention of a worker may be reduced because of because of too frequent warnings. However, by adopting such a configuration described above, the worker receives those warnings only when the presentation of a danger by showing the characteristic posture is not effective. Consequently, attention of the worker increases, and the worker can make a more effective determination for a danger of collision such as a collision with a robot approaching from the back.

FIG. 15is a diagram illustrating a view range of the worker51who faces in the same direction as the movement direction of the movable unit1203in Embodiment 3.

The worker51is inside the dangerous area31. However, because of the movement of the worker51, a collision of the worker51with the movable unit1203is predicted at time k−1. However, the movable unit1203does not appear in the view74at time k−1. The danger presentation system1020in Embodiment 3 presents the characteristic position or the characteristic posture in an area which can be viewed by the worker51, thus does not determine the posture of the robot1200at time k−1 as the characteristic posture.

Next, a collision of the worker51with the movable unit1203at time k is predicted. The movable unit1203appears in the view74at time k−1. When the position and posture of the robot1200at time k are presented to the worker51as the characteristic position and the characteristic posture, there is a possibility that the worker51erroneously recognizes that the movable unit1203of robot1200is approaching from the front.

Therefore, the danger presentation system1020does not present the characteristic position or the characteristic posture at the time when the robot1200passes through the position of the worker51, or at the later time. In this case, the danger presentation system1020causes the warning sound output device1400to output a warning sound in order to inform the worker51of a danger. Alternatively, the danger presentation system1020causes the display device1300to output a warning.

As described above, the display device1300and the warning sound output device1400may serve as a warning device which outputs a warning. The display device1300and the warning sound output device1400are examples of a warning device, thus the danger presentation system1020may include another warning device. The danger presentation device1120may include a warning device which outputs a warning.

FIG. 16is a flowchart illustrating the process in which the position/posture determination unit1123illustrated inFIG. 12determines a collision position for presentation.FIG. 17is a graph illustrating the relationship between the movement range of the movable unit1203and a predicted worker position in Embodiment 3. Hereinafter, the flow of the process performed by the position/posture determination unit1123is described with reference toFIGS. 16 and 17.

Because the process of acquiring the worker position (S101), the process of setting the variable k to 1 (S102), the process of acquiring the (k−1)th motion planning (S103), the process of acquiring the k-th motion planning (S104), the process of generating a movement range of the movable unit1203(S105), and the process of adding 1 to k (S108) that are illustrated inFIG. 16are the same as respective processes in Embodiment 1 illustrated inFIG. 6, description of those processes inFIG. 16is omitted.

In addition, the process of predicting the worker position at the k-th time (S201), the process of determining whether or not the predicted worker position is included in the movement range (S202), and the process of determining a characteristic position (S203) that are illustrated inFIG. 16are the same as the respective processes in Embodiment 2 illustrated inFIG. 10, thus description of the those processes inFIG. 16is omitted.

The position/posture determination unit1123, after acquiring a worker position (S101), determines a view range by using the worker view range determination unit1125(S301).

When the predicted worker position is included in the movement range (YES in S202), the position/posture determination unit1123determines whether or not part of the view area is included in part of the movement range (S302). Specifically, the position/posture determination unit1123determines whether or not at least a predetermined number of points (X, Y) on the XY plane satisfy the following two conditions:

The first condition is that the point (X, Y) is a point in a movement range acquired by the process (S105) of generating the movement range of the movable unit1203.

The second condition is that the point (X, Y) is a point in a view area acquired by the process (S301) of acquiring the view area.

Here, when the view area is not contained in part of the movement range (NO in S302), the position/posture determination unit1123stores the value k incremented by 1 to k (S108).

On the other hand, when the view area is contained in part of the movement range (YES in S302), the position/posture determination unit1123determines whether or not the initial position of the worker is included in the movement range (S305).

Here, when the worker's initial position is included in the movement range (YES in S305), as in the example illustrated inFIG. 15, there is a possibility that the worker erroneously recognizes the moving direction of the robot1200by displaying the posture of the robot1200after the movement. Thus, in this case, the position/posture determination unit1123causes the warning sound output device1400to output a warning sound (S304). The position/posture determination unit1123may cause the display device1300to output a warning via the image generation unit1104. The position/posture determination unit1123may cause the display device1300to directly output a warning not via the image generation unit1104.

On the other hand, when the worker's initial position is not included in the movement range (NO in S305), the position/posture determination unit1123determines the characteristic posture similarly to Embodiment 2 (S203).

The position/posture determination unit1123then determines a characteristic position (S306). Here, the position/posture determination unit1123may determine a characteristic position similarly to Embodiment 2. Alternatively, the position/posture determination unit1123may determine a portion near the center of the view as the characteristic position.

When the k-th motion planning cannot be acquired by the process (S104) of acquiring the k-th motion planning, the position/posture determination unit1123determines whether or not there has been a state in which the predicted worker position is included in the movement range (S303). That is to say, the position/posture determination unit1123determines whether or not it has been determined that a predicted worker position is included in the movement range in the determination of whether or not a predicted worker position is included in the movement range (S202).

Specifically, the position/posture determination unit1123prepares a variable with an initial value of 0, and when a predicted worker position is included in the movement range (YES in S202), the position/posture determination unit1123substitutes 1 for the variable. When the value of the variable is 1, the position/posture determination unit1123determines that there has been a state in which a predicted worker position is included in the movement range (YES in S303), and when the value of the variable is 0, the position/posture determination unit1123determines that there has not been a state in which a predicted worker position is included in the movement range (NO in S303).

Here, when there has not been a state in which a predicted worker position is included in the movement range (NO in S303), the position/posture determination unit1123terminates the process.

On the other hand, when there has been a state in which a predicted worker position is included in the movement range (NO in S303), the position/posture determination unit1123causes the warning sound output device1400to output a warning sound (S304). Alternatively, the position/posture determination unit1123causes the display device1300to output a warning. Accordingly, the danger presentation system1020solves a problem of not displaying a characteristic posture or a characteristic position in the view even in a situation that there is possibility of a collision.

The position/posture determination unit1123determines that the movement position is included in the view area when at least a predetermined number of points satisfy the two conditions in the process (S302) of determining whether or not the movement area is included in the view area. However, the position/posture determination unit1123may determine that the movement position is included in the view area when the ratio of the number of points satisfying the two conditions with respect to the number of points satisfying the second condition is a predetermined ratio or more.

Accordingly, for example, even in the case where the area included in the view area is small because a worker is at a place near the end of a room, and is looking in the direction to the end, the danger presentation system1020can present the characteristic position or the characteristic posture.

The predetermined number used for determination in the process (S302) of determining whether or not the movement area is included in the view area may be decided based on the size (area) of the worker.

That is to say, in general, a human tends to be scared of a creature bigger than the human. Therefore, the predetermined number may be determined based on whether or not the size (area) of the robot presented in the view is greater than the size of the worker. The size of the worker may be determined by extracting a human form from an image acquired by a camera or the like. Alternatively, the size of the worker may be given in advance. A human tends to be scared of a creature which is two times or three times bigger than the human. Therefore, the predetermined number may be determined by comparing a fixed multiple of the worker's size with the robot rather than directly comparing the worker's size with the robot.

The value of the predetermined number may be 1. Accordingly, the position/posture determination unit1123determines that the movement position is included in the view area when at least a part of the movement area overlaps with the view area.

FIG. 18is a diagram illustrating a characteristic posture and a characteristic position which are presented in a worker's view in Embodiment 3.

The robot1200is in the view74. Accordingly, the worker51can recognize the existence of the robot1200. However, the worker cannot recognize the direction in which the robot1200is approaching. In the case of the example illustrated inFIG. 18, the robot1200rotates counter clockwise based on a motion planning and approaches the worker51from the left of the worker51.

At this moment, the worker51cannot recognize a danger appropriately even when the danger presentation system1020presents the image of the characteristic posture81and the characteristic position82that are not completely in the view74. For this reason, the danger presentation system1020presents the image of the characteristic posture71and the characteristic position72that are completely in the view74.

The case has been illustrated where the danger area generation unit1101determines the range of the dangerous area31using all the planning held by the motion planning unit1201, however, a configuration is desirably made such that the dangerous area31is generated based on the motion planning in the range until the robot1200reaches the characteristic posture71(or the characteristic position72).

For example, when the robot1200illustrated inFIG. 18rotates 360 degrees, the dangerous area31is an area where the robot1200moves during one rotation. Therefore, even when the characteristic posture71is presented, it is not easy to recognize whether the robot1200is approaching clockwise or counterclockwise. However, when the dangerous area31is generated as the area where the robot1200moves until the robot1200reaches the characteristic posture71, it is clear to recognize in which direction the robot1200is approaching.

The danger presentation system1020may have a configuration in which an animation is dynamically displayed by switched between a plurality of characteristic postures instead of continuing to present the characteristic posture71. For example, the danger presentation system1020may display the characteristic posture81, as the first display, which is the first posture that starts to appear in the view, and may display the characteristic posture71, as the last display, which is a posture that is sufficiently visible in the view.

The danger presentation system1020may present an animation including the postures between the above-mentioned first display and the above-mentioned last display. With such a configuration, it is much easier to recognize that the robot1200is approaching counter clockwise.

When an animation is presented based on the planning held by the motion planning unit1201instead of using the above configuration, there may be a case where the animation is started before the robot1200appears in the view of the worker51, and is not ended even after the robot1200disappears from the view of the worker51.

On the other hand, in the present configuration, the start and end times of the animation are determined based on the time when the robot1200starts to appear in the view as well as the time when the robot1200is sufficiently visible in the view. Therefore, it is easy for the worker51to recognize the animation. Then the worker51can easily recognize in which direction the robot1200is approaching. The danger presentation system1020may continue to present the characteristic posture71after the end of the animation, or may return to the start of the animation to repeat the display.

As described above, the danger presentation system1020in Embodiment 3 presents the dangerous area31around the robot1200. The danger presentation system1020further presents the image of the movable unit in the characteristic posture71. By recognizing the dangerous area and the characteristic posture displayed by the dangerous presentation system1020, the worker51can determine in which direction the worker51should move to secure safety.

In addition, the danger presentation system1020in Embodiment 3 predicts the position of the movable unit1203in the time order based on the motion planning. The danger presentation system1020presents to the worker51, as the characteristic position or the characteristic posture, the position and/or posture of the movable unit1203at a time when the movable unit1203is included in the view area of the worker51, the time being separately predicted. Accordingly, the characteristic position or the characteristic posture for presentation is presented at the location which is visible from the worker51. Therefore, this prevents the worker51from failing to recognize the characteristic position or the characteristic posture.

In addition, the danger presentation system1020continues to estimate the position of the movable unit1203in the time order until the characteristic position or the characteristic posture satisfies a predetermined condition. The danger presentation system1020then determines a presentation position which satisfies the condition. Accordingly, the danger presentation system1020can avoid or reduce the possibility that the characteristic position or the characteristic posture is presented in the corner of the view. Therefore, possibility of overlooking the presented characteristic position or characteristic posture is reduced.

Furthermore, when a characteristic position or a characteristic posture is not found in the view area in spite of a possibility of collision of the worker51with the movable unit1203, the danger presentation system1020in Embodiment 3 emits a warning sound. Accordingly, the danger presentation system1020can give a warning of a collision with the movable unit1203outside the view area to the worker51.

In Embodiments 1 to 3, the danger presentation system for a rotary robot has been shown. When a posture of the rotary robot in the near future is presented as a danger, a worker may not be able to recognize the danger depending on the posture. The danger presentation system with an improved feature regarding this point has been shown.

The danger presentation system according to Embodiment 4 is directed to a mobile robot. When a posture of a mobile robot in the near future is presented as a danger, the presented image of the robot may hide behind a wall depending on the position of the robot. In such a case, a worker cannot recognize the danger. Here, a danger presentation system which can present a danger even in such a case is described.

FIG. 19is a diagram illustrating a workspace2701in Embodiment 4. The danger presentation system1030is a device which performs presentation for informing the worker51of a danger.

The danger presentation system1030includes the robot1200, a danger presentation device1130, and the display device1300. The display device1300is a display unit such as a projector which performs display for informing the worker51of a danger. The danger presentation device1130generates data for displaying danger as an image or the like.

The workspace2701includes the worker51, the robot1200, and a wall2802. The positional relationship between the worker51, the robot1200, and the wall2802inFIG. 19is an example. The worker51may not be able to directly view the robot1200depending on the positional relationship.

In such a situation, it is difficult for the worker51to perceive a danger of collision with the robot1200. Thus, it is effective for safety purposes to present a danger to the worker51by some method. The danger presentation system1030in Embodiment 4 presents a danger to the worker51by visually displaying from which direction the robot1200appears.

FIG. 20is a diagram illustrating the danger presentation system1030in Embodiment 4. The danger presentation system1030includes the robot1200, the danger presentation device1130, and the display device1300. The robot1200in Embodiment 4 is a mobile robot, and includes a motion planning unit1201, a motion control unit1202, and a movable unit1203.

The motion planning unit1201plans the near-future position of the robot1200. Specifically, for example, the motion planning unit1201outputs a position P_T at a designated time T (T=0, 1, . . . ). In a simple configuration, the motion planning unit1201holds the position P_T of the robot1200at time T in a table in advance. The motion planning unit1201then refers to the table to output the position P_T of the robot1200at the designated time.

Movable portion1203is a portion which moves physically in robot1200. Typically, the movable unit1203is a wheel or a leg. The motion control unit1202controls the movable unit1203. The motion control unit1202then moves the robot1200based on the motion planning held by the motion planning unit1201. Specifically, the motion control unit1202controls the movable unit1203so that the position of the robot1200at time T is the position P_T.

The danger presentation device1130includes the worker position acquisition unit1102, a worker orientation acquisition unit1141, the worker view range determination unit1125, a position/posture determination unit1133, the image generation unit1104, and an obstruction information acquisition unit1142.

The worker position acquisition unit1102is a unit to acquire the position of the worker51. For example, the worker51is equipped with a position sensor in advance, and the worker position acquisition unit1102acquires a position from the equipped position sensor.

The worker orientation acquisition unit1141acquires the direction in which the worker51faces within the motion space of the robot1200. The worker orientation acquisition unit1141determines the orientation of the worker51under the assumption that, for example, the worker51wears a helmet equipped with a position sensor each in the front and in the back, and the worker51looks in the direction of the line joining the two positions acquired by the two sensors.

The obstruction information acquisition unit1142is a unit to acquire position information of an obstruction which obstructs the view of the worker51. Specifically, the obstruction information acquisition unit1142acquires the position information of the wall2802and the like.

The worker view range determination unit1125determines the view range of the worker51.FIG. 21is a diagram illustrating the view range of the worker51.

First, the worker view range determination unit1125acquires the position of the worker51using the worker position acquisition unit1102. Next, the worker view range determination unit1125acquires the direction in which the worker51faces using the worker orientation acquisition unit1141. InFIG. 21, a worker orientation vector4201is a vector which indicates the direction (worker orientation) in which worker51faces.

Next, worker view range determination unit1125acquires the information of an obstruction in the workspace2701from the obstruction information acquisition unit1142. For example, the position of the wall2802inFIG. 21is acquired by the worker view range determination unit1125. Next, the worker view range determination unit1125determines, as a worker view range74, a set of point X which is in the workspace2701and satisfies the following first to third conditions.

The first condition is that the angle formed by the vector connecting the position of the worker51and the point X, and the worker orientation vector4201is 55 degrees or less. The second condition is that there is no obstruction such as the wall2802on the line segment connecting the position of the worker51and the point X. The third condition is that the distance between the position of the worker51and the point X is 25 cm or more.

The worker view range74illustrated inFIG. 21is an example of the worker view range which satisfies the above-described conditions. The first condition indicates that the range with the central angle of 55 degrees to the right and left centered on the front of the worker51is included in the worker view range74. The second condition indicates that the space obstructed by the wall2802is not included in the worker view range74. The third condition indicates that an area very near the worker51such as the foot area of the worker51is not included in the worker view range74.

Here, the worker view range74is defined as the range with the central angle of 55 degrees to the right and left centered on the front of the worker51, however, this is an example, and the worker view range74may be a range with the central angle of other than 55 degrees to the right and left. The narrower the worker view range74is, the easier for the worker51to recognize the range. The central angle may be changed depending on the movement speed of the worker51.

For example, when the worker51is moving slower than a predetermined speed, the worker view range74may be defined with 55 degree angle, while when the worker51is moving faster than the predetermined speed, the worker view range74may be defined with 30 degree angle. By using such a configuration, even when the worker51is moving quickly with less attention to the surroundings, the worker51cannot easily overlook an object. On the other hand, when the worker51is moving slowly, an object is presented at a position, presentation at which would not interfere with the view of the worker51.

Here, the area which is apart from the worker51by a distance of 25 cm or more, is considered to be the worker view range74. However, the distance of 25 cm is an example and may be another distance. The longer the distance from the worker51, the easier for worker51to find the worker view range74. The distance may be changed depending on the movement speed of the worker51.

For example, when the worker51is moving slower than the predetermined speed, the worker view range74may be apart from the worker51by 25 cm, while when the worker51is moving faster than the predetermined speed, the worker view range74may be apart from the worker51by 50 cm. With such a configuration, even when the worker51is moving quickly with less attention to the surroundings, the worker51cannot easily overlook an object. On the other hand, when the worker51is moving slowly, an object is presented at a position, presentation at which would not interfere with the view of the worker51.

Here, the worker view range determination unit1125determines the worker view range74using the position of the worker, the orientation of the worker, and information of obstruction. However, the worker view range determination unit1125may estimate the worker view range74using any one of the above factors.

The position/posture determination unit1133determines the position of the image of the robot1200displayed on the workspace2701in order to inform the worker51of approach of the robot1200. The details are described later.

The image generation unit1104generates an image of the robot1200displayed on the workspace2701in order to inform the worker51of approach of the robot1200. The image generation unit1104also determines a display format of the robot1200to be displayed in order to inform the worker51of approach of the robot1200. The process of determining a display format may be performed by a display format determination unit, which is not shown. The detail of the display format determination process performed by the image generation unit1104is described later.

The display device1300presents the robot1200in the display format determined by the image generation unit1104at the position determined by the position/posture determination unit1133. Specifically, the display device1300is a unit such as a projector.

Hereinafter, the problem to be solved by the present embodiment is described. First, it is described that no problem occurs without the presence of the wall2802. Next, it is described that a problem occurs with the presence of the wall2802. Subsequently, a method of preventing a problem from occurring even with the presence of the wall2802is described.

First, a case where the position/posture determination unit1133simply presents the position of the robot1200at a time after a predetermined time interval is described. A case where there is no wall2802in the workspace2701, and a case where there is the wall2802in the workspace2701are described in this order. First, a summary of the case where there is no wall2802is described with reference toFIGS. 22 to 25.

FIG. 22is a diagram illustrating the positional relationship between the robot1200, the worker51, and an image2902of the robot1200at time 0. The image2902of the robot1200is the image at the position of the robot1200at the time after the predetermined time from time 0. Here, the predetermined time is assumed to be 2. The image2902of the robot1200is the image which indicates the position of the robot1200at time 2. The arrow inFIG. 22illustrates the movement path of the robot1200after time 0. Such a line is not actually indicated in the workspace.

The movement path includes the place at which the worker51is present. Therefore, the robot1200comes into contact with the worker51unless the worker51avoids the contact. Otherwise, the robot1200needs to detour or stop in order to avoid the contact. However, detouring or stopping of the robot1200reduces the working efficiency of the robot1200. In order for the worker51and the robot1200to share the workspace2701and work efficiently, it is preferable that the worker51grasps the planning of the robot1200and determines whether or not the worker51should dodge.

The worker view range74is a view range of the worker51at time 0. The worker view range74does not include either of the robot1200and the image2902of the robot1200. Therefore, the worker51cannot predict approach of the robot1200.

FIG. 23is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 4. The image2902of the robot1200is the image which indicates the position of the robot1200after a predetermined time from time 4 (i.e., at time 6). The worker view range74is the view range of the worker51at time 4. The worker view range74includes the image2902of the robot1200. Therefore, the worker51can predict the approach of the robot1200. That is to say, the worker51can start to prepare for avoidance action such as dodging back.

FIG. 24is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 6. The image2902of the robot1200is the image which indicates the position of the robot1200after a predetermined time from time 6 (i.e., at time 8). The worker view range74is the view range of the worker51at time 6. The worker view range74includes the robot1200and the image2902of the robot1200. The worker51has already started to prepare for avoidance at the time ofFIG. 23. That is to say, the worker51can start actual movement at the time. Thus, the worker51starts to move at the time.

FIG. 25is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 8. The image2902of the robot1200is the image which indicates the position of the robot1200after a predetermined time from time 8 (i.e., at time 10). The worker view range74is the view range of the worker51at time 6. The worker view range74includes the robot1200and the image2902of the robot1200. The position of the robot1200at time 8 is almost same as the position at which the worker51is at time 0. However, collision can be avoided because the worker51has already started to avoid collision.

As described above, the worker51can avoid collision with the robot1200by presenting the image2902of the near-future position of the robot1200. However, there is a possibility that the worker51cannot avoid collision when there is the wall2802. This is described with reference toFIGS. 26 to 29.

FIG. 26is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 0. Similarly to the case ofFIG. 22, the worker view range74at time 0 does not include either of the robot1200and the image2902of the robot1200. Therefore, the worker51cannot predict approach of the robot1200.

FIG. 27is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 4. The image2902of the robot1200is the image which indicates the position of the robot1200after a predetermined time from time 4 (i.e., at time 6). The worker view range74is the view range of the worker51at time 4. The worker view range74does not include the image2902of the robot1200. Therefore, the worker51cannot predict approach of the robot1200. Consequently, unlike the case ofFIG. 23, the worker51cannot start to prepare for avoidance action.

FIG. 28is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 6. The image2902of the robot1200is the image which indicates the position of the robot1200after a predetermined time from time 6 (i.e., at time 8). At the time, the image2902of the robot1200is included in the worker view range74at last. At the time, the worker51starts to prepare for avoidance action.

In the case ofFIG. 24, the worker51has started to avoid collision, while in the case ofFIG. 28, the worker51has barely started to prepare for the avoidance at the same time. That is to say, the worker51is late in avoiding collision.

FIG. 25is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 8. As a result of the late avoidance, the worker51collides with the robot1200.

As described above, the worker51cannot view the presented image2902of the robot1200because of the wall2802. Therefore, in some cases, the worker51is late in starting an avoidance action, and collides with the robot1200. Thus, the danger presentation system1030in Embodiment 4 determines the position of the image2902of the robot1200in consideration of the worker view range74of the worker51. The process performed by the danger presentation system1030is described with reference toFIG. 30.

First, the position/posture determination unit1133issue an instruction (command) to the worker view range determination unit1125to acquire the worker view range74. In response to the received command, the worker view range determination unit1125acquires the position of the worker51from the worker position acquisition unit1102(S2501).

Next, the worker view range determination unit1125acquires the orientation of the worker51from the worker orientation acquisition unit1141(S2502).

Next, the worker view range determination unit1125determines the worker view range74based on the position of the worker51acquired from the worker position acquisition unit1102, and the orientation of the worker51acquired from the worker orientation acquisition unit1102(S2503). The detail is similar to that of Embodiment 3. The worker view range determination unit1125informs the position/posture determination unit1133of the acquired worker view range74.

Next, the position/posture determination unit1133substitutes T0 for a variable T (S2504). The variable T holds a temporary value indicating an offset interval, at the time offset from the current time by the offset interval, the position of the robot1200is presented. For example, when the value of the variable T is 2, the image2902indicating the robot1200is presented at a predicted position of the robot1200at the time 2 seconds after the current time. In the following process, the position/posture determination unit1133determines the final value of the variable T while changing the value of the variable T. Here, it is assumed that the initial value T0 to be substituted for the variable T is “2.”

Next, the position/posture determination unit1133acquires the position of the robot1200at the time T seconds after the current time, from the motion planning unit1201(S2505).

Next, the position/posture determination unit1133determines whether or not the position of the robot1200time T later is included in the worker view range74(S2506). The method of determination is the same as that of Embodiment 3.

In the above step, when the position of the robot1200time T later is not included in the worker view range74(NO in S2506), the position/posture determination unit1133adds 1 to the variable T (S2507). The position/posture determination unit1133then acquires the position of the robot1200time T later again from the motion planning unit1201(S2505). That is to say, when the position of the robot1200at time T is not included in the worker view range74, the position/posture determination unit1133increments the time by one unit at a time until the position is included in the worker view range74.

On the other hand, when the position of the robot1200time T later is included in the worker view range74(YES in S2506), the position/posture determination unit1133determines the position of the robot1200time T later as a characteristic position (hereinafter also referred to as a display position). The image generation unit1104then determines a display format such as a color, a thickness of outline of the image2902of the robot1200to be displayed (S2508). The image generation unit1104then generates image data of the image2902to be displayed at the display position.

Here, by changing the display format of the image2902of the robot1200by the image generation unit1104, it becomes easy for the worker51to recognize how much time elapses until a collision occurs. The detail is described later.

Finally, the display device1300displays the image2902of the robot1200in the display format determined by the image generation unit1104at the position determined by the position/posture determination unit1133(S2509).

FIG. 31is a flowchart illustrating a modification of the process shown inFIG. 30. InFIG. 31, the position/posture determination unit1133, after adding 1 to the variable T (S2507), determines whether or not the variable T is less than a maximum T_MAX (S2510).

When the variable T is less than the maximum T_MAX (Yes in S2510), the position/posture determination unit1133again acquires the position of the robot1200time T later from the motion planning unit1201similarly to the process illustrated inFIG. 30(S2505).

On the other hand, when the variable T is greater than or equal to the maximum T_MAX (No in S2510), the danger presentation system1030terminates the process. Accordingly, the image2902of the robot1200in the distant future is not displayed. That is to say, the image2902is not displayed in a state where danger of collision is small.

So far, the danger presentation system1030that performs danger presentation one time has been described, however, the danger presentation system1030may actually perform danger presentation for a plurality of times with regular time intervals.

Next, the display format determination process performed by the image generation unit1104is described. In the danger presentation system1030, the image2902of the robot1200presented in the workspace2701may be presented at the position of the robot1200time T0 later, or at the positions of the robot1200time T0+1 later, T0+2 later, . . . in some cases. Thus, the worker51cannot tell when the robot1200is approaching based on the image2902of the robot1200viewed by the worker51.

Then, when the time until the robot1200reaches the determined display position is short, the image generation unit1104determines a display format so that the image2902of the robot1200is displayed in the display format more conspicuously. On the other hand, when the time until the robot1200reaches the determined display position is long, the image generation unit1104determines a display format so that the image2902of the robot1200is displayed in the display format less conspicuously.

For example, when the time until the robot1200reaches the display position is short, the image generation unit1104sets the brightness of the image2902of the robot1200to a high value. That is to say, the image generation unit1104causes the image2902to be displayed brightly. On the other hand, when the time until the robot1200reaches the display position is long, the image generation unit1104sets the brightness of the image2902of the robot1200to a low value.

For example, when the time until the robot1200reaches the display position is shorter than a predetermined time, the image generation unit1104causes the image2902to be displayed brightly so as to make the image2902more noticeable than in the case where the time until the robot1200reaches the display position is longer than or equal to the predetermined time. However, when a room is already bright with a light, the image generation unit1104may obstruct light to the image2902to cause the image2902to be displayed dark so as to make the image2902more noticeable.

For example, when the time until the robot1200reaches the display position is short, the image generation unit1104increases the thickness of the outline of the image2902indicating the robot1200.

On the other hand, when the time until the robot1200reaches the display position is long, the image generation unit1104decreases the thickness of the outline of the image2902indicating the robot1200.

For example, when the time until the robot1200reaches the display position is shorter than a predetermined time, the image generation unit1104increases the thickness of the outline of the image2902so as to make the image more noticeable than in the case where the time until the robot1200reaches the display position is longer than or equal to the predetermined time.

For example, the image generation unit1104generates image data around the image2902of the robot1200, the image data presenting a value indicating the time until the robot1200reaches the display position.

Hereinafter, the flow of the process of the image generation unit1104is described with reference toFIG. 32. The following process is an example of the display format determination process (S2508) in the flowchart illustrated inFIG. 30.

First, the image generation unit1104determines a display color for the robot (S2601). For example, the image generation unit1104calculates a value according to “(maximum brightness)×T0/(value of the variable T)” (Expression 1), and assigns the calculated value to the display color of the robot1200. Here, for example when it is assumed that brightness of white is “255” and brightness of black is “0”, the maximum brightness is “255” (brightness of white).

The value of “T0/(value of the variable T)” (Expression 2) is “1” when the value of the variable T is “T0”, and is “½” when the value of variable T is “T0×2.” That is to say, when the time until robot1200reach the display position is short, the value of Expression 2 is “1”, and the longer the time, the smaller the value of Expression 2 than “1.” Therefore, when the time until the robot1200reaches the display position is short, the image generation unit1104sets the brightness of the image2902of the robot1200to a high value, and when the time is long, sets the brightness to a low value.

Next, the image generation unit1104determines a thickness of the outline of the image2902of the robot1200to be displayed (S2602).

For example, the image generation unit1104determines a thickness of the outline based on “5×T0/(value of the variable T)” (Expression 3). The value of Expression 3 is “5” when the value of the variable T is “T0”, and is “2.5” when the value of the variable T is “T0×2.” That is, when the time until the robot1200reaches the display position is short, the value of Expression 3 is “5”, and the longer the time, the smaller the value of Expression 3 than “5.”

That is to say, when the time until the robot1200reaches the display position is short, the image generation unit1104increases the thickness of the outline of the image2902, and when the time is long, decreases the thickness of the outline of the image2902.

Next, the image generation unit1104determines a value of the time until the robot1200reaches the display position (S2603). The determined value is presented around the image2902of the robot1200by the display device1300. Specifically, the above-mentioned value is the value of the variable T, and indicates the time until the robot1200reaches the display position. The image generating unit1104truncates the decimal places of the value, or converts the value to the equivalent value in seconds as needed.

The summary of operation using the present technique is described with reference toFIGS. 33 to 36.

FIG. 33is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 0. The image2902of the robot1200is the image at the position of the robot1200at the time after time 8 from time 0. Here, “8” is the time at which the image2902of the robot1200is included in the worker view range74after the predetermined time “2” from the current time. At time 8, a reaching time 3702 to the position is displayed around the robot1200.

The brightness of the image2902of the robot1200is “255×2/8” (Expression 1). That is, the brightness is “63.75.” The thickness of the outline is “5×2/8” from Expression 3. That is, the thickness of the outline is “1.25.”

The worker51illustrated inFIG. 33can recognize that the robot1200is approaching from the back of the wall. Although the position where the image2902of the robot1200is presented is close to the worker51, the brightness of the image2902is low and the outline of the image2902is relatively thin. Therefore, the worker51can tell that there is still some time before collision, thus the current work can be continued.

FIG. 34is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 4. The image2902of the robot1200is the image at the position of the robot1200at the time after time 4 from time 4. The worker view range74is the view range of the worker51at time 4. At time 4, the worker51can predict the approach of the robot1200because the image2902of the robot1200is included in the worker view range74.

The brightness of the image2902of the robot1200is “255×2/4” (Expression 1). That is, the brightness is “127.5.” The thickness of the outline is “5×2/4” from Expression 3. That is, the thickness of the outline is “2.5.” The worker51can recognize that the approach of the robot1200is much closer because the image2902of the robot1200has become brighter and the outline also has become thicker than inFIG. 33. The worker51then can start to prepare for the avoidance.

FIG. 35is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 6. The image2902of the robot1200is the image at the position of the robot1200at the time after time 2 from time 6. The worker view range74is the view range of the worker51at time 6. At time 6, the worker view range74includes the image2902of the robot1200.

The brightness of the image2902of robot1200is “255×2/2” (Expression 1). That is, the brightness is “255.” The thickness of the outline is “5×2/2” from Expression 3. That is, the thickness of the outline is “5.” The worker51can recognize that the approach of the robot1200is much closer because the image2902of the robot1200has become brighter and the outline also has become thicker than inFIG. 34. Because the worker51starts to prepare for the avoidance at the time ofFIG. 34, the worker51can start to avoid collision at the time ofFIG. 35.

FIG. 36is a diagram illustrating the positional relationship between the robot1200, the worker51, and the image2902of the robot1200at time 8. The image2902of the robot1200is the image which indicates the position of the robot1200after a predetermined time from time 8 (i.e., at time 10). The worker view range74is the view range of the worker51at time 8, and includes the robot1200and the image2902of the robot1200.

The position of the robot1200at time 8 is almost same as the position at which the worker51is at time 0. However, collision can be avoided because the worker51has already started to avoid collision.

FIG. 37is a diagram illustrating a state where the worker51faces away from the direction of the robot1200approaching the worker51. InFIG. 37, the image2902of the robot1200is presented at the place of the worker51. In this case, the worker51receives light for displaying the image2902. In the case where the worker51knows that light receiving indicates a danger, the worker51can recognize the danger.

As the robot1200approaches the worker51, the brightness is increased, and the worker51receives brighter light. Therefore, the worker51can recognize that a danger is approaching from a difference in the brightness. In this manner, the danger presentation system1030can inform the worker51of a danger without depending on a warning sound or the like.

As described above, the danger presentation system1030in Embodiment 4 presents the near-future image2902of the robot1200in the worker view range74. Therefore, the danger presentation system1030can appropriately present a dangerous situation. In Embodiment 4, the characteristic position of a mobile robot is presented, however, the characteristic posture of a rotary robot may be presented.

As shown in a plurality of embodiments in the above, the danger presentation system according to an aspect of the present invention appropriately presents a dangerous situation around a robot. In addition, the danger presentation device according to an aspect of the present invention can generate data for appropriately presenting a dangerous situation around a robot. The generated data is displayed as an image of the robot by a display device. Accordingly, a worker can appropriately recognize the danger situation around the robot.

These embodiments are only examples, and the present invention is not limited to these embodiments. The embodiments obtained by making various modifications, which occur to those skilled in the art, to the present embodiments, and other embodiments which are achieved by arbitrarily combining the components of these embodiments are also capable of generating data for appropriately presenting a dangerous situation, and thus are included in the scope of the present invention.

The present invention can be implemented not only as a danger presentation device, but also as a method including the steps performed by the processing units constituting the danger presentation device. The steps of the method are typically executed by a computer. The present invention can be achieved as a program which causes a computer to execute the steps of the method. Furthermore, the present invention can be achieved as a computer-readable storage medium such as a CD-ROM in which the program is stored. Although only some exemplary embodiments of this invention have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention.

The danger presentation device according to an aspect of the present invention can generate data for appropriately presenting a dangerous situation accompanied by a motion of a robot, and can be utilized, particularly in a production site or the like where workers work near a robot.