DETECTION DEVICE, DETECTION METHOD, AND COMPUTER READABLE STORAGE MEDIUM

A detection device including: a sensor configured to emit a light and detect an object by detecting the light reflected from the object, and a processor configured to determine, when the object is detected in a first region that is narrower than a range where the light reaches, a motion of the object to be a gesture input for the detection device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2015-252108, filed on Dec. 24, 2015, the entire contents of which are incorporated herein by reference.

FIELD

Disclosed techniques are related to a detection device, a detection method, and a detection program.

BACKGROUND

In recent years, application of a wearable display device such as a head mounted display (HMD) has been being promoted as a measure to view information at a site of work. At a site where a worker who carries out input operation to an operation screen displayed on the HMD or the like frequently wears a work glove or the like, it is difficult to carry out the input operation by operating an input device such as a touch panel. Therefore, a user interface (UI) with which input operation may be carried out without directly operating an input device such as a touch panel may be required.

As one UI, a gesture input method in which a finger, a hand, or the like that makes a gesture representing input operation is shot by a camera and the gesture is recognized from the shot image has been proposed. However, at a site of work, it is sometimes difficult to stably carry out the gesture input due to the influence of the movement of the worker, change in the posture of the worker, environmental conditions such as the background color and illumination, and so forth.

Therefore, a technique in which gesture input is carried out by using a laser sensor that is robust regarding the environmental conditions such as illumination has been proposed.

For example, there has been proposed a control device based on gesture recognition in which the existence position of a detection-target object is detected from a distance measured by a laser range sensor that measures the distance to the detection-target object that exists in a detection plane. In this control device, the motion of the detection-target object may be detected from time-series data of the detected existence position of the detection-target object and a gesture may be extracted from the motion of the detection-target object. Then, a control command according to the extracted gesture may be generated to be given to control target equipment.

Furthermore, there has been proposed a method in which a user blocks light from a laser tracker at least partly and thereby a temporal pattern corresponding to one command selected from plural commands by the user is generated.

CITATION LIST

Patent Documents

SUMMARY

According to an aspect of the embodiments, a detection device includes a sensor configured to emit a light and detect an object by detecting the light reflected from the object, and a processor configured to determine, when the object is detected in a first region that is narrower than a range where the light reaches, a motion of the object to be a gesture input for the detection device.

DESCRIPTION OF EMBODIMENTS

In the method in which a detection-target object is detected by a laser range sensor, it is difficult to identify whether an object existing in a detection plane is an instructing body (object such a hand or a finger) that makes an instruction of an input by a gesture or an object other than the instructing body.

Furthermore, the laser range sensor or the like may be set on the environment side. However, at a site of work, it is preferable that gesture input may be carried out not at a fixed place but at arbitrary various places. Therefore, for example, it is conceivable that a worker wears the laser range sensor and thereby the gesture input at arbitrary places is enabled.

However, in this case, the possibility that an object other than the instructing body that makes an instruction of an input enters the detection plane of the laser range sensor becomes higher. As a result, there is a possibility that an object existing in the detection plane is detected as the instructing body although being an object other than the instructing body. Furthermore, the case in which the instructing body enters the detection plane although a gesture input is not intended is also envisaged. Also in this case, there is a possibility that this instructing body is detected as the instructing body although this instructing body is preferably not detected as the instructing body.

Disclosed techniques intend to stably detect the instructing body that makes an instruction of an input as one aspect.

One example of embodiments according to the disclosed techniques will be described in detail below with reference to the drawings.

First Embodiment

As illustrated inFIG. 1, a gesture input system100according to a first embodiment includes mounted equipment16, an HMD20, and a server30. In the gesture input system100, a gesture input carried out by a user who wears the mounted equipment16to an operation screen displayed on the HMD20based on information provided from the server30is accepted. The mounted equipment16and the HMD20are coupled by short distance wireless communication or the like, for example. The HMD20and the server30are coupled via a network such as the Internet.

The mounted equipment16includes a detection device10, a laser range scanner17, and a vibrator18that is a vibration mechanism to give vibrations to the mounted equipment16.

The mounted equipment16is mounted on part of the body of a user60. For example, as illustrated inFIG. 2, the mounted equipment16may be mounted on a body trunk60A (for example, waist) of the user60by being fixed to a belt or the like or being directly attached to clothing.

The laser range scanner17is a measurement device of a plane scanning type that measures the distance to an object existing in the surroundings. For example, the laser range scanner17includes an emitting unit that emits light such as laser light with scanning in given directions, a light receiving unit that receives reflected light obtained by reflection of the light emitted from the emitting unit by an object existing in the measurement range, and an output unit that outputs the measurement result.

A measurement range62is a plane defined by an aggregation of vectors64indicating the emission direction of one time of light emission by the emitting unit corresponding to one scan as illustrated inFIG. 2. In the example ofFIG. 2, the case in which the scanning direction of light by the emitting unit of the laser range scanner17is substantially the horizontal direction is illustrated. Therefore, the measurement range62is also defined as a plane along substantially the horizontal direction. Furthermore, as illustrated inFIG. 3, if the scanning direction of light by the emitting unit of the laser range scanner17is substantially the vertical direction, the measurement range62is also defined as a plane along substantially the vertical direction. However, the measurement range62is not limited to the case in which the measurement range62is defined as a plane along substantially the horizontal direction or substantially the vertical direction and may be defined as a plane having an inclination in the horizontal direction or the vertical direction.

Moreover, the output unit calculates a distance d to a position on an object by which emitted light is reflected based on the time from the emission of the light from the emitting unit to reception of the reflected light by the light receiving unit, and acquires an angle θ when the reflected light from the position is incident on the light receiving unit. Then, the output unit outputs data (d, θ) of the combination of the distance d and the angle θ as the measurement result, for example. If M times of light emission are carried out in one time of scanning, the output unit outputs M pieces of data (d, θ) as the measurement result corresponding to the one time of scanning. This measurement result represents the position of the object in the measurement range62.

As illustrated inFIG. 1, the detection device10includes an acquiring unit11, a setting unit12, and a detecting unit13functionally.

The acquiring unit11accepts the measurement result output from the laser range scanner17and transfers the measurement result to the setting unit12.

The setting unit12sets a gesture region66defined with envisaging of a region in which an instructing body that makes an input instruction acts in the measurement range62of the laser range scanner17. For example, suppose that the instructing body that makes an input instruction is the right hand part of the user60in the case in which the mounted equipment16is mounted on the body trunk60A of the user60as illustrated inFIG. 2and the measurement range62is along substantially the horizontal direction. In the present embodiment, a range including the region from the upper arm to the fingertips of the user60will be referred to as the hand part. In this case, for example, as illustrated inFIG. 4, the range the right hand part is able to reach from the body trunk60A may be set as the gesture region66.FIG. 4is a schematic diagram when the user60who wears the mounted equipment16and the measurement range62are viewed from above and the right side on the plane of paper is the right hand side of the user60.

Here, for example, as illustrated inFIG. 5, the measurement range62of the laser range scanner17included in the mounted equipment16is poorly affected by change in the posture of the user60if the mounted equipment16is mounted on the body trunk60A. On the other hand, when gesture input is carried out with the hand part of the user60, if the posture of the user60changes, particularly if the posture of the upper body changes, the position at which the gesture is carried out with the hand part is readily affected by the change in the posture. Therefore, the setting unit12does not set the gesture region66fixed with respect to the measurement range62but sets the gesture region66depending on the change in the posture of the user60.FIG. 5is a schematic diagram when the user60who wears the mounted equipment16and the measurement range62are viewed from a lateral side in the case in which the measurement range62is along substantially the vertical direction.

The detecting unit13detects an object existing in the gesture region66as the instructing body that makes an input instruction based on the measurement result of the laser range scanner17and the gesture region66set by the setting unit12. Furthermore, the detecting unit13recognizes a gesture based on the motion of the detected instructing body in the gesture region66. The detecting unit13transmits the input instruction represented by the recognized gesture to the HMD20.

Moreover, when detecting the instructing body in the gesture region66, the detecting unit13causes the vibrator18to vibrate in order to notify the start of gesture recognition.

Details of the setting method of the gesture region66in the setting unit12and the recognition method of the gesture in the detecting unit13will be described later.

As illustrated inFIG. 1, the HMD20includes a display unit21on which various kinds of information are displayed and a control unit22that controls displaying of information to the display unit21. On the display unit21, for example, an operation screen like one illustrated inFIG. 6is displayed based on information transmitted from the server30. When the user60who wears the HMD20and the mounted equipment16makes a gesture in the gesture region66, an input instruction is transmitted from the detecting unit13as described above. When accepting this input instruction, the control unit22carries out display control of the movement of a pointer68displayed on the display unit21, highlighting of a selected item, or the like in accordance with the input instruction, for example. Furthermore, the control unit22transmits information on the selected item to the server30. Moreover, the control unit22accepts information newly transmitted from the server30according to the selected item and carries out display control of the display unit21.

The server30is an information processing device such as a personal computer or a server device.

The detection device10included in the mounted equipment16may be implemented by a computer40illustrated inFIG. 7, for example. The computer40includes a central processing unit (CPU)41, a memory42as a temporary storage area, and a non-volatile storing unit43. Furthermore, the computer40includes an input-output device44, a read/write (R/W) unit45that controls reading and writing of data from and to a recording medium49, and a communication interface (I/F)46. The CPU41, the memory42, the storing unit43, the input-output device44, the R/W unit45, and the communication I/F46are coupled to each other via a bus47.

The storing unit43may be implemented by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. In the storing unit43as a storage medium, a detection program50for causing the computer40to function as the detection device10is stored. The detection program50includes an acquisition process51, a setting process52, and a detection process53.

The CPU41reads out the detection program50from the storing unit43and loads the detection program50into the memory42to sequentially execute the processes the detection program50has. The CPU41operates as the acquiring unit11illustrated inFIG. 1by executing the acquisition process51. Furthermore, the CPU41operates as the setting unit12illustrated inFIG. 1by executing the setting process52. Moreover, the CPU41operates as the detecting unit13illustrated inFIG. 1by executing the detection process53. This causes the computer40that executes the detection program50to function as the detection device10.

It is also possible that functions implemented by the detection program50are implemented by a semiconductor integrated circuit for example, an application specific integrated circuit (ASIC) or the like for more detail.

Next, operation of the gesture input system100according to the first embodiment will be described. The user60wears the mounted equipment16and the HMD20. Then, when an application offered by the gesture input system100is activated, information representing an operation screen is transmitted from the server30to the HMD20and the operation screen is displayed on the display unit21of the HMD20. Then, measurement and output of the measurement result by the laser range scanner17included in the mounted equipment16are started and detection processing illustrated inFIG. 8is executed in the detection device10.

First, in a step S11, the acquiring unit11accepts a measurement result output from the laser range scanner17and transfers the measurement result to the setting unit12.

Next, in a step S12, the setting unit12identifies the measurement range62of the laser range scanner17based on the measurement result of the laser range scanner17. For example, the setting unit12identifies whether the measurement range62of the laser range scanner17is the measurement range62along the horizontal direction like that illustrated inFIG. 2or the measurement range62along the vertical direction like that illustrated inFIG. 3.

Next, in a step S13, the setting unit12estimates the posture of the user60based on the measurement result of the laser range scanner17. For example, the setting unit12estimates the posture of the user60based on the position of a region60B that is part of the body of the user60detected in the measurement range62and is other than the region serving as the instructing body. As the region60B of the user60, the left hand part or part of the body trunk60A (for example, waist) may be employed if the mounted equipment16is mounted on the body trunk60A and the measurement range62is along the horizontal direction and the instructing body is the right hand part, for example. Furthermore, as the region60B of the user60, the head or part of the body trunk60A (for example, chest) may be employed if the mounted equipment16is mounted on the body trunk60A and the measurement range62is along the vertical direction, for example. The measurement result of the laser range scanner17indicates the position of an object existing in the measurement range62. In addition, from a succession of the position, the shape of the object surface on the side of the laser range scanner17may also be recognized. Therefore, the setting unit12identifies the region60B from the inside of the measurement range62based on this shape of the object surface and estimates the position of the identified region60B in the measurement range62as the posture of the user60.

Next, in a step S14, the setting unit12sets the gesture region66based on parameters defined in advance in order to set the gesture region66as illustrated inFIG. 9and the posture of the user estimated in the above-described step S13, for example.

In the example of the parameters represented inFIG. 9, it is defined that the position of the region60B when a sensor 0 point representing one limit point of the scanning direction of the laser range scanner17is defined as 0° is employed as a reference angle Th0. The position of the region60B is displaced relative to the sensor 0 point and thus the reference angle Th0 is a variable. Furthermore, in the example of the parameters represented inFIG. 9, an angle (near region end angle) Th_a from the reference angle Th0 to the near end part of the gesture region66and an angle (far region end angle) Th_b to the far end part are defined. Moreover, in the example of the parameters represented inFIG. 9, a distance (near region distance) N from the laser range scanner17to the near end part of the gesture region66and a distance (far region distance) F to the far end part are defined. InFIG. 10, the relationship among the laser range scanner17, the sensor 0 point, and the parameters Th0, Th_a, Th_b, N, and F is illustrated.

Here, when the posture of the user60changes, the region60B of the user60with respect to the sensor 0 point is also displaced. Therefore, by employing a variable according to the region60B as the reference angle Th0 for defining the gesture region66, when the posture of the user60changes, the position of the set gesture region66also changes as illustrated inFIG. 10.

Furthermore, the proper setting position of the gesture region66differs depending on to what region of the user60and toward which direction the mounted equipment16including the laser range scanner17is attached. Therefore, a table like that illustrated inFIG. 9is prepared for each of the measurement ranges62corresponding to patterns different from each other in the attachment position and attachment direction of the mounted equipment16. Then, the parameters Th0, Th_a, Th_b, N, and F for identifying the optimum gesture region66when the mounted equipment16is attached with the pattern corresponding to a respective one of the measurement ranges62are defined for each of the measurement ranges62corresponding to a respective one of the patterns. Furthermore, the setting unit12selects the table corresponding to the measurement range62identified in the step S12to acquire the parameters, and sets the gesture region66based on the acquired parameters.

Next, in a step S15, the detecting unit13determines whether or not an object exists in the gesture region66based on the measurement result of the laser range scanner17and the gesture region66set in the above-described step S14. The detecting unit13determines that an object exists in the gesture region66if a position included in the gesture region66defined by the above-described parameters exists among positions represented by plural pieces of data (d, θ) as measurement results of the laser range scanner17. For example, suppose that the gesture region66is defined with Th0=30 degrees, Th_a=40 degrees, Th_b=90 degrees, N=20 cm, and F=60 cm. In this case, if data of (d, θ)=(40 cm, 80 degrees) exists in measurement results of the laser range scanner17, the position represented by (d, θ) is in the gesture region66and thus the detecting unit13determines that an object exists in the gesture region66. θ is an angle of the clockwise direction from the sensor 0 point.

Then, if an object exists in the gesture region66, the detecting unit13detects the object as an instructing body70that makes an input instruction and the processing makes transition to a step S16. On the other hand, if an object does not exist in the gesture region66, the processing returns to the step S11.

In the step S16, the detecting unit13temporarily stores the detection result of the above-described step S15in a given storage area. In this storage area, detection results of a given time are stored. The detection result of the object is represented as one shape like a heavy line part in an ellipse A inFIG. 11through succession of plural measurement results (d, θ). Therefore, the detecting unit13stores the measurement result group representing this one shape as the detection result representing one instructing body70. Furthermore, if plural instructing bodies70are detected in the gesture region66, identification information is given to each instructing body70and the detection result is stored about each of the instructing bodies70.

Next, in a step S17, the detecting unit13causes the vibrator18to vibrate in order to notify the start of gesture recognition.

Next, in a step S18, the detecting unit13recognizes whether or not the motion of the instructing body70is a gesture defined in advance as an input instruction to the operation screen displayed on the display unit21of the HMD20, based on time-series change in the detection result of the instructing body70stored in the given storage area.

As gestures of the input instruction, a gesture of a direction instruction, gestures of a tap and a double tap, and so forth may be defined, for example. The recognition method of the respective gestures will be described below.

InFIG. 12, one example of gesture recognition is schematically illustrated. The example ofFIG. 12represents that the instructing body70is a hand part with a pointing pose and the instructing body70enters the gesture region66at point A (72B) from the state in which the instructing body70has not entered the gesture region66(72A). Furthermore, the example ofFIG. 12represents that the instructing body70moves from point A to point B in the gesture region66(72C→72D) and exits from the gesture region66at point B (72E). Moreover, in72C ofFIG. 12, it is represented that the size of the detected instructing body70is large compared with in72B. This represents that, first, a small section (equivalent to the heavy line part in the ellipse A inFIG. 11) is detected when the fingertip enters the gesture region66and then a region such as a wrist, whose section is larger than the fingertip, is detected through further advancement of the hand part in such a direction as to pass through the gesture region66. As the size of the section, the number of measurement results included in the measurement result group stored as the detection result of one instructing body70, the length of the shape represented by the measurement result group, the area of a sectional shape estimated from the shape represented by the measurement result group, and so forth may be used. In addition, in72E ofFIG. 12, it is represented that the size of the detected instructing body70is small compared with in72D.

Suppose that the detection result of the instructing body70stored in the given storage area represents time-series change of72A→72B→72C→72D→72E→72A inFIG. 12(flow of white block arrows inFIG. 12). In this case, the detecting unit13may recognize that the motion of the instructing body70is a gesture of a direction instruction between point A and point B. Furthermore, suppose that the detection result of the instructing body70represents time-series change of72A→72B→72C→72B→72A inFIG. 12(flow of hatched block arrows inFIG. 12). In this case, the detecting unit13may recognize that the motion of the instructing body70is a gesture of a tap at point A. Furthermore, if a similar tap action is repeated twice in a short time, the detecting unit13may recognize the motion of the instructing body70as a double tap.

If plural instructing bodies70have been detected in the step S15, the position and size of the instructing body70are compared between the detection result of the previous time and the detection result of the present time, and the instructing bodies70estimated to be the same are associated between the times and are given the same identification information. Then, the motion of each instructing body70is identified from time-series change in the detection result given the same identification information.

If the detecting unit13recognizes a gesture of an input instruction, the processing makes transition to a step S19. If a gesture of an input instruction is not recognized, the processing returns to the step S11.

In the step S19, the detecting unit13transmits the input instruction represented by the gesture recognized in the above-described step S18to the HMD20and the processing returns to the step S11.

Due to this, in the HMD20, the control unit22carries out display control of the movement of the pointer68displayed on the display unit21, highlighting of a selected item, or the like based on the input instruction accepted from the detecting unit13, for example. Then, the control unit22transmits information on the selected item to the server30.

The server30transmits information according to the item selected by the user60to the HMD20based on the information accepted from the control unit22. In the HMD20, the control unit22accepts the newly-transmitted information and carries out display control of the display unit21based on the accepted information.

As described above, according to the gesture input system100in accordance with the first embodiment, the user60wears the mounted equipment16including the laser range scanner17. Furthermore, the detection device10included in the mounted equipment16sets, as the gesture region66, a region in which an instructing body70that makes an input instruction is conceived to make a gesture in the measurement range62of the laser range scanner17. Moreover, the detection device10detects an object existing in the set gesture region66as the instructing body70and recognizes a gesture representing an input instruction based on the motion of the instructing body70in the gesture region66. Due to this, even when an object other than the instructing body70or the instructing body70that does not intend a gesture of an input instruction enters the measurement range62of the laser range scanner17, the object or the instructing body70is not detected as the instructing body70that makes an input instruction if it is not in the gesture region66. Therefore, the instructing body70may be stably detected.

Furthermore, according to the detection device10in accordance with the first embodiment, the gesture region66is set at a proper position according to the posture of the user60who wears the mounted equipment16including the laser range scanner17. Therefore, the instructing body70may be stably detected even in work involving posture change.

Second Embodiment

Next, a second embodiment will be described. Regarding a gesture input system according to the second embodiment, the part similar to that of the gesture input system100according to the first embodiment is given the same numeral and detailed description of the part is omitted.

In the first embodiment, description is made about the case in which, if an object exists in the gesture region66set by the setting unit12, the object is detected as the instructing body70that makes an input instruction. In this case, also when an object other than the instructing body70or the instructing body70that does not intend a gesture of an input instruction enters the gesture region66, the object or the instructing body70is detected as the instructing body70that makes an input instruction. If the detected instructing body70is an object other than the instructing body70or the instructing body70that does not intend a gesture of an input instruction, the possibility that these objects make an action similar to a gesture representing an input instruction defined in advance will be low. Therefore, the possibility that a problem of erroneous recognition of a gesture occurs will also be low, and processing of unnecessary gesture recognition occurs regarding the object other than the instructing body70and the instructing body70that does not intend a gesture of an input instruction.

Therefore, in the second embodiment, a target whose gesture is to be recognized as the instructing body70among objects that have entered the gesture region66is limited so that processing of such unnecessary gesture recognition is reduced.

As illustrated inFIG. 1, a gesture input system200according to the second embodiment includes mounted equipment216, the HMD20, and the server30. The mounted equipment216includes a detection device210, the laser range scanner17, and the vibrator18. The detection device210functionally includes the acquiring unit11, a setting unit212, and a detecting unit213.

The setting unit212sets the gesture region66similarly to the setting unit12according to the first embodiment. Furthermore, as illustrated inFIG. 13, the setting unit212sets a partial region in contact with the gesture region66as a gesture start preparation region74. The gesture start preparation region74is a region for determining to start gesture recognition by the detecting unit213when the instructing body70passes through this region and enters the gesture region66. Therefore, a region through which entry into the gesture region66is difficult for an object other than the instructing body70and the instructing body70that does not intend a gesture of an input instruction is defined as the gesture start preparation region74.

For example, suppose that a range the right hand part is able to reach from the body trunk60A is set as the gesture region66as illustrated inFIG. 4. In this case, from the far side and the right side of the gesture region66as viewed from the body trunk60A, an object other than the instructing body70will readily enter the gesture region66. Furthermore, from the near side of the gesture region66, the instructing body70that does not intend a gesture of an input instruction will readily enter the gesture region66due to a swing of a hand in normal walking or the like. Therefore, the setting unit212may set the gesture start preparation region74at the left end part of the gesture region66as illustrated inFIG. 13, for example.

Furthermore, as illustrated inFIG. 13, the setting unit212sets at least a partial region in contact with the gesture region66as a gesture end region76. The gesture end region76is a region for determining the end of the gesture recognition by the detecting unit213when the instructing body70moves from the gesture region66to this region. For example, the setting unit212may set the gesture end region76around the gesture region66as illustrated inFIG. 13.

For example, as illustrated inFIG. 14, parameters for setting each of the gesture start preparation region74and the gesture end region76are defined in addition to the parameters for setting the gesture region66. In the example ofFIG. 14, it is defined that a region with a width S having the end part of the gesture region66on the side closer to the sensor 0 point as one side is employed as the gesture start preparation region74. Furthermore, it is defined that a region that is a region outside the gesture region66and corresponds to a margin E from the gesture region66is employed as the gesture end region76.

In the case of using the parameters ofFIG. 14, the setting unit212sets the gesture region66based on the parameters Th0, Th_a, Th_b, N, and F similarly to the setting unit12in the first embodiment. Furthermore, based on the set gesture region66, the setting unit212sets each of the gesture start preparation region74and the gesture end region76based on each of the parameters S and E.

The detecting unit213detects, as the instructing body70, an object that passes through the gesture start preparation region74set by the setting unit212and enters the gesture region66. Then, the detecting unit213carries out gesture recognition regarding the detected instructing body70similarly to the detecting unit13in the first embodiment. Furthermore, the detecting unit213ends the recognition of a gesture and the detection of the instructing body70if the instructing body70moves from the gesture region66to the gesture end region76.

For example, as illustrated inFIG. 15, if the position of an object represented by the measurement result makes time-series change of 1→2→3→4, the detecting unit213detects this object as the instructing body70and recognizes a gesture from time-series change in the detection result between 2 and 4. Furthermore, if change in the position of an object represented by the measurement result is 5→6→7, the detecting unit213does not detect this object as the instructing body70because the object does not pass through the gesture start preparation region74when entering the gesture region66.

The detection device210included in the mounted equipment16may be implemented by the computer40illustrated inFIG. 7, for example. In the storing unit43of the computer40, a detection program250for causing the computer40to function as the detection device210is stored. The detection program250includes the acquisition process51, a setting process252, and a detection process253.

The CPU41reads out the detection program250from the storing unit43and loads the detection program250into the memory42to sequentially execute the processes the detection program250has. The CPU41operates as the acquiring unit11illustrated inFIG. 1by executing the acquisition process51. Furthermore, the CPU41operates as the setting unit212illustrated inFIG. 1by executing the setting process252. Moreover, the CPU41operates as the detecting unit213illustrated inFIG. 1by executing the detection process253. This causes the computer40that executes the detection program250to function as the detection device210.

It is also possible that functions implemented by the detection program250are implemented by a semiconductor integrated circuit for example, an ASIC or the like for more detail.

Next, operation of the gesture input system200according to the second embodiment will be described. In the second embodiment, detection processing illustrated inFIG. 16is executed in the detection device210. Regarding the detection processing in the second embodiment, the processing similar to the detection processing (FIG. 8) in the first embodiment is given the same numeral and detailed description of the processing is omitted.

First, the steps S11to S14are carried out and the gesture region66is set in the measurement range62. Then, in the next step S21, the setting unit212sets the gesture start preparation region74and the gesture end region76.

Next, in a step S22, the detecting unit213determines whether or not an object exists in the gesture start preparation region74based on the measurement result of the laser range scanner17and the gesture start preparation region74set in the above-described step S21. If an object exists in the gesture start preparation region74, the processing makes transition to a step S23and the detecting unit213sets a preparation flag F1 indicating that an object has entered the gesture start preparation region74to “ON,” and the processing returns to the step S11.

On the other hand, if an object does not exist in the gesture start preparation region74, the processing makes transition to the step S15and the detecting unit213determines whether or not an object exists in the gesture region66. If an object exists in the gesture region66, the processing makes transition to a step S24. In the step S24, the detecting unit213sets a gesture region flag F2 indicating that an object exists in the gesture region66to “ON,” and the processing makes transition to a step S25.

In the step S25, the detecting unit213determines whether or not the preparation flag F1 is “ON.” In the case of F1=“ON,” the preparation flag F1 indicates that the object has passed through the gesture start preparation region74and has entered the gesture region66. Therefore, the detecting unit213detects the object as the instructing body70that makes an input instruction and carries out gesture recognition in the subsequent steps S16to S19. On the other hand, in the case of F1≠“ON,” the preparation flag F1 indicates that the object has entered the gesture region66without passing through the gesture start preparation region74. Therefore, the detecting unit213regards the object as an object other than the instructing body70or the instructing body70that does not intend a gesture of an input instruction, and returns to the step S11without carrying out gesture recognition.

Furthermore, if the negative determination is made in the step S15, the processing makes transition to a step S26. In the step S26, the detecting unit213determines whether or not an object exists in the gesture end region76based on the measurement result of the laser range scanner17and the gesture end region76set in the above-described step S21. If an object exists in the gesture end region76, the processing makes transition to a step S27.

In the step S27, the detecting unit213determines whether or not the gesture region flag F2 is “ON.” In the case of F2=“ON,” the gesture region flag F2 indicates that the instructing body70that has existed in the gesture region66has moved to the gesture end region76, and it may be determined that the end of a gesture is intended. Therefore, the processing makes transition to a step S28and the detecting unit213sets both the flags F1 and F2 to “OFF.” Furthermore, in a step S29, the detecting unit213stops the vibrator18in actuation and the processing returns to the step S11.

On the other hand, in the case of F2≠“ON,” the object has not moved from the gesture region66to the gesture end region76and recognition processing of a gesture is not currently being executed. Thus, the processing returns to the step S11without execution of the processing of the steps S28and S29.

Furthermore, in the case of the negative determination in the step S26, the object as the processing target does not exist in the measurement range62and thus the processing returns to the step S11.

As described above, according to the gesture input system200in accordance with the second embodiment, the detection device210included in the mounted equipment16sets the gesture start preparation region74adjacent to the gesture region66. Furthermore, the detection device210executes processing of gesture recognition regarding the instructing body70that has passed through the gesture start preparation region74and has entered the gesture region66. This may reduce processing of unnecessary gesture recognition in the case in which an object other than the instructing body70or the instructing body70that does not intend a gesture of an input instruction enters the gesture region66.

Third Embodiment

Next, a third embodiment will be described. Regarding a gesture input system according to the third embodiment, the part similar to that of the gesture input system100according to the first embodiment is given the same numeral and detailed description of the part is omitted.

As illustrated inFIG. 17, a gesture input system300according to the third embodiment includes mounted equipment316, the HMD20, and the server30. The mounted equipment316includes a detection device310, the laser range scanner17, and the vibrator18. The detection device310functionally includes an acquiring unit311, the setting unit12, the detecting unit13, and an environment recognizing unit14.

The acquiring unit311accepts a measurement result output from the laser range scanner17and transfers the measurement result to the setting unit12. In addition, the acquiring unit311transfers the measurement result also to the environment recognizing unit14.

The environment recognizing unit14recognizes the surrounding environment of the user60based on the measurement result of the laser range scanner17. For the environment recognition, measurement results of the whole of the measurement range62are used. Furthermore, if a hazardous place defined in advance is included in the recognized surrounding environment, the environment recognizing unit14vibrates the vibrator18in order to inform the user60of the existence of the hazardous place.

As the hazardous places, a step in a floor, an obstacle existing in the traveling direction, and so forth are envisaged, for example. In the measurement result of the laser range scanner17, the shapes of objects existing in the surroundings may be recognized. Thus, patterns of the shapes representing the hazardous places are defined in advance. Furthermore, the environment recognizing unit14may detect the hazardous places by comparing the measurement result of the laser range scanner17and the patterns defined in advance. Moreover, for example, the value of the measurement result suddenly changes at a step part in a floor as illustrated in a part of an ellipse B inFIG. 18. Thus, the environment recognizing unit14may detect the hazardous places based on such a change in the value of the measurement result.

The detection device310included in the mounted equipment16may be implemented by the computer40illustrated inFIG. 7, for example. In the storing unit43of the computer40, a detection program350for causing the computer40to function as the detection device310is stored. The detection program350includes an acquisition process351, the setting process52, the detection process53, and an environment recognition process54.

The CPU41reads out the detection program350from the storing unit43and loads the detection program350into the memory42to sequentially execute the processes the detection program350has. The CPU41operates as the acquiring unit311illustrated inFIG. 17by executing the acquisition process351. Furthermore, the CPU41operates as the environment recognizing unit14illustrated inFIG. 17by executing the environment recognition process54. The other processes are similar to the detection program50according to the first embodiment. This causes the computer40that executes the detection program350to function as the detection device310.

It is also possible that functions implemented by the detection program350are implemented by a semiconductor integrated circuit for example, an ASIC or the like for more detail.

Next, operation of the gesture input system300according to the third embodiment will be described. In the third embodiment, in the detection device310, the detection processing similar to the detection processing (FIG. 8) in the first embodiment is executed and environment recognition processing illustrated inFIG. 19is executed.

First, in a step S31, the acquiring unit311accepts a measurement result output from the laser range scanner17and transfers the measurement result to the environment recognizing unit14. Next, in a step S32, the environment recognizing unit14recognize the surrounding environment of the user60based on the measurement result of the laser range scanner17. Next, in a step S33, the environment recognizing unit14determines whether or not a hazardous place defined in advance is included in the recognized surrounding environment. If a hazardous place is included in the surrounding environment, the processing makes transition to a step S34and the vibrator18is vibrated in order to inform the user60of the existence of the hazardous place. Then, the processing returns to the step S31. On the other hand, if a hazardous place is not included in the surrounding environment, the processing returns to the step S31without execution of the step S34.

As described above, according to the gesture input system300in accordance with the third embodiment, the configuration used for gesture recognition may be used also for recognition of the surrounding environment of the user60.

In the third embodiment, the case in which hazardous places are detected based on the recognized surrounding environment is described. However, the configuration is not limited to the case. For example, the surrounding environment recognized from a measurement result of the laser range scanner17may be collated with known environment data to estimate the position of the user60in the environment.

Furthermore, in the third embodiment, an example of the detection device310obtained by adding the environment recognizing unit14to the detection device10according to the first embodiment is described. However, a configuration obtained by adding the environment recognizing unit14to the detection device210according to the second embodiment may be employed.

Furthermore, in the above-described respective embodiments, the case in which the laser range scanner17of a plane scanning type is used is described. However, the configuration is not limited to the case. A laser range scanner of a three-dimensional scanning type that emits light while an emitting unit obtained by arranging plural light sources in the direction orthogonal to the scanning direction is scanned in the scanning direction may be used. In this case, the gesture region66may also be set as a three-dimensional region.

In addition, in the above-described respective embodiments, the case of a hand part of the user60is described as one example of the instructing body70that makes an input instruction. However, the instructing body70may be another region of the user60such as a foot. Furthermore, if the user60makes gesture input while holding an instructing bar or the like, the instructing bar may be detected as the instructing body70.

Moreover, in the above-described respective embodiments, the case in which the posture of a user60is estimated by using a measurement result of the laser range scanner17is described. However, the configuration is not limited to the case. A posture sensor consisting of an acceleration sensor, a gyro sensor, or the like may be mounted on the user60and the posture of the user60may be estimated based on a sensor value detected by the posture sensor. The posture sensor may be mounted on the user60separately from the mounted equipment16or a configuration in which the posture sensor is included in the mounted equipment16may be employed.

Furthermore, in the above-described respective embodiments, the case in which the mounting position of the mounted equipment16is the body trunk60A (waist) of the user60is described. However, the mounted equipment16may be mounted on another region such as the head, the chest, or an arm. However, in the head, an arm, or the like, the flexibility in the region itself (movable range when the position of the user60is fixed) is high. Therefore, when the mounted equipment16is mounted, variation in the positional relationship between the mounted equipment16and the position at which the instructing body70makes a gesture (for example, position the right hand is able to reach) also becomes large. In the case of mounting the mounted equipment16on such a region having high flexibility, the gesture region66is set in consideration also of variation in the position at which the mounted equipment16is mounted. If the mounted equipment16is mounted on the body trunk60A as in the above-described embodiments, variation in the position at which the mounted equipment16is mounted is small and thus the instructing body70may be detected more stably.

In the above-described respective embodiments, the modes in which the detection programs50,250, and350are stored (installed) in the storing unit43in advance are described. However, the configuration is not limited to the modes. It is also possible to provide the detection programs according to the disclosed techniques in a form of being recorded on a recording medium such as a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD)-ROM, or a universal serial bus (USB) memory.