Object-detecting device and method of extracting operation object

An object-detecting device detects an operation object for operating an operating apparatus. The object-detecting device includes an imaging portion for imaging the operation object and an image processor for extracting the operation object. The operation object has a first face, on which outside light falls from outside into the operating apparatus, and a second face opposite to the first face. The imaging portion images the second face of the operation object. The object-detecting device can stably detect the operation object, even when an illuminance variation of outside light is large.

CROSS REFERENCE TO RELATED APPLICATION

This application is based on Japanese Patent Application No. 2006-97923 filed on Mar. 31, 2006, the disclosure of which is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to an object-detecting device and a method of extracting an operation object.

BACKGROUND OF THE INVENTION

JP-A-2000-6687 discloses a safety operation system. When a driver operates an apparatus, e.g., vehicle navigation device, in a vehicle, the system extracts an operation object, e.g., finger of the driver or touch pen, by using an image processing. The extracted operation object is displayed on a display screen, and overlaps with other images on the display screen. Thus, the driver can safely operate the apparatus, because the driver can see the operation object on the display screen in front of the driver.

JP-A-6-84081 discloses a method of extracting the operation object using a chromakey image. In the method, a color gain of a video signal is variable. After the color gain of the video signal is controlled, the chromakey image having a predetermined color is extracted. Then, the extracted chromakey image is determined to have a predetermined shape or not. When the extracted chromakey image has the predetermined shape, the operation object can be extracted.

However, outside light passing through a window of the vehicle may change its brightness and color. Therefore, the operation object may not stably be detected, because the brightness and the color in the vehicle are changed by direct sunlight, shadow, or tunnel lighting, when a certain brightness and a certain color are used in the method. Further, in order to deal with an illuminance variation of outside light, a camera having a wide sensitivity range is used, or lens aperture is controlled in a wide range. Thus, cost of the safety operation system may be increased.

SUMMARY OF THE INVENTION

In view of the foregoing and other problems, it is an object of the present disclosure to provide an object-detecting device and a method of extracting an operation object for an operating apparatus such as a vehicle navigation system, a personal computer and so on.

According to a first aspect of the disclosure, an object-detecting device detects an operation object, which is used for operating an operating apparatus. The object-detecting device includes an imaging portion for imaging the operation object, and an image processor for extracting the operation object. The operation object has a first face, on which outside light falls from outside into the operating apparatus, and a second face opposite to the first face. The imaging portion images the second face of the operation object.

According to a second aspect of the disclosure, a method of extracting an operation object for operating an apparatus includes a first imaging step, a second imaging step and a comparing step. The operation object has a first face, on which outside light falls, and a second face opposite to the first face. In the first imaging step, a first image of the operation object is imaged, when a controller turns on a lighting portion to light the second face of the operation object. In the second imaging step, a second image of the operation object is imaged, when the controller turns off the lighting portion. In the comparing step, the first image and the second image are compared so as to extract the operation object.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

First Embodiment

As shown inFIG. 1, a vehicle navigation system1is mounted in a vehicle compartment, and includes a display screen2and an object-detecting device5.

The display screen2displays information, e.g., a subject vehicle position on a map, or a road traffic information, for a driver A of the vehicle. The display screen2is made of liquid crystal, for example, and positioned at a front part of the compartment such that the driver A can easily look at the screen2.

The object-detecting device5extracts an external shape (outline) of a finger B (hand) of the driver A by using an image processing, when the driver A operates the vehicle navigation system1. As shown inFIG. 2, the object-detecting device5includes an imaging portion12, a lighting portion14, a controller16, an image memory18and an operation panel20. An image processor is composed of the controller16, the image memory18and the like.

The imaging portion12obtains an infrared image. Specifically, the imaging portion12includes a mirror for reflecting infrared light, a lens for collecting infrared light, a light wavelength conversion element for converting infrared light into visible light, and an imaging element (tube) for imaging a visible image. The light wavelength conversion element is made of a nonlinear optical material. The imaging element may be separated from optical parts such as the mirror or the lens.

The imaging portion12has a focus, which is spaced from a top face of the operation panel20toward the finger B (upside). Outside light falls on a top face of the finger B of the driver A. The imaging portion12is positioned at a bottom side of the finger B of the driver A so as to image the top face of the panel20. Specifically, the imaging portion12is located at a bottom side of the operation panel20. Alternatively, only the optical parts of the imaging portion12may be located at the bottom side of the finger B. In this case, the imaging portion12may not be located at the bottom side of the finger B.

The finger B has the top face, on which outside light falls, and a bottom face, on which outside light does not fall. A brightness change of the bottom face of the finger B is small, even when an illuminance of outside light changes.

The lighting portion14emits infrared light. For example, an infrared light-emitting diode (LED) is used for the lighting portion14, and directly emits infrared rays. Alternatively, an illuminant, e.g., incandescent lamp, Nernst glow lamp, globe, arc-discharge tube or tungsten lamp, covered with an infrared rays transmittable filter may be used for the lighting portion14. The lighting portion14is disposed in the device5such that infrared light is emitted to the finger B facing the operation panel20. The lighting portion14controls the brightness of the bottom face of the finger B.

The image memory18stores images captured by the imaging portion12. The imaging portion12captures a first image (light image), when the lighting portion14is lighting. The imaging portion12captures a second image (non-light image), when the lighting portion14is not lighting. The image memory18has a memory size, in which one or more first images and one or more second images can be stored.

The driver A touches the operation panel20by the finger B without looking at the operation panel20, in order to operate the vehicle navigation system1. For example, the driver A inputs a target destination into the operation panel20, or the driver A changes a setting (e.g., display magnification) of the display screen2through the operation panel20. The display magnification of the display screen2can be increased or reduced through the operation panel20. The operation panel20is made of a material capable of transmitting infrared rays, e.g., acrylic board.

An operation object sensor22detects the finger B of the driver A, when the finger B is at an operation position. The operation position represents a position, at which the finger B can touch the operation panel20. Specifically, the sensor22is made of a micro switch or pressure-sensitive sheet. The sensor22is adjacent to the operation panel20. When the driver A touches the operation panel20by the finger B, the driver A pushes the sensor22by the palm or the wrist. The sensor22may be disposed on a palm rest. When the driver A touches the operation panel20by the finger B, the sensor22is turned on.

The controller16controls the imaging portion12, the lighting portion14and the image memory18, and is made of a CPU, a ROM, a RAM and an I/O (not shown). For example, the controller16controls a timing for imaging by the imaging portion12, and an optical condition such as a lens aperture. The sensor22outputs a signal into the controller16.

When the sensor22is turned on, that is when the driver A touches the operation panel20by the finger B, the controller16captures an image of the finger B, based on a differential image between the first image and the second image stored in the image memory18.

More specifically, the controller16actuates the lighting portion14, captures the first image from the imaging portion12, and stores the first image into the image memory18. Then, the controller16stops the lighting portion14, captures the second image from the imaging portion12, and stores the second image into the image memory18. Thus, the controller16captures the image of the finger B based on the differential image between the first image and the second image.

Here, a principle for obtaining the image of the finger B based on the differential image will be described with reference toFIGS. 3 and 4. A first curve a represents a brightness of the finger B, when outside light and artificial light by the lighting portion14exist. A second curve β represents the brightness of the finger B, when only the outside light exists. A third curve γ represents the brightness of the finger B, when only the artificial light by the lighting portion14exists.

As shown inFIG. 3, the third curve γ has a constant brightness, because outside light does not exist. The brightness of the second curve β is increased, as the illuminance of outside light increases, due to reflection of outside light. The brightness of the first curve α corresponds to a sum of the brightnesses of the second curve β and the third curve γ. The first curve a has a constant luminance, when the illuminance of outside light is small, similarly to the third curve γ. The brightness of the first curve α is increased, as the illuminance of outside light increases. Thus, the brightness of the finger B has a small sensitivity range SR, as shown inFIG. 3.

Therefore, as shown inFIG. 4A, when the illuminance of outside light is large, the first image (i.e., light-on image) has a slight image of the finger B, and the second image (i.e., light-off image) has a strong image of the finger B as a shadow. Thus, the differential image of the finger B can be provided by a difference between the first and second images. In contrast, as shown inFIG. 4B, when the illuminance of outside light is small, the first image (i.e., light-on image) has a clear image of the finger B, and the second image (i.e., light-off image) has no image of the finger B. Thus, the differential image of the finger B can be provided by a difference between the first and second images.

Further, the controller16controls the display screen2to display the differential image of the finger B, which overlaps with other displays used for operating the vehicle navigation system1. In addition, the controller16performs a variety of processes, e.g., route searching process, route introducing process or displaying process, for controlling the vehicle navigation system1, other than the control of the object-detecting device5.

When the driver A touches the operation panel20by the finger B, the sensor22is turned on. Then, the controller16controls the lighting portion14and the imaging portion12so as to capture the first and second images. The first and second images are stored in the image memory18. Based on the differential image between the first and second images, the controller16captures the image of the finger B of the driver A. The image of the finger B of the driver A is displayed on the display screen2, together with other displays used for operating the vehicle navigation system1.

The imaging portion12is located at a palm side of the driver A. That is, the imaging portion12is positioned at a bottom side of the finger B of the driver A. Outside light does not fall on the bottom side of the finger B of the driver A. When outside light makes a shadow of the finger B at the palm side of the driver A, the imaging portion12images the shadow as the second image. When the lighting portion14lights the finger B from the palm side of the driver A, the imaging portion12captures the first image. The image of the finger B is obtained by the differential image between the first and second images.

Therefore, even when the illuminance of outside light changes, control range (e.g., sensitivity range or lens aperture range) of optical parts in the imaging portion12can be made narrow, because the device5extracts the image of the finger B based on the differential image. The change of the shadow of the finger B is small.

When the sensitivity range is made narrow, an imaging tube or an imaging element in the imaging portion12can be inexpensive, small and light. When the control range of the optical parts such as the lens aperture range is made narrow, the optical parts can be inexpensive, small and light. Thus, the object-detecting device5can be inexpensive, small and light.

Because the device5includes the image memory18, the first and second images can be separately captured with a time lag between the first and second images. Therefore, the single device5can provide the differential image. Thus, the device5can be inexpensive, small and light.

Because the focus of the imaging portion12is spaced upward from the top face of the operation panel20, an image captured by the imaging portion12has a defocus. Due to the defocus, a flaw (scratch) or a dust on the operation panel20is not captured as the image, because the flaw and the dust are small relative to the finger B. Thus, the device5can accurately extract the image of the finger B.

The imaging portion12captures the infrared image. The driver A cannot see infrared rays. Therefore, infrared rays do not affect a safe driving by the driver1, even when infrared rays successively blink.

Only when the finger B is at the operation potion, the lighting portion14is turned on, and the imaging portion12captures an image. That is, when the driver A does not operate the panel20, the imaging portion12and the lighting portion14are not actuated. Thus, consumption energy can be reduced.

The imaging portion12is constructed with the optical parts and the imaging elements. Therefore, the imaging portion12can capture a clear image, because the optical parts and the imaging elements can set an appropriate condition for taking the clear image.

Second Embodiment

The imaging portion12includes a low pass filter (not shown). The imaging portion12has a focus on a top face of the operation panel20in the second embodiment. When the imaging portion12outputs an image as an output signal, the low pass filter smoothes the output signal. That is, the low pass filter does not transmit the output signal having a frequency equal to or higher than a predetermined frequency.

Here, the predetermine frequency is set based on a characteristic of noise to be removed. For example, as shown inFIG. 5A, a scratch or a dust may be on the operation panel20. A size of the scratch or the dust defines the predetermined frequency. When the size of the scratch or the dust is small, the predetermined frequency is made higher, because noise has a spike shape, as shown inFIG. 5B. In contrast, when the size of the scratch of the dust is large, the predetermined frequency is made lower.

Due to the low pass filter, high-frequency bandwidth signal (noise) such as the scratch or the dust shown inFIG. 5Bcan be removed, as shown inFIG. 5C. Therefore, the output signal can be smoothed. This effect is also provided when the imaging portion12has the focus spaced from the top face of the operation panel20.

As shown inFIG. 6, an object-detecting device7includes an illuminance sensor24in the second embodiment. The illuminance sensor24detects an illuminance of outside light. For example, a photo diode or a phototransistor is used as the illuminance sensor24.

The controller16controls an illuminance of the lighting portion14in accordance with the illuminance of outside light detected by the illuminance sensor24. Specifically, when the illuminance of outside light is large, the illuminance of the lighting portion14is made small. When the illuminance of outside light is small, the illuminance of the lighting portion14is made large. Thus, the brightness of the bottom face of the finger B can be made constant.

When the brightness of the bottom face of the finger B is constant, a control range of the imaging portion12can be made narrow. Thus, the imaging portion12can be inexpensive, small and light. Further, when the illuminance of outside light is large, the illuminance of the lighting portion14can be reduced. Thus, consumption energy can be reduced.

Moreover, the controller16controls the lighting portion14to alternately have a lighting period and a non-lighting period, which are approximately equal to each other. The image memory18alternately stores the first image taken in the lighting period and the second image taken in the non-lighting period. The differential image is provided between the first image and the second image. Then, the next differential image is provided between the second image and the next first image. Thus, the number of the differential images can be increased.

Further, the controller16has a constant interval between the imaging of the first image and the imaging of the second image. In this case, if the finger B is slightly moved, the movement of the finger B in the interval may be small. Therefore, the differential image can be accurately provided.

Third Embodiment

FIG. 7shows a control of the lighting portion14by the controller16. A first differential image is input into the controller16(S100). The controller16binarizes the first differential image (S105). The controller16extracts an outline of the finger B of the driver A from the binarized first differential image (S110).

Then, a second differential image is input into the controller16(S115). The controller16binarizes the second differential image (S120). The controller16extracts an outline of the finger B of the driver A from the binarized second differential image (S125).

The controller16compares positions of the outlines extracted at S110and S125. The controller16determines a position difference between the positions of the outlines is equal to or larger than a predetermined value R or not. When the position difference is equal to or larger than the predetermined value R (YES at S130), the controller16reduces the lighting period and the non-lighting period of the lighting portion14(S135). Then, the controller16returns to S100. When the position difference is smaller than the predetermined value R (NO at S130), the controller16increases the lighting period and the non-lighting period of the lighting portion14(S140). Then, the controller16returns to S100.

Thus, the movement of the finger B of the driver A is detected, and the lighting period and the non-lighting period of the lighting portion14are controlled based on a speed of the movement. When the movement of the finger B is fast, the lighting period and the non-lighting period of the lighting portion14are reduced. Therefore, the finger B can be accurately extracted, because defocus of the differential image is reduced.

Further, the controller16controls timing for imaging the first and second images based on the speed of the movement of the operation object.

Other Embodiments

The object-detecting device5,7extracts the finger B of the driver A in the above embodiments. However, the driver A may use a touch pen, and the object-detecting device5,7may extract the touch pen.

The imaging portion12is constructed with the light wavelength conversion element for converting infrared light into visible light, and the imaging element for imaging the visible image in the above embodiments. Alternatively, the imaging portion12may be constructed with an infrared camera including an imaging element for imaging an infrared image. The imaging element of the infrared camera is a semiconductor element made of InPb or HgCdTe, or a current collecting sensor, e.g., thermocouple or bolometer. Further, the imaging portion12may be constructed with a visible-light camera having the light wavelength conversion element and an infrared filter.

The imaging portion12has a focus spaced from the top face of the operation panel20toward the finger B (upward) in the first embodiment. Alternatively, the imaging portion12may have a focus spaced from the top face of the operation panel20toward the imaging portion12(downward).

The first image of the finger B is taken when the lighting portion14is lighting the finger B, and the second image of the finger B is taken when the lighting portion is not lighting the finger B. However, the illuminance of the lighting portion14may be gradually varied. The imaging portion12may capture images of the finger B in plural illuminance conditions. Then, images appropriate for the image processing may be selected from the images captured in the plural illuminance conditions. For example, an image having a sufficient brightness difference between the finger B and the background can be easily processed. In this case, the finger B can be easily extracted, even when illuminance variation of outside light is large. Further, control range of the optical parts can be reduced, because the illuminance conditions can be determined in advance.

The object-detecting device5,7is used in the vehicle. Specifically, the device5,7is used in a car, a train, a ship, a plane or with a human moving in a space.

The object-detecting device5,7is used for the vehicle navigation system1. However, the device5,7may be used for a ship steering system or a flight controlling system, that is a machinery used in a mobile unit. Further, the device5,7may be used for an entertainment machinery such as a head mounted display (HMD) for a human.

The controller16captures the image of the finger B, based on the differential image between the first image and the second image. However, the image of the finger B may be based on a quotient image between the first image and the second image. For example, the first image is divided by the second image. Further, the image of the finger B may be provided by just comparing the first image and the second image using if statement.

While the invention has been described with reference to a preferred embodiment thereof, it is to be understood that the invention is not limited to the preferred embodiment and constructions. The invention is intended to cover various modification and equivalent arrangements. The invention is intended to cover various combinations and configurations, which are preferred, other combinations and configurations, including more, less or only a single element, are also within the spirit and scope of the invention.