IMAGING APPARATUS

An imaging apparatus includes a gesture sensor that detects a movement of a detection target object near an imaging surface, and a camera that captures an object body placed on the imaging surface, wherein an optical path length of a light beam Sa of the gesture sensor is made shorter than an optical path length of a light beam Ia of the camera (Ia>Sa).

TECHNICAL FIELD

The present invention relates to an imaging apparatus including a first imaging unit for detecting a user movement and a second imaging unit for capturing an object body.

BACKGROUND ART

A user interface system that recognizes a gesture of a user on a projector-projected video to allow the user to perform an intuitive operation is used. A system like this recognizes a user's gesture on a projected video using a touch panel or a video recognition technology.

US2014/0292647 discusses an interactive projector that projects a video from a projection unit onto an object to be projected such as a table, captures and analyzes a hand movement of a user for a projected image with a first camera, and projects an image corresponding to the hand movement from the projection unit onto a projection surface. To record character information placed on the projection surface, a second camera is used to capture the character information for recording it as an image.

A depth of field is used as an index for determining whether a camera can read an object body correctly. The greater the depth of field is, the wider is the range for bringing the object body into focus. One way to increase the depth of field of a camera is to extend an optical path length of the camera. When an image to be captured by a camera is a document script, a greater depth of field is required so that characters included in a whole area of the captured image can be correctly read. This requirement is further increased when optical character reader (OCR) processing is performed for a captured image.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Solution to Problem

According to an aspect of the present invention, an imaging apparatus includes a first imaging unit configured to include a first imaging element, and to detect a movement of a detection target object near an imaging surface, and a second imaging unit configured to include a second imaging element, and to capture an object body placed on the imaging surface, wherein a relation, “an optical path length from the second imaging element to the imaging surface>an optical path length from the first imaging element to the imaging surface” is satisfied.

DESCRIPTION OF EMBODIMENTS

First Exemplary Embodiment

A first exemplary embodiment of the present invention is described in detail below with reference to the attached drawings. The components described in the exemplary embodiment below are only exemplary, and the scope of the present invention is not limited to those components.

<Usage State of Information Processing Apparatus109>

FIG. 1is a schematic diagram illustrating a usage state of an information processing apparatus109which is an imaging apparatus in the exemplary embodiment.

The information processing apparatus109includes a projector106serving as a projection unit, a gesture sensor107serving as a first imaging unit, a camera105serving as a second imaging unit, and a lens barrel132(SeeFIG. 2AandFIG. 4.).

The projector106projects an image111onto a projection surface110(Because an imaging surface301to be described below is equivalent to the projection surface110, only the projection surface110is described).

A user performs an operation for this image111. The projected image111includes a menu button122via which the user uses a finger to select a power ON/OFF operation or other operations. A user's operation selection, which is detected by the gesture sensor107, functions as an interface.

When the user wants to capture a document using the information processing apparatus109, an object body (a document) to be captured is placed on the projection surface110to allow the camera105to capture the document as an image.

In the apparatus itself, a side on which an image is projected is a front side and its opposite side is a back side. The respective sides of the apparatus viewed from the front side are a right side and a left side.

<Description of Information Processing Apparatus109>

FIG. 2Ais a diagram illustrating a hardware configuration of the information processing apparatus109in the present exemplary embodiment. InFIG. 2A, a central processing unit (CPU)101, which is composed of a microcomputer, performs calculation and logic determination for various types of processing, and controls respective components connected to a system bus108. A read only memory (ROM)102is a program memory in which programs for use by the CPU101for controlling operations are stored. A random access memory (RAM)103is a data memory having a work area used by the programs to be executed by the CPU101, a data saving area in which data is saved when an error occurs, and a program loading area in which the control programs are loaded. A storage device104, which is composed of a hard disk drive or an externally connected storage device, stores various types of data, such as electronic data used in the present exemplary embodiment, and programs. The camera105, a second imaging unit, captures a work space where the user performs an operation, and supplies the captured image to a system as an input image. The projector106, serving as a projection unit, projects a video, which includes electronic data and user interface components, onto the work space. The gesture sensor107, serving as a first imaging unit, is a red-green-blue (RGB) or monochrome charge coupled device (CCD) camera. The gesture sensor107detects a movement of a detection target object such as a user's hand in the work space and, based on such detection, detects whether the user has touched an operation button and so on projected on the projection surface110(seeFIG. 1). In the present exemplary embodiment, the projection surface110is a flat surface below the information processing apparatus such as a surface of a table on which the information processing apparatus109is placed. Another configuration is also possible. For example, the projection surface110may be provided as a part of the information processing apparatus109so that an image from the projector106can be projected thereon.

FIG. 2Bis diagram illustrating a functional configuration of the information processing apparatus109in the present exemplary embodiment. InFIG. 2B, the camera105captures an object body, such as a document hand-written by the user, placed on the projection surface110, and determines characters and such in that document. The projector106projects a screen, such as a user interface, onto the projection surface110(seeFIG. 1). The projector106can also project an image captured by the camera105. The gesture sensor107detects, in the work space on the projection surface110(seeFIG. 1), an operation by a hand of the user on the user interface projected onto the projection surface110by the projector106. When the user interface is operated by the hand of the user, an image projected by the projector106is changed or an image is captured by the camera105. A detection unit202, which is composed of the CPU, ROM, and RAM (hereinafter called the CPU101and so on), detects an area where a user's hand is present and an area where a user's finger is present based on the detection signal by the gesture sensor107. In the description below, the detection of these areas is called the detection of a user's hand/finger (a detection target object).

A recognition unit203, which is composed of a CPU and other components, tracks a user's hand/finger detected by the gesture sensor107and the detection unit202to recognize a gesture operation performed by the user. An identification unit204, which is composed of a CPU and other components, identifies which user's finger was used to perform the gesture operation recognized by the recognition unit203. A holding unit205, which is composed of a CPU and other components, holds, in a storage area provided in the RAM103, object information included in the projected electronic data and specified by a user via a gesture operation in association with the finger used for the gesture operation. An acceptance unit206, which is composed of a CPU and other components, accepts an editing operation specified for electronic data via the gesture operation recognized by the recognition unit203, and, as necessary, updates electronic data stored in the storage device104. The storage device104stores electronic data that is to be processed via an editing operation. The CPU101refers to the information held in the holding unit205according to the gesture recognized by the recognition unit203, and generates a projection image to be projected on the work space. The projector106projects a projection video generated by the CPU101onto the work space that includes the projection surface110and the user's hand near the projection surface110.

FIG. 3illustrates a block diagram of the projector106.

The projector106includes a liquid crystal control unit150, liquid crystal elements151R,151G, and151B, a light source control unit160, a light source161, a color separation unit162, a color combination unit163, an optical system control unit170, and a projection optical system171.

The liquid crystal control unit150controls a voltage applied to liquid crystals of pixels of the liquid crystal elements151R,151G, and151B based on an image signal, which has been processed by an image processing unit140, to adjust transmittance of the liquid crystal elements151R,151G, and151B.

The liquid crystal control unit150includes a microprocessor for a control operation.

Each time one frame image is received from the image processing unit140when an image signal is input to the image processing unit140, the liquid crystal control unit150controls the liquid crystal elements151R,151G, and151B so that the transmittance corresponds to the image.

The liquid crystal element151R, a liquid crystal element corresponding to red, adjusts the transmittance of red light that is included in the light output from the light source161and is one of the colors separated by the color separation unit162into red (R), green (G), and blue (B).

The liquid crystal element151G, a liquid crystal element corresponding to green, adjusts the transmittance of green light that is included in the light output from the light source161and is one of the colors separated by the color separation unit162into red (R), green (G), and blue (B).

The liquid crystal element151B, a liquid crystal element corresponding to blue, adjusts the transmittance of blue light that is included in the light output from the light source161and is one of the colors separated by the color separation unit162into red (R), green (G), and blue (B).

The light source control unit160, which controls a ON/OFF state of the light source161and controls the amount of light, includes a microprocessor for a control operation.

The light source161is to output light for projecting an image onto the projection surface. For example, a halogen lamp is used as the light source161.

The color separation unit162is to separate the light, which is output from the light source161, into red (R), green (G), and blue (B). For example, a dichroic mirror is used as the color separation unit162.

When a light emitting device (LED) corresponding to each of colors is used as the light source161, the color separation unit162is not necessary.

The color combination unit163is to combine light components of red (R), green (G), and blue (B) respectively transmitted through the liquid crystal elements151R,151G, and151B. For example, a dichroic mirror is used as the color combination unit163.

The light generated by combining the components of red (R), green (G), and blue (B) by the color combination unit163is sent to the projection optical system171.

The liquid crystal elements151R,151G, and151B are controlled by the liquid crystal control unit150so that the each transmittance becomes the transmittance of light corresponding to an image input from the image processing unit140.

When the light combined by the color combination unit163is projected onto the screen by the projection optical system171, an image corresponding to the image input by the image processing unit140is displayed on the projection surface.

The optical system control unit170, which controls the projection optical system171, includes a microprocessor for a control operation.

The projection optical system171is to project the combined light, which is output from the color combination unit163, onto the projection surface. The projection optical system171includes a plurality of lenses.

A light source unit119includes the light source161, the color separation unit162, the liquid crystal elements151R,151G, and151B, and the color combination unit163.

FIG. 4is a perspective view of the whole information processing apparatus109. The configuration of the projector106is described with reference toFIG. 4.

The projector106includes the light source unit119and a lens barrel unit115in which the projection optical system171is stored.

The light source unit119and the lens barrel unit115are connected via a bending portion135. The light source unit119is arranged in the back side of the bending portion135. A reflection mirror136(seeFIG. 6) is arranged at the position of the bending portion135.

Another reflection mirror134is arranged on the upper front side of the lens barrel unit115. The reflection mirror134reflects light toward the projection surface110to project an image on the projection surface110. The reflection mirror136arranged in the bending portion135reflects light output from the light source unit119toward the reflection mirror134.

A cooling mechanism137is provided next to the light source unit119to radiate heat generated by the light source unit119.

FIG. 6is a schematic cross section diagram of the projector.FIG. 6illustrates the liquid crystal element151R only, and the liquid crystal elements151G and151B are omitted here. The projection surface110and the liquid crystal elements151R,151G, and151B are conjugated to each other, and light from each liquid crystal element passes through the color combination unit163and the projection optical system171and, after being reflected by the reflection mirror134, reaches the projection surface110. Let Ja be a light beam that is directed toward the center of the projection surface110when an image is projected onto the projection surface110. An optical path length of the projector106is defined by an optical path length of the light beam Ja. The optical path length of the light beam Ja is the sum of a distance between the point Ra1that is an intersection with the projection surface110and the point Ra2that is an intersection with the reflection surface of the reflection mirror134, a distance between the point Ra2that is the intersection with the reflection surface of the reflection mirror134and the point Ra3that is an intersection with the reelection surface of the reflection mirror136provided in the bending portion135, and a distance between the point Ra3that is the intersection with the reflection surface of the reflection mirror136and the liquid crystal element151R.

The configuration of the camera105and other components is described below with reference toFIGS. 4 and 5.

The camera105includes a CCD sensor114(seeFIG. 5) serving as a second imaging element. A main frame113is fixed on a pedestal112. A camera attachment130is attached to the main frame113. The camera105is mounted to a camera mount131via the camera attachment130. The lens barrel132, in which a plurality of lenses207(seeFIG. 5) serving as a second imaging optical system is included, is mounted on the camera mount131. An imaging mirror117, which is a concave curved mirror, is assembled on the main frame113. The imaging mirror117is arranged in the back side of an optical axis of the lenses207.

When an object body placed on the projection surface110is read by the camera105, an image of the object body is reflected by the imaging mirror117into the lens barrel132, and the reflected image, which passes through the lenses207inside the lens barrel132, is read by the CCD sensor114(seeFIG. 5).

FIG. 5is a schematic cross section diagram of the camera105and the gesture sensor107. The camera105and its optical path length are described with reference toFIG. 5.

The CCD sensor114is installed approximately horizontally to the projection surface110. The lenses207are installed with its optical axis approximately perpendicular to the projection surface110.

An object image of the object body placed on the imaging surface301, which is the same surface as the projection surface110, passes through the imaging mirror117and the plurality of lenses207and, after that, an image is formed on the light receiving surface of the CCD sensor114.

An image plane IMG formed on the light receiving surface of the CCD sensor114is a shift optical system in which the image plane IMG shifts toward a right side in the figure with respect to the optical axis of the plurality of lenses207.

Let Ia be a light beam directed toward the center of the imaging surface301when the imaging surface301is captured. Let Ib and Ic be light beams directed toward both left and right end-sides of the imaging surface301, respectively, when the imaging surface301is captured.

The optical path length of the camera105is defined by an optical path length of the light beam Ia.

The optical path length of the light beam Ia is the sum of a distance between the point Pa1that is an intersection with the imaging surface301and the point Pa2that is an intersection with the reflection surface of the imaging mirror117and a distance between the point Pa2that is the intersection with the reflection surface of the imaging mirror117and the point Pa3where an image is formed on the IMG.

The configuration of the gesture sensor107is described with reference toFIG. 4andFIG. 5.

The gesture sensor107is attached to the main frame113. The gesture sensor107includes a CCD sensor107aserving as a first imaging unit and at least one lens107b(a first imaging optical system) made of resin. The gesture sensor107is attached to the leading edge of the imaging mirror117.

In order for the gesture sensor107to detect a movement of a user's hand/finger extended above the projection surface110, it is necessary to reserve a detection area such that an area A having a height of 100 mm above the projection surface110can be detected. The gesture sensor107recognizes a movement of a user's hand/finger with a viewing angle of 60 degrees in front and back directions, and 90 degrees in right and left directions, with respect to the optical axis. The gesture sensor107is arranged in an area where it does not interfere with the light beams of the camera105and of the projector106.

Let Sa be a light beam that passes through the optical axis of the lens107bin the gesture sensor107.

The optical path length of the gesture sensor107is defined by an optical path length of the light beam Sa. The optical path length of the light beam Sa is a distance between the point Qa1that is an intersection with the imaging surface301and the point Qa2where an image is formed on the CCD sensor107a.

<Relation Between Optical Path Length of Camera105and Optical Path Length of Gesture Sensor107>

The optical path lengths of the components such as the camera105are described below with reference toFIG. 5andFIG. 6.

The relation among the optical path length of the light beam Ia of the camera105, the optical path length of the light beam Sa of the gesture sensor107, and the optical path length of the light Ja of the projector106is as follows; the optical path length of the camera105>the optical path length of the projector106>the optical path length of the gesture sensor107.

The reason for the relation, “the optical path length of the camera105>the optical path length of the gesture sensor107” is as follows. The camera105sometimes reads a document image to be processed via an optical character reader (OCR). For this reason, a readable range (range for bringing an object into focus) of a reading image is increased by extending the optical path length of the light beam Ia using the imaging mirror117to increase a depth of field. On the other hand, the gesture sensor107is required only to detect a user's hand/finger and not required to have a reading ability with accuracy as high as that of the camera105. Accordingly, the optical path length of the light beam Sa of the gesture sensor107is made shorter than the optical path length of the light beam Ia of the camera105(Ia>Sa).

To make the optical path length of the light beam Sa of the gesture sensor107the same length as the optical path length of the light beam Ia of the camera105, a mirror to reflect the light beam Sa of the gesture sensor107should be added. Adding such a mirror makes the information processing apparatus109larger. Therefore, the relation, Ia>Sa, if satisfied, makes the apparatus more compact. The mirror mentioned here for reflecting the light beam Sa of the gesture sensor107refers not to an optical system included in the gesture sensor107, but to a mirror provided externally to the gesture sensor107.

The relation among the optical path length of the projector106, the optical path length of the camera105, and the optical path length of the gesture sensor107is described below. There is no need for the projector106to have a long optical path length as that of the camera105, because the projector106is not required to have reading performance equivalent to that of the camera105. On the other hand, it is preferable for the projector106to have an optical path length longer than that of the gesture sensor107, because the projector106is required to project the image111onto the projection surface110. Thus, the optical path length relation, “the optical path length of the camera105>the optical path length of the projector106>the optical path length of the gesture sensor107” is required.

<Relation of Viewing Angles>

The relation between a viewing angle of the camera105and a viewing angle of the gesture sensor107is described below with reference toFIG. 5.

InFIG. 5, let DI be a viewing angle of the lenses207of the camera105, and let DS be a viewing angle of the lens107bof the gesture sensor107.

The viewing angle DS of the lens107bof the gesture sensor107is set wider than the viewing angle DI of the lenses207of the camera105. Setting the viewing angles in this manner allows a readable area of the gesture sensor107to be set approximately in the same range as that of a readable area of the camera105while satisfying the relation, “the optical path length of Ia>the optical path length of Sa”.

FIG. 7illustrates a schematic diagram of the information processing apparatus109viewed from above (viewed from the direction perpendicular to the projection surface110). The imaging mirror117is arranged in the back side of the optical axis of the lenses207. The light source unit119and the reflection mirror134are arranged so that the imaging mirror117and the lenses207are respectively arranged in the right-to-left direction. In addition, as illustrated inFIG. 5, the imaging mirror117and the gesture sensor107are arranged so that at least a part of the imaging mirror117and at least a part of the gesture sensor107overlap in the height direction. In other words, when viewed in a back-to-front direction (horizontal direction) inFIG. 5, the imaging mirror117and the gesture sensor107are arranged so that at least a part of the imaging mirror117and at least a part of the gesture sensor107overlap. This arrangement makes the apparatus more compact in a front-to-back direction and in a right-to-left direction and, at the same time, in a height direction.

Second Exemplary Embodiment

A second exemplary embodiment of the present invention is described in detail below with reference to the attached drawings. Only a part different from the first exemplary embodiment is described, and the description of a part similar to that in the first exemplary embodiment is omitted.

FIG. 8is a perspective view of a whole information processing apparatus109in the second exemplary embodiment. The first exemplary embodiment and the second exemplary embodiment are different in a gesture sensor120. In the first exemplary embodiment, a CCD camera is used as the gesture sensor serving as the first imaging unit. On the other hand, in the second exemplary embodiment, an infrared camera is used as the gesture sensor120. The gesture sensor120includes a light emitting unit120athat emits infrared light and a light receiving unit120bthat receives infrared light reflected by an object. The light emitting unit120aemits infrared light toward a predetermined area on the projection surface110so that a user's hand/finger near the projection surface110can be recognized. The light receiving unit120bincludes a light receiving element120b1, which is an imaging element, and a lens120b2(seeFIG. 9). The light receiving element120b1receives light emitted by the light emitting unit120aand reflected by the projection surface110or the user's hand/finger. The light receiving element120b1uses an area sensor that can receive light reflected by a predetermined area.

FIG. 9is a schematic cross section diagram of the camera and the gesture sensor in the second exemplary embodiment. The basic configurations of the camera105and the gesture sensor120are similar to those in the first exemplary embodiment.

<Relation Between Optical Path Length of Camera105and Optical Path Length of Gesture Sensor120>

The optical path lengths of the components, such as the camera105, are described below with reference toFIG. 9. The optical path length of the camera105and the optical path length of the projector106are the same as those in the first exemplary embodiment, and, therefore, the description is omitted. Let Sa be a light beam that passes through the optical axis of the lens120b2in the gesture sensor120.

The optical path length of the gesture sensor120is defined by the optical path length of the light Sa. The optical path length of the light Sa is a distance between the point Qa1that is an intersection with the imaging surface301and the point Qa2where an image is formed on the light receiving element120b1.

At this time, the relation among the optical path lengths of the components, such as the camera105, is as follows. the optical path length of the camera105>the optical path length of the projector106>the optical path length of the gesture sensor120.

The reason for the relation, “the optical path length of the camera105>the optical path length of the gesture sensor120” is as follows. The camera105sometimes reads a document image to be processed via an OCR. For this reason, a readable range (range for bringing an object into focus) of a reading image is increased by extending the optical path length of the light beam Ia using the imaging mirror117to increase a depth of field. On the other hand, the gesture sensor120is required only to detect a user's hand/finger and not required to have a reading ability with accuracy as high as that of the camera105. In addition, when an infrared camera is used for the gesture sensor120, light attenuation in the infrared light used for the gesture sensor120increases as the optical path length between the light receiving element120b1and the projection surface110becomes longer. Therefore, it is desirable that the optical path length between the light receiving element120b1and the projection surface110be short. For this reason, the optical path length of the light beam Sa of the gesture sensor120is made shorter than the optical path length of the light beam Ia of the camera105(Ia>Sa).

The relation among the optical path length of the projector106, the optical path length of the camera105, and the optical path length of the gesture sensor120is the same as that in the first exemplary embodiment.

When an infrared camera is used for the gesture sensor120as in the second exemplary embodiment, the apparatus can also be made compact while maintaining the imaging performance of the camera105of the information processing apparatus109as in the first exemplary embodiment.

This application claims the benefit of Japanese Patent Application No. 2015-169628, filed Aug. 28, 2015, which is hereby incorporated by reference herein in its entirety.