Image projection apparatus with operation image generation based on distance to the projection surface, and method for controlling the same

An image projection apparatus in which a measuring unit measures a distance from the image projection apparatus to a projection surface. A generating unit generates a first operation image if the measured distance is less than a threshold value and generates a second operation image if the measured distance is not less than the threshold value. The first operation image is an image by which an operation can be input by pointing on the first operation image that is being projected with a body part of a user. The second operation image is an image by which an operation can be input by operating a cursor displayed in the second operation image that is being projected using a predetermined operating unit. A projecting unit projects an operation image generated by the generating unit onto the projection surface.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image projection apparatus and a method for controlling the image projection apparatus.

2. Description of the Related Art

Currently, a projector (an image projection apparatus) for projecting an image onto a projection surface such as a desk or a wall is known (Japanese Patent Laid-Open No. 2005-236746). Moreover, a technique that makes it possible to indicate a desired area of a projected image with a finger or the like is known (Japanese Patent Laid-Open No. 2008-152622). According to Japanese Patent Laid-Open Nos. 2005-236746 and 2008-152622, the projector does not take the distance to the projection surface into account when generating an image to be projected.

If the distance from a projector to a projection surface changes, the size of an image on the projection surface also changes. Moreover, in the case where, for example, a user holds the projector by hand, the distance from the user to the projection surface also changes with the change in the distance from the projector to the projection surface. Therefore, the type of projection image convenient for the user changes depending on the distance from the projector to the projection surface. However, according to Japanese Patent Laid-Open Nos. 2005-236746 and 2008-152622, the distance to the projection surface is not taken into account when the projector generates a projection image.

SUMMARY OF THE INVENTION

The present invention has been made in view of the above circumstances, and it is a feature thereof to provide an image projection apparatus for projecting an image that has been generated with the distance to the projection surface being taken into account and a method for controlling the image projection apparatus.

According to an aspect of the present invention, there is provided an image projection apparatus comprising: a measuring unit that measures a distance from the image projection apparatus to a projection surface; a generating unit that generates a first operation image if the measured distance is less than a threshold value and generates a second operation image if the measured distance is not less than the threshold value, wherein the first operation image is an image with respect to which an operation can be input by pointing on the first operation image that is being projected with a body part of a user, and the second operation image is an image with respect to which an operation can be input by operating a cursor displayed in the second operation image that is being projected using a predetermined operating unit; and a projecting unit that projects an operation image generated by the generating unit onto the projection surface.

According to another aspect of the present invention, there is provided a method for controlling an image projection apparatus, comprising: a measuring step of measuring a distance from the image projection apparatus to a projection surface; a generating step of generating a first operation image if the measured distance is less than a threshold value and generating a second operation image if the measured distance is not less than the threshold value, wherein the first operation image is an image with respect to which an operation can be input by pointing on the first operation image that is being projected with a body part of a user, and the second operation image is an image with respect to which an operation can be input by operating a cursor displayed in the second operation image that is being projected using a predetermined operating unit; and a projecting step of projecting an operation image generated in the generating step onto the projection surface.

According to yet another aspect of the present invention, there is provided an image projection apparatus that is capable of projecting an image and a GUI and that, if a GUI is being projected, allows the GUI to be operated using an operating unit that is capable of operating the GUI, the apparatus comprising: a measuring unit that measures a distance from the image projection apparatus to a projection surface; and a control unit that, if the distance measured by the measuring unit is less than a threshold value, performs control so as to change the GUI so that the user can operate the GUI by touching the projection surface instead of operating using the operating unit and also to execute a recognition function for recognizing a touch operation by the user.

According to another aspect of the present invention, there is provided a method for controlling an image projection apparatus that is capable of projecting an image and a GUI and that, if a GUI is being projected, allows the GUI to be operated using an operating unit that is capable of operating the GUI, the method comprising: a measuring step of measuring a distance from the image projection apparatus to a projection surface; and a control step of, if the distance measured in the measuring step is less than a threshold value, performing control so as to change the GUI so that the user can operate the GUI by touching the projection surface instead of operating using the operating unit and also to execute a recognition function for recognizing a touch operation by the user.

With the foregoing configurations, the present invention can provide an image projection apparatus for projecting an image that has been generated with the distance to the projection surface being taken into account and a method for controlling the image projection apparatus.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will now be described with reference to the attached drawings. It should be noted that the technical scope of the present invention is defined by claims, and is not limited by each embodiment described below. In addition, not all combinations of the features described in the embodiments are necessarily required for realizing the present invention.

First, a basic configuration of a projector (an image projection apparatus) will be described with reference toFIGS. 1 to 4.FIG. 1is a block diagram showing a basic configuration of a projector. A CPU108creates display data (an image for projection) on a display memory107. The display data is supplied to a liquid crystal panel104from the display memory107and displayed on the liquid crystal panel104. A lamp105is disposed behind the liquid crystal panel104. When the lamp105irradiates the liquid crystal panel104, the image displayed on the liquid crystal panel104is projected onto a projection surface (a screen) through an optical system101disposed in front of the liquid crystal panel104.

The optical system101includes a focus lens102for adjusting the focus of an image to be projected (projection image) and a zoom lens103for scaling the image to be projected. Position information of the focus lens102is detected using a focus encoder111, and zoom information of the zoom lens103is detected using a zoom encoder112. A motor110drives the focus lens102in an optical axis direction.

The distance from the projector to the projection surface is calculated using phase difference information obtained from image information (light intensity distribution) of a subject (detection target) detected by a distance measurement sensor106. The CPU108calculates a driving target position (focusing position) of the focus lens102at which the projection image can be focused based on this phase difference information. The CPU108further obtains a driving amount of the motor110using this driving target position and the position information detected by the focus encoder111and transmits the result to the motor110. The motor110drives the focus lens102to the focusing position according to the driving amount.

FIG. 2is a diagram showing the principle of an external triangulation distance measuring method for obtaining the distance from a projector to a projection surface.201indicates a projection surface onto which an image is projected, and204and205each indicate a photoelectric conversion sensor array in which a plurality of photoelectric conversion sensors are arranged side by side. The photoelectric conversion sensor arrays204and205are, for example, area sensors or line sensors, such as CCDs.

Focus sensor lenses202and203are disposed in front of the photoelectric conversion sensor arrays204and205, respectively. An image of the detection target projected onto the projection surface201passes through each detection optical path and is detected by the photoelectric conversion sensor array204or205via the focus sensor lens202or203. Here, the focus sensor lenses202and203and the photoelectric conversion sensor arrays204and205are collectively called a focus sensor unit206. The focus sensor unit206corresponds to the distance measurement sensor106inFIG. 1.

The distance S from the focus sensor unit206to the projection surface201can be obtained using the following formula:
S=B×F/X
where the baseline length of the focus sensor unit206is B, the focal length is F, and the phase difference of the photoelectric conversion sensor array204relative to the photoelectric conversion sensor array205is X.

FIG. 3is a flowchart illustrating the flow of a focus adjustment process performed by the projector shown inFIG. 1.

In step S301, the CPU108reads output signals resulting from images of the detection target formed on the photoelectric conversion sensor arrays204and205and obtains light intensity distributions on the respective photoelectric conversion sensor arrays204and205based on the output signals. The obtained light intensity distributions are stored in a storage unit within the CPU108.

In step S302, the CPU108obtains a phase difference from the light intensity distributions of the photoelectric conversion sensor arrays204and205. An example of the method for obtaining the phase difference is a method of calculating an evaluation value for evaluating the similarity between the light intensity distributions of the photoelectric conversion sensor arrays204and205by correlation calculation and obtaining the phase difference from the relationship between the evaluation value and a shift between the images detected by the photoelectric conversion sensor arrays204and205. Next, the CPU108obtains the distance to the projection surface201based on the external triangulation distance measuring method.

In step S303, the CPU108calculates an extension amount that the focus lens102has to move to reach the focusing position, based on the distance obtained in step S302. An extension amount D of the focus lens102from the position thereof at the time when projecting an image at infinity is calculated using the following formula:
D=1/(1/F−1/S)−F
where the distance to the projection surface201is S and the focal length of the focus lens102is F.

Next, indication of a projected image with a finger will be described.FIG. 4is a flowchart illustrating the flow of a finger detection process performed by the projector shown inFIG. 1.

In step S401, the CPU108acquires image data being projected from the projector. In step S402, the CPU108acquires image data of the projection surface201taken by a camera (not shown). In step S403, the CPU108compares the image data acquired in step S401and the image data acquired in step S402and performs a finger detection (including detection of a fingertip position) process (with respect to the finger detection process, a technique described in Japanese Patent Laid-Open No. 2008-152622 etc. can be used). In step S404, the CPU108outputs the result of the detection in step S403.

First Embodiment

An embodiment in which the image projection apparatus of the present invention is applied to a mobile telephone will be described.FIG. 5is a diagram showing an outer appearance of a mobile telephone501according to a first embodiment. The mobile telephone501includes a projector unit503. The mobile telephone501also includes a projector button504and a focus adjustment button502in addition to operation buttons508used for common telephone operations. The projector button504is a button for turning on/off the projector unit503. The focus adjustment button502is a button used for an operation for performing focus adjustment of a projected image. The mobile telephone501also includes a distance measurement sensor506for measuring the distance to a projection surface and a camera507for use for the finger detection process.

FIG. 6is a functional block diagram showing the configuration of the mobile telephone501. InFIG. 6, the same elements as the elements shown inFIG. 5are denoted by the same reference numerals, and descriptions thereof will be omitted. A focus adjustment unit605adjusts the focus by controlling the projector unit503using an input from the distance measurement sensor506. A finger operation detecting unit606(an area detecting unit) performs a process (finger detection process) for detecting an area, of an image on the projection surface, indicated by a finger on the projection surface by comparing an image taken by the camera507with an image projected by the projector unit503. It should be noted that the detection target is not limited to a finger and may be, for example, an object such as a pointing rod. A user input processing unit610accepts a button operation on the focus adjustment button502, the projector button504, or the like and performs processing corresponding to the operated button.

A GUI control unit607generates an image (image for projection) to be projected by the projector unit503. A display memory604stores the image for projection generated by the GUI control unit607. A communication control unit608performs speech communication, data communication, or other communication via an antenna609.

FIG. 7is a flowchart illustrating the flow of an image projection process executed by the mobile telephone501. InFIG. 7, processing at steps S703, S710, and S714, which are enclosed by the dashed lines, can be omitted (the details will be described later). If the projector button504is pressed, the projector unit503is activated, and if a button operation is performed when the projector unit503is operating, the process of this flowchart is started. Moreover, even if the button operation is not performed, the process of this flowchart may be started from step S706when, for example, the projector unit503is activated.

In step S702, the user input processing unit610determines whether or not the operated button is the focus adjustment button502. If the determination result is “Yes”, the process proceeds to step S706, and if the determination result is “No”, the process proceeds to step S703.

In step S706, the focus adjustment unit605measures the distance to the projection surface using the distance measurement sensor506and performs focus adjustment (seeFIG. 3and the description thereof for details). In step S707, the GUI control unit607determines whether or not the measured distance is less than a threshold value. If the measured distance is less than the threshold value, the process proceeds to step S708, and if not, the process proceeds to step S712. This threshold value is used as a benchmark for determining whether or not the user can reach the projection surface. That is to say, if the distance to the projection surface is less than the threshold value, there is a good possibility that the user can reach the projection surface, and accordingly, a GUI for finger operation is projected as will be described later. The threshold value is, for example, 70 cm, but the mobile telephone501may be configured so as to allow the user to set the threshold value.

In step S708, the GUI control unit607determines whether or not the projector unit503is projecting a GUI for finger operation (a first image). If the determination result is “Yes”, the process proceeds to step S711, and if the determination result is “No”, the process proceeds to step S709.

In step S709, the GUI control unit607generates a GUI for finger operation as an image for projection and stores the GUI in the display memory604. In step S710, the GUI control unit607enables the finger operation detecting unit606.

A GUI for finger operation801inFIG. 8is an example of the GUI for finger operation generated in step S709. If the distance to the projection surface is small, the size of an image that appears on the projection surface is relatively small, and accordingly, areas of an “End” button and the like are generated so as to be relatively large in order to improve the visibility and operability. Although not shown inFIG. 7, the finger operation detecting unit606detects an area indicated by a finger in the manner illustrated inFIG. 4, and the user input processing unit610executes processing corresponding to the detected area (e.g., scrolling of a TV guide811). Moreover, the GUI control unit607generates an updated GUI for finger operation based on the processing content that the user input processing unit610has executed.

In step S711, the projector unit503projects an image for projection (a GUI for finger operation if step S709has been performed) stored in the display memory604.

If it is determined in step S707that the distance is not less than the threshold value, in step S712, the GUI control unit607determines whether or not the projector unit503is projecting a GUI for button operation (a second image). If the determination result is “Yes”, the process proceeds to step S711, and if the determination result is “No”, the process proceeds to step S713.

In step S713, the GUI control unit607generates a GUI for button operation as an image for projection and stores the GUI in the display memory604. In step S714, the GUI control unit607disables the finger operation detecting unit606. A GUI for button operation802inFIG. 8is an example of the GUI for button operation generated in step S713. The GUI for button operation802has a cursor that moves in response to an operation on a button such as the operation buttons508. Moreover, if the distance to the projection surface is large, the size of an image that appears on the projection surface is relatively large, and accordingly, the GUI for button operation802generally contains more information than the GUI for finger operation801.

If it is determined in step S702that a button other than the focus adjustment button502has been operated, in step S703, the user input processing unit610determines whether or not the projector unit503is projecting a GUI for button operation. If the determination result is “Yes”, the process proceeds to step S704, and if the determination result is “No”, the process is ended.

In step S704, the user input processing unit610executes processing (e.g., moving the cursor) corresponding to the button operation. In step S705, the GUI control unit607generates an updated GUI for button operation based on the processing content that has been executed in S704.

Incidentally, in step S705, the GUI control unit607may store the processing content that the user input processing unit610has executed (i.e., the state of the updated GUI for button operation). Similarly, the GUI control unit607may also store the processing content that the user input processing unit610has executed during projection of the GUI for finger operation. In this case, in step S709, the GUI control unit607can generate a GUI for finger operation based on the processing content that has been executed during projection of the GUI for button operation. Also, in step S713, the GUI control unit607can generate a GUI for button operation based on the processing content that has been executed during projection of the GUI for finger operation.

Thus, if the type of GUI is switched, the continuity of the GUI content between before and after the switching can be maintained. For example, in the GUI for button operation802inFIG. 8, the cursor is located in the “PM3” row, under the “BS103” column. Then, if switching occurs in this state, a GUI for finger operation is generated so that, as shown in the GUI for finger operation801, the “PM3” row appears at the top and the “BS103” column appears at the left.

Moreover, although the finger operation detecting unit606is enabled in step S710and the finger operation detecting unit606is disabled in step S714, the finger operation detecting unit606may be always operative. That is to say, the mobile telephone501may accept an operation by a finger even when a GUI for button operation is being projected. Similarly, the mobile telephone501may accept an operation through a button even when a GUI for finger operation is being projected. In this case, the determination in step S703is no longer necessary, and if it is determined “NO” in step S702, the process proceeds to step S704.

As described above, according to this embodiment, the mobile telephone501projects an image with the distance to a projection surface being taken into account, and thus, the convenience for the user is improved. In particular, when the user performs an operation with respect to an image projected from the image projection apparatus, the image that is easy to operate is different between the cases where the image is projected at a position close to the user and where the image is projected at a distant position as compared to the former case, that is to say, depending on the distance to the projection surface. As described above, it is convenient that, in the case where the image is projected at the close position, the user inputs an operation with respect to the image using a finger, and in the case where the image is projected at a position beyond the reach of the finger, an operation with respect to the image is input using some kind of pointing device. Conventional techniques have not been able to solve such a problem, but the present invention employs the above-described configuration and therefore can solve the problem by generating and displaying an image corresponding to the distance to the projection surface when an operation is input using an image projected from the image projection apparatus.

First Variation

FIG. 9is a functional block diagram showing the configuration of a mobile telephone901according to a first variation of the first embodiment. InFIG. 9, the same elements as the elements shown inFIG. 6are denoted by the same reference numerals, and descriptions thereof will be omitted. The mobile telephone901includes a hand sensor902. The hand sensor902determines whether the user is holding the mobile telephone901with the right hand or the left hand.

In the first variation, in the above-described image projection process, the GUI control unit607takes into account which hand the user is holding the mobile telephone901with when generating a GUI for finger operation, as described below with reference toFIG. 10.

FIG. 10is a diagram illustrating processing that replaces the processing at step S709shown inFIG. 7. Moreover, the processing inFIG. 10is also executed when the hand sensor902detects that the hand that holds the mobile telephone901has changed during projection of a GUI for finger operation.

In step S1001, the GUI control unit607determines whether or not the hand that holds the mobile telephone901is the right hand. If it is the right hand, the process proceeds to step S1002, and if not, the process proceeds to step S1003. In step S1002, the GUI control unit607generates a GUI for finger operation1101shown inFIG. 11. The GUI for finger operation1101is configured so as to be appropriate for operation with the left hand. In step S1003, the GUI control unit607generates the GUI for finger operation801shown inFIG. 8. The GUI for finger operation801is configured so as to be appropriate for operation with the right hand. Thus, the convenience for the user is further improved.

Second Variation

FIG. 12is a functional block diagram showing the configuration of a mobile telephone1201according to a second variation of the first embodiment. InFIG. 12, the same elements as the elements shown inFIG. 6are denoted by the same reference numerals, and descriptions thereof will be omitted. The mobile telephone1201includes a projection direction detecting unit1202(a movement detecting unit) and an acceleration sensor1203. The projection direction detecting unit1202detects movement of the mobile telephone1201based on information from the acceleration sensor1203.

If the projection direction detecting unit1202detects movement, the image projection process illustrated inFIG. 7is started from step S706. Although the distance to the projection surface can change with movement of the mobile telephone1201, according to the second variation, an image corresponding to the distance after the change is projected, and thus, the convenience for the user is further improved.

Other Embodiments

Aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment, and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment. For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (e.g., computer-readable medium).

This application claims the benefit of Japanese Patent Application No. 2009-161237, filed on Jul. 7, 2009, which is hereby incorporated by reference herein in its entirety.