Operation input device and method, program, and electronic apparatus

An operation input device includes: angular velocity detecting means for detecting an angular velocity; relative velocity detecting means for contactlessly detecting a relative velocity to a target object; distance detecting means for detecting a distance to the target object; and computing means for computing an amount of movement based on the angular velocity, the relative velocity, and the distance.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an operation input device and method, a program, and electronic apparatus, and specifically, to an operation input device and method, a program, and electronic apparatus that realize input with good operability.

2. Description of the Related Art

In personal computers, as controllers of GUI (Graphical User Interface), pointing devices such as mice and touch pads are mainly used.

Recently, in television receivers, in order to deal with the complexity of operation by addition of additional functions and networking, free cursor operation input devices of the same GUI as that of the personal computers have been desired.

However, it takes significant time to master input devices such as mice and touch pads, and it is difficult for young children and elderly people to adapt to them.

As input devices advantageous in operability to solve the problem, pen-shaped input devices usable either in proximity or contact are suggested.

As a representative of the devices, pen tablets using electromagnetic induction have been put into practical use (for example, Wacom|Tablet Homepage, Internet<URL:http://tablet.wacom.co.jp/index.html>).

SUMMARY OF THE INVENTION

However, in a technique disclosed in Wacom|Tablet Homepage, operation may be performed only on a tablet for exclusive use.

In addition, a method of printing a special pattern on paper and obtaining coordinates by imaging the pattern using an imaging sensor and a method of determining the position of the pen using an ultrasonic generator, a magnetic generator, or an infrared generator externally provided have begun to be suggested and commercialized. However, there are restraint conditions that operation may be performed only on the special paper and operation may be performed only near the external generator, and they deteriorate the operability.

Thus, it is desirable to realize input with good operability.

According to an embodiment of the invention, there is provided an operation input device and an operation input method including angular velocity detecting means for detecting an angular velocity, relative velocity detecting means for contactlessly detecting a relative velocity to a target object, distance detecting means for detecting a distance to the target object, and computing means for computing an amount of movement based on the angular velocity, the relative velocity, and the distance, and a program or electronic apparatus for allowing a computer to function as the operation input device.

The relative velocity detecting means may include light emitting means for emitting light to illuminate the target object, and imaging means for imaging the target object irradiated with the light from the light emitting means, and the distance detecting means may detect the distance based on an image imaged by the imaging means.

The operation input device may further include contact detecting means for detecting contact with the target object.

The contact detecting means may detect two-step contact.

The operation input device may further include a buffer material that suppresses transfer of vibration generated at contact with the target object to the angular velocity detecting means.

If the distance to the target object is equal to or more than a reference range, screen operation may be stopped, if the distance to the target object is less than the reference range, screen operation may be executed, and, if there is contact with the target object, special operation may be executed.

If the distance to the target object is equal to or more than a reference range, first screen operation may be executed, and, if the distance to the target object is within the reference range, second screen operation may be executed.

If the distance to the target object is equal to or more than a first range within the reference range, screen operation based on a first physical quantity may be executed, and, if the distance to the target object is less than the first range within the reference range, screen operation based on a second physical quantity may be executed.

The operation input device may further include frequency detecting means for detecting a frequency of vibration at sliding if there is contact with the target object, and acquiring means for acquiring a correction value based on the frequency detected by the frequency detecting means, and the computing means may compute a velocity at the sliding based on the correction value and the angular velocity.

The frequency detecting means may detect the frequency of the vibration at the sliding from sound. The frequency of the vibration at the sliding may be detected from vibration propagated via component elements.

The frequency detecting means may be the angular velocity detecting means, and the angular velocity detecting means may further detect the frequency of vibration at sliding.

The operation input device may further include gesture detecting means for detecting a gesture based on the angular velocity if there is contact with the target object. Further, the gesture detecting means may detect a gesture based on an image imaged by the imaging means.

In the embodiment of the invention, the angular velocity is detected, the relative velocity to the target object is contactlessly detected, the distance to the target object is detected, and the amount of movement is computed based on the angular velocity, the relative velocity, and the distance.

According to the embodiment of the invention, input with good operability may be realized.

DESCRIPTION OF PREFERRED EMBODIMENTS

Configuration of Operation Input Device

FIGS. 1A and 1Bshow a configuration example as one embodiment of an operation input device1to which the invention is applied. The operation input device1is operated by a user when a predetermined command is input to a television receiver, for example. That is, the operation input device1functions as a remote controller of the television receiver.FIG. 1Ashows an overall configuration of the operation input device1.FIG. 1Bshows details of a configuration of an end part of the operation input device1.

As shown inFIGS. 1A and 1B, the pen-shaped operation input device1includes a main board11, a sensor board12, an angular velocity sensor13, a power switch14, a button15, a microphone16, a buffer material17, a battery18, a light emitting part19, a lens20, an image sensor21, a contact part22, a tactile switch23, and a spring24.

On the main board11, a microcomputer71, which will be described later with reference toFIG. 4, etc. are mounted.

On the sensor board12, the angular velocity sensor13, the microphone16, the image sensor21, etc. are mounted.

The angular velocity sensor13as angular velocity detecting means detects an angular velocity when the operation input device1is operated. The angular velocity sensor13includes a gyro sensor or the like, for example.

Note that, using an acceleration sensor in place of the angular velocity sensor13, an angular velocity may be obtained by differentiating its output. Specifically, the acceleration sensor measures acceleration in directions of three axes of X, Y, Z axes. Then, an angular velocity may be obtained by calculating a tilt angle relative to the gravity vector direction from the acceleration measured on the respective axes and differentiating the calculated tilt angle with respect to time.

The power switch14is connected to the main board11, and operated by the user when power is turned on or off.

The button15is connected to the main board11. The button15is operated when the user performs predetermined input.

The microphone16is connected to the sensor board12. The microphone16collects vibration sound at sliding of the operation input device1.

The buffer material17is sandwiched between a casing1A that supports the angular velocity sensor13and a casing1B that supports the contact part22. The buffer material17suppresses transfer of vibration generated when the contact part22comes into contact with a target object from the casing1B to the casing1A, and to the angular velocity sensor13.

The battery18is connected to the main board11. When the power switch14is turned on, the battery supplies necessary power to the respective parts.

The light emitting part19as light emitting means irradiates the target object with parallel light via the lens20. For example, the light emitting part19includes an LED (Light Emitting Diode), a semiconductor laser, or the like.

The image sensor21as relative velocity detecting means and imaging means images an image containing the target object irradiated with the light. For example, the image sensor21includes a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or the like.

The contact part22as contact detecting means is supported by one end of the spring24with the other end supported by the casing1B, and urged by the spring24in a direction projecting outward. When coming into contact with the target object, the contact part22moves rearward against the urging force of the spring24and turns on the tactile switch23located at the rear side. When moving away from the target object, the contact part22returns to the original position by the urging force of the spring24and turns off the tactile switch23.

In the embodiment, two values of on/off are detected by the tactile switch23, or the on-stage is divided into two steps and three values are detected. Alternatively, using a pressure sensor, processing based on pressure may be executed.

FIG. 2is an enlarged view of an end part of the operation input device1. The contact part22is in contact with a target object31. Concurrently, the image sensor21is separated from the target object31at a distance L. That is, when the operation input device1is in contact with the target object31, the distance L between the image sensor21and the target object31takes a value larger than zero.

FIG. 3shows details of the image sensor21.

In front of the image sensor21, a lens with deep depth of field (not shown) may be provided. Note that, in the embodiment, as an alternative of the lens, a pinhole plate52with a pinhole51formed therein is provided. Thereby, within the range of the depth of field, focus may be obtained independent of the distance.

FIG. 4is a block diagram showing a configuration example of the main board11of the operation input device1inFIGS. 1A and 1B.

In the example ofFIG. 3, on the main board11, the microcomputer71, a transmitter72, an antenna73, and a storage device74are mounted. The microcomputer71has functional blocks of a control part91, an acquisition part92, an image processing part93, a detection part94, a computation part95, a command generation part96, a transmission part97, and a determination part98. The respective blocks of the microcomputer71can send and receive signals (data) between one another according to need.

The control part91of the microcomputer71performs processing of control of the angular velocity sensor13and the image sensor21and the like. The acquisition part92acquires various information from the angular velocity sensor13, the image sensor21, the storage device74, the detection part94, and the computation part95. The image processing part93performs processing of extracting data from the image necessary in the detection part94and the computation part95. The detection part94as distance detecting means detects a distance between the operation input device1and the target object. Further, the detection part94detects various information from the storage device74based on the information acquired by the acquisition part92. The computation part95as computing means computes an amount of movement of the operation input device1. The command generation part96generates a command based on the distance between the operation input device1and the target object. The transmission part97transmits various information to a receiving device (not shown). The determination part98determines whether the various information satisfies a condition or not.

The transmitter72transmits various information to a receiving device (not shown) such as a television receiver via the antenna73according to the control of the transmission part97. The various information includes the amount of movement of the operation input device1, commands, etc.

In the storage device74, information corresponding to the various sensor information is defined in advance and stored. The storage device74may be provided within the main board11or provided in the receiving device (not shown).

Next, with reference toFIGS. 5 to 7B, the velocity computation processing of the operation input device1will be explained. This processing is processing performed at step S23inFIG. 8, which will be described later, but explained in advance for convenience.

FIG. 5is a flowchart for explanation of the velocity computation processing of the operation input device1. In the embodiment, the direction and the amount in which the operation input device1has moved is detected based on the image imaged by the image sensor21, and the velocity of the operation input device1is computed.

When moving a cursor displayed on the television receiver (not shown), for example, the user translates the operation input device1on the target object31. The target object31may be a desk, a table, a sofa, a hand of the user, a trouser (the thigh of the user sitting there), or the like.

FIG. 6shows movement of the operation input device1when the operation input device1is in contact with the target object31. Though the illustration is omitted, inFIG. 6, the contact part22is in contact with the target object31. Further, inFIG. 6, the illustration of the contact part22is omitted. This is applicable toFIG. 9,FIG. 11, and FIG.13, which will be described later.

InFIG. 6, a movement distance per unit time (i.e., velocity) of the operation input device1is indicated by a distance L61. Further, inFIG. 6, a movement distance per unit time obtained from the image sensor21is indicated by a distance L62.

When the operation input device1is in contact with the target object31, the distance between the image sensor21and the target object31is short enough, and the distance L62is nearly equal to the distance L61.

The distance L61and the distance L62are movement distances per unit time, however, they may be movement distances per sampling time by the image sensor21.

When the distances are movement distances per sampling time, the velocity of the operation input device1(i.e., the relative velocity to the target object31) is obtained by dividing the distance L61or the distance L62by the sampling time.

At step S1ofFIG. 5, the acquisition part92acquires an image from the image sensor21. This image is an image of the target object31irradiated with light via the lens20by the light emitting part19.

At step S2, the image processing part93binarizes the image. Through the binarization processing, in the image, the range irradiated with light is “1”, for example, and the range not irradiated with light is “0”, for example.

At step S3, the image processing part93clips the image in the range irradiated with light.

At step S4, the image processing part93compares the clipped image with an image clipped one unit time before. The clipped images are shown inFIGS. 7A and 7B, for example.

FIGS. 7A and 7Bshow examples of images of frames imaged by the image sensor21.FIG. 7Ashows an image of a frame clipped one unit time before, andFIG. 7Bshows an image of a frame clipped at the present time. InFIGS. 7A and 7B, bright spots131-1,131-2corresponding to ranges irradiated with light and marks132-1-1to132-1-6,132-2-1to132-2-6of the target object31are shown. Because of irradiation of light, the image of the marks132of the target object31may be obtained even when the usage environment is dark. Further, by providing a wavelength selection filter that passes only the wavelength of light radiated by the light emitting part19in front of the image sensor21, even when the usage environment is light, the influence of disturbance light may be reduced.

The marks132-2-1to132-2-6correspond to the marks132-1-1to132-1-6.

Hereinafter, if it is unnecessary to individually distinguish the marks132-1-1to132-1-6,132-2-1to132-2-6of the target object31, they may collectively be referred to as the marks132of the target object31. The marks132of the target object31may be marks of a table, marks of cloths, marks of fabrics, or the like, for example.

The image processing part93moves the image of the bright spot131-2ofFIG. 7Bin all directions and obtains a position where the correlation coefficient with the bright spot131-1ofFIG. 7Abecomes the maximum.

InFIG. 7A, all of the marks132-1-1to132-1-3are within the bright spot131-1, and parts of the marks132-1-4to132-1-6are contained within the bright spot131-1. InFIG. 7B, parts of the marks132-2-1to132-2-4are contained within the bright spot131-2, and all of the marks132-2-5,132-2-6are within the bright spot131-2.

Accordingly, it is known that the bright spot131-2ofFIG. 7Bhas moved from the position of the bright spot131-1ofFIG. 7Atoward the marks132-2-5,132-2-6. That is, it is known that the operation input device1has moved from the position inFIG. 7Atoward the upper left.

At step S5ofFIG. 5, the computation part95computes the movement direction in which the correlation coefficient is the maximum and the movement distance. The bright spot131-2ofFIG. 7Bhas the maximum correlation coefficient toward the upper left of theFIG. 7A, and the movement distance in the direction is computed. The movement distance is obtained from the distance between the positions of the bright spot131-1and the bright spot131-2within the frame of the image sensor21.

At step S6, the computation part95computes a velocity. Specifically, the velocity is obtained by dividing the distance computed at step S5by unit time.

Next, processing of transmitting a command of the operation input device1will be explained.

FIG. 8is a flowchart for explanation of the command transmission processing of the operation input device1.

When moving the cursor displayed on the television receiver, for example, the user moves the operation input device1in the proximity of the target object31in a predetermined direction in parallel to the target object31as shown inFIG. 9.

At step S21, the detection part94detects the distance between the image sensor21and the target object31.

Here, referring to a flowchart ofFIG. 10, distance detection processing of the operation input device1will be explained in detail.

FIG. 10is a flowchart for explanation of the detection processing of the operation input device1. In the example ofFIG. 10, the target object31irradiated with light emitted by the light emitting part19is imaged by the image sensor21, image processing is executed thereon, and the distance between the image sensor21and the target object31is detected.

FIG. 11shows a principle of detecting the distance between the image sensor21and the target object31.FIG. 11is a diagram showing detection of the distances to the positions in which the target object31is respectively shown as the target objects31-1to31-3from the image sensor21.

The light emitting part19radiates irradiation light171as parallel light at a fixed angle via the lens20.

The irradiation light171is respectively reflected by the target objects31-1to31-3and enters the image sensor21as reflected light172-1to172-3.

For example, when the target object31-1is irradiated with the irradiation light171, the incident position of the reflected light172-1on the image sensor21is a position near the light emitting part19.

On the other hand, the incident position of the reflected light172-2by the target object31-2farther than the target object31-1is a position farther from the light emitting part19than the incident position of the reflected light172-1. The incident position of the reflected light172-3by the target object31-3even farther than the target object31-2is a position farther from the light emitting part19than the incident position of the reflected light172-2. Therefore, the distance of the target object31from the image sensor21may be detected from the incident position of the reflected light172.

Returning toFIG. 10, processing at steps S41to S43of the operation input device1is the same as the processing at steps S1to S3of the operation input device1inFIG. 5. Accordingly, the detailed explanation of the processing will appropriately be omitted for avoiding a repetition.

That is, at step S41, the acquisition part92acquires an image from the image sensor21. At step S42, the image processing part93binarizes the image. At step S43, the image processing part93clips the image in the range irradiated with light.

At step S44, the image processing part93obtains a centroid of the image from the clipped image. The applied light and the centroid are as shown inFIGS. 12A and 12B, for example.

FIGS. 12A and 12Bshow examples of images imaged by the image sensor21. InFIGS. 12A and 12B, the marks of the target object31are not shown, but only bright spots131-11,131-12as ranges of reflected light are shown. The image processing part93obtains centroids133-11,133-12from the bright spots131-11,131-12.

Hereinafter, if it is unnecessary to individually distinguish the centroids133-11,133-12they may collectively be referred to as the centroid133.

At step S45ofFIG. 10, the detection part94detects the distance between the image sensor21and the target object31. A distance L11between the image sensor21and the target object31is measured in advance with respect to each position coordinate of the centroid133based on experiments or simulations, and stored in the storage device74.

Therefore, the distance L11is detected by comparing the position coordinate of the detected centroid133with the position coordinate of the centroid defined in advance.

In the example ofFIG. 12A, assuming that the upper side of the image is the light emitting part19side inFIG. 11, the bright point131-11is at the upper side of the image. Accordingly, it is known that the distance between the image sensor21and the target object31is shorter.

The bright point131-12ofFIG. 12Bis at the lower side of the image. Accordingly, it is known that the distance between the image sensor21and the target object31is longer than that inFIG. 12A.

Note that, in the embodiment, the distance between the operation input device1and the target object31is detected by obtaining the position of the centroid133of the bright point131, however, the distance may be detected by using diverging light as the irradiation light171and obtaining a diameter or an area of the bright point131.

The diameter of the diverging light becomes larger as the distance from the light emitting part19is longer. Accordingly, if the diameter of the bright point131is shorter, the distance between the operation input device1and the target object31may be detected to be shorter, and, if the diameter of the bright point131is longer, the distance between the operation input device1and the target object31may be detected to be longer.

For the detection of the distance, in addition, focus adjustment, an infrared sensor, or an ultrasonic sensor may be used.

Returning toFIG. 8, after the distance is detected at step S21in the above described manner, at step S22, the determination part98determines whether the distance is equal to or more than a reference value or not. If the distance detected at step S21is smaller than the preset reference value, the process moves to step S23.

At step S23, the computation part95computes velocity computation processing using the image sensor21. The velocity computation processing is performed in the manner explained with reference toFIG. 5.

At step S24, the acquisition part92acquires the angular velocity detected by the angular velocity sensor13.

At step S25, the computation part95corrects the velocity according to the following equation (1).
V=Vi+ω×L(1)

In the equation (1), V indicates the corrected velocity of the operation input device1, L indicates the distance L11detected at step S21, Vi indicates the velocity computed at step S23(step S6inFIG. 5), and ω indicates the angular velocity acquired at step S24, respectively. By independently computing the equation (1) in the X-axis direction and the Y-axis direction, the velocity of the operation input device1in the proximity may be obtained. Thereby, not only in the case where the surface of the target object31is a flat surface but also even in the case where the surface is a convex surface, a concave surface, or a composition of them, the velocity may accurately be detected.

Since the velocity is a movement distance per unit time, the distance may be obtained by multiplying the velocity by time (i.e., integration by time).

That is, as shown inFIG. 9, the operation input device1and the target object31are separated at a distance L91. Assuming that the operation input device1is translated by a distance L92in parallel to the target object31in the position separated at the distance L91from the target object31, the distance when an axis101of the operation input device1is translated to the position shown as an axis1C2is L92. However, when the operation input device1is rotated during operation from the position shown as the axis1C2to the position shown as an axis1C3, the distance obtained by multiplying the velocity obtained by the computation explained with reference toFIG. 5by time is L93. The distance L92does not become equal to the distance L91.

That is, when the operation input device1is translated, the image sensor21images the target object31in a position P1. However, when the operation input device1is rotated by an angle α, what is imaged is the target object31in a position P2. The distance L93is shorter than the distance L92by the distance L94between the position P1and the position P2.

Accordingly, in the equation (1), a velocity Vi corresponding to the distance L92is corrected by a velocity ωL, corresponding to the distance L94.

Note that, when the operation input device1is in contact with the target object31, because the distance L of the equation (1) is sufficiently small, the detected velocity Vi may be considered as the velocity of the operation input device1.

At step S26, the command generation part96generates a command corresponding to the corrected velocity, for example. Thereby, for example, a command for moving a cursor at the velocity by the distance corresponding to the corrected velocity in the detected direction is generated.

At step S27, the transmission part97transmits a command from the transmitter72to the television receiver via the antenna73. The television receiver moves the cursor being displayed in response to the command.

Note that the information including the direction, the velocity, and the distance may be transmitted to the television receiver without change, and commands based on the information may be generated at the television receiver side and recognized.

If the distance detected at step S22is determined to be equal to or more than the preset reference value, a command for stopping the movement of the cursor is generated at step S26, and the command is transmitted at step S27. In other words, in this case, the operation of moving the cursor is substantially prohibited.

Note that, when the contact part22is in contact with the target object31, a command corresponding thereto may be transmitted or information representing the contact may be transmitted together with the information of the direction, the velocity, the distance, or the like.

By obtaining the velocity of the operation input device1in the above described manner, the operation velocity from the contact to a predetermined range (the range in which the distance L may be detected or the range that may be imaged by the image sensor21) may be consistent with the output velocity signal, and the operation input device1with good operability in a range from contact to proximity can be provided. As a result, not only pointing but also character input on the target surface including, for example, a desk surface or a curved surface, on the lap, for example, can easily be performed.

FIG. 13shows differences among operation commands in response to the distances between the image sensor21and the target object31.

If the distance between the image sensor21and the target object31is a distance L141equal to or more than a distance L0as a reference value, the operation command of the operation input device1is for screen operation of “stop cursor operation”.

If the distance between the image sensor21and the target object31is a distance L142less than the distance L0as the reference value, and additionally, the contact part22is not in contact with the target object31, the operation command of the operation input device1is for screen operation of “cursor operation”. That is, the cursor is moved in response to the operation.

If the distance between the image sensor21and the target object31is a distance at which the contact part22is not in contact with the target object31, that is, “on” at the first step is detected by the tactile switch23, the operation command of the operation input device1may be special operation such as “drag operation”, “drawing operation”, or the like in addition to “cursor operation”. Further, if the contact part22is pressed down to the position where the second step of tactile switch23is turned on, a command of “determination operation” may be executed.

Note that, when the user presses down the button15, the command of “stop cursor operation” may be executed. Further, if the distance between the image sensor21and the target object31is the distance L142, when the user presses down the button15, the command of “drag operation” or “drawing operation” may be executed.

Further, under the condition that the contact part22is in contact with the target object31(that is, the first step of the tactile switch23is turned on), when the user presses down the button15, or the contact part22is continuously pressed down to the position where the second step of the tactile switch23is turned on, the command for screen operation of “drag operation” or “drawing operation” may be executed.

Furthermore, if the distance between the image sensor21and the target object31is the distance L141, a command for screen operation of “scroll” may be executed, and, if the distance between the image sensor21and the target object31is the distance L142or less, a command for screen operation of “cursor operation” may be executed.

Moreover, the image sensor21and the target object31is at the distance L141larger than the distance L0as the reference value, a command for screen operation based on the angular velocity as a physical quantity acquired from the angular velocity sensor13may be executed, and, if the image sensor21and the target object31is at the distance L142smaller than the distance L0as the reference value or less, a command for screen operation based on the velocity as another physical quantity acquired at step S25ofFIG. 8may be executed.

In addition, using a pressure sensor for the contact part22, if the contact part22is in contact with the target object31, the command for screen operation of “drawing operation” may be executed while changing the thickness of the line in response to the strength with which the user presses down the target object31.

Next, with reference toFIGS. 14 to 15B, a configuration for suppressing the influence by noise when the operation input device1is in contact with the target object31will be explained.

FIG. 14shows the case where the operation input device1is in contact with the target object31having a coarse surface. The target object31having a coarse surface is jeans that the user is wearing, a pearskin-finished table, or the like.

When the operation input device1is slid on the coarse surface, the angular velocity sensor13shows waveforms with a lot of noise as shown inFIG. 15A.

FIG. 15Ashows output waveforms of the angular velocity sensor13when the user uses the operation input device1on the target object31having a coarse surface and describes a circle.

InFIG. 15A, the horizontal axis indicates time (ms) and the vertical axis indicates output (DIGIT) of the angular velocity sensor13, respectively. InFIG. 15A, an output waveform211-1in the X-axis direction and an output waveform212-1in the Y-axis direction of the angular velocity sensor13are shown.

As shown inFIG. 15A, the output waveform211-1in the X-axis direction and the output waveform212-1in the Y-axis direction contain a lot of noise. Accordingly, even using a filter, it may be impossible to sufficiently remove the noise components.

Accordingly, in the embodiment, as shown inFIG. 14, the buffer material17is sandwiched between the casing1B that supports the contact part22and the casing1A that supports the angular velocity sensor13.

The buffer material17includes a synthetic resin of a urethane viscoelastic material such as SORBOTHANE (registered trademark), rubber, or another viscoelastic material. The transfer of the vibration generated in the contact part22to the angular velocity sensor13via the casing1B, the casing1A is suppressed by the buffer material17.

FIG. 15Bshows output waveforms of the angular velocity sensor13when the user uses the operation input device1with the buffer material17sandwiched between the contact part22and the angular velocity sensor13on the target object31having a coarse surface and describes a circle.

Note that,FIG. 15Bshows the output waveforms of the angular velocity sensor13when the user describes a circle different from the circle described inFIG. 15A.

It is known that, in an output waveform211-2in the X-axis direction and an output waveform212-2in the Y-axis direction of the angular velocity sensor13ofFIG. 15B, the noise as high-frequency components is pressed down compared to the output waveform211-1, the output waveform212-1ofFIG. 15A.

Therefore, by sandwiching the buffer material17between the contact part22and the angular velocity sensor13, the vibration of the contact part22may be relaxed and the noise of the output waveforms of the angular velocity sensor13may be reduced. That is, the operation input device1may reflect the operation even when it is in contact with the target object31having the coarse surface.

[Calibration of Angular Velocity Sensor13]

Next, with reference toFIG. 16, calibration processing of the angular velocity sensor13using the image sensor21will be explained.

FIG. 16is a flowchart for explanation of the calibration processing of the angular velocity sensor13.

At step S61, the acquisition part92acquires the velocity obtained in the velocity computation processing inFIG. 5.

At step S62, the determination part98determines whether the acquired velocity is equal to or less than a predetermined threshold value or not.

If the acquired velocity is determined to be equal to or less than the predetermined threshold value, at step S63, the control part91executes calibration of DC (Direct Current) offset of the angular velocity sensor13.

The calibration of DC offset is to set the DC output of the angular velocity sensor13to a predetermined reference output, i.e., the output when the angular velocity is zero.

At step S62, if the acquired velocity is determined to be more than the predetermined threshold value, the processing at step S63is skipped and the process is ended.

This is the end of the calibration processing of the angular velocity sensor13.

Next, with reference toFIG. 17, processing of computing another velocity of the operation input device1than that inFIG. 5will be explained.FIG. 17is a flowchart for explanation of slide velocity computation processing of the operation input device1.

In the embodiment, if the operation input device1is slid in contact with the target object31, the velocity is estimated using the angular velocity sensor13in place of the image sensor21.

At step S81, the acquisition part92acquires the angular velocity detected by the angular velocity sensor13.

At step S82, the acquisition part92acquires a peak frequency of vibration generated at sliding of the operation input device1from the angular velocity acquired at step S81. The peak frequency of the vibration may be acquired from a signal obtained by collection of the sound at sliding using the microphone16. The frequency of the vibration at sliding may be detected from vibration propagated via component elements.

The frequency of the vibration of the operation input device1is proportional to the velocity of the operation input device1. That is, when the frequency of the vibration of the operation input device1is high, the movement velocity of the operation input device1is high.

At step S83, the acquisition part92acquires a correction value in response to the peak frequency. The correction value is defined in response to the peak frequency and stored in the storage device74in advance.

At step S84, the computation part95computes the slide velocity of the operation input device1by multiplying the angular velocity acquired in the processing at step S81by the correction value acquired in the processing at step S83.

That is, in the embodiment, the angular velocity detected by the angular velocity sensor13is used as a representative of amounts of operation of the operation input device1. Using only the angular velocity, the translation component at sliding operation is not sensed and the amount of operation by the user is inconsistent with the operation output. Accordingly, the above described correction processing is performed.

Note that, in place of the angular velocity, acceleration may be detected and differentiated, and thereby, the slide velocity may be estimated.

This is the end of the slide velocity computation processing of the operation input device1.

Thereby, the operation input device1may compute the slide velocity at contact without using the image sensor21.

Next, with reference toFIGS. 18A to 18C, gesture detection of the operation input device1will be explained. A gesture is detected based on output waveforms of the angular velocity sensor13.

The gesture refers to flick, reciprocating operation, operation of describing a circle, or the like by the operation input device1in contact with the target object31, for example.

FIG. 18Ashows outputs of the angular velocity sensor13when the user executes the reciprocating operation and the operation of describing a circle under the condition that the operation input device1is in contact with the target object31.

InFIG. 18A, an output waveform211-11in the X-axis direction and an output waveform212-11in the Y-axis direction of the angular velocity sensor13with the two axes are shown.

The output waveform211-11in the X-axis direction takes a positive value when the operation input device1is moved rightward, and takes a negative value when the device is moved leftward. Further, the output waveform212-11in the Y-axis direction takes a positive value when the operation input device1is moved toward the front of the user, and takes a negative value when the device is moved in the depth direction of the user.

FIG. 18Ashows a waveform231of the rightward operation, a waveform232of the leftward operation, a waveform233of the operation toward the front, a waveform234of the operation in the depth direction, a waveform235of the operation of describing a circle clockwise, and a waveform236of the operation of describing a circle counter-clockwise.

As the waveforms231to234in the reciprocating operation, waveforms of operation in the respective directions twice are shown. As the waveforms235,236in the operation of describing circles, waveforms of operation in the respective directions three times are shown.

In the waveform231of the rightward operation, the output waveform211-11in the X-axis direction largely changes in the positive direction. In the waveform232of the leftward operation, the output waveform211-11in the X-axis direction largely changes in the negative direction.

In the waveform233of the operation toward the front, the output waveform212-11in the Y-axis direction largely changes in the negative direction. In the waveform234of the operation in the depth direction, the output waveform212-11in the Y-axis direction largely changes in the positive direction.

As described above, the gestures of the reciprocating operation may be detected from the changes of the output waveforms of the angular velocity sensor13as gesture detecting means respectively corresponding to them.

In the waveform235of the operation of describing a circle clockwise, the output waveform211-11in the X-axis direction first changes, and then, the output waveform212-11in the Y-axis direction changes with a phase difference of 90 degrees. Further, in the waveform236of the operation of describing a circle counter-clockwise, the output waveform212-11in the Y-axis direction first changes, and then, the output waveform211-11in the X-axis direction changes with a phase difference of 90 degrees.

As described above, regarding the gesture of the operation of describing a circle, its rotational direction may be detected based on either the output waveform211-11in the X-axis direction or the output waveform212-11in the Y-axis direction first changes.

FIG. 18Bshows outputs of the angular velocity sensor13when the user executes the operation of describing a circle on various target objects.

InFIG. 18B, an output waveform211-12in the X-axis direction and an output waveform212-12in the Y-axis direction of the angular velocity sensor13are shown.

FIG. 18Bshows an output waveform251when a palm is used as a target surface, an output waveform252when there is no contact with the object surface (in the air), an output waveform253when a desk surface is used as the target surface, an output waveform254when jeans are used as the target surface, an output waveform255when a housing of a personal computer is used as the target surface, and an output waveform256when a liquid crystal display of a personal computer is used as the target surface.

As shown inFIG. 18B, the respective waveforms251to256show the similar waveforms. This means that the gestures of the operation of describing circles may be detected independent of the target object.

FIG. 18Cshows output waveforms of the angular velocity sensor13when the user executes flick operation. InFIG. 18C, an output waveform212-13in the Y-axis direction of the angular velocity sensor13is shown.

FIG. 18Cshows an output waveform271when a desk surface is used as the target surface and an output waveform272in no contact with the object surface (in the air). In the output waveform271when the desk surface is used as the target surface and the output waveform272when there is no contact with the object surface, respectively, six flicks are shown and the times to flick are sequentially longer from the left side.

The output waveform271when the desk surface is used as the target surface and the output waveform272when there is no contact with the object surface show the similar waveforms.

When flicking is executed on the desk, the contact part22is pressed down at flicking. The locations where the contact part22is pressed down at flicking are shown by flicking locations291-1to291-6.

The flicking time in the flicking location291-N (N is a natural number from “1” to “6”) is longer as the N is larger as described above.

In this manner, the flick gestures may be detected independent of the target object, and the differences in time of flicking may be detected.

The gestures may be detected based on the outputs of the image sensor21.

The operation input device1may be electronic apparatus.

The embodiments of the invention are not limited to the above described embodiments, but various changes may be made without departing from the scope of the invention.

The present application contains subject matter related to that disclosed in Japanese Priority Patent Application JP 2009-286218 filed in the Japan Patent Office on Dec. 17, 2009, the entire contents of which is hereby incorporated by reference.