ELECTRONIC APPARATUS, METHOD FOR CONTROLLING ELECTRONIC APPARATUS, AND NON-TRANSITORY COMPUTER READABLE MEDIUM

An electronic apparatus includes: control means for performing control to record, in recording means, characteristic data on a line-of-sight; authentication means for authenticating a user; and detection means for detecting, in a case where characteristic data on a line-of-sight of the authenticated user is recorded in the recording means, a line-of-sight of the user by using the recorded characteristic data on the line-of-sight.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an electronic apparatus, a method for controlling an electronic apparatus, and a non-transitory computer readable medium.

Background Art

For electronic apparatus, line-of-sight input technologies as technologies for receiving input of line-of-sights are known. PTL 1 discloses a video camera for detecting a viewpoint position (position that user views). For biometric authentication, a technology for specifying (authenticating) a user individual by an electronic apparatus is known. PTL 2 discloses a technology of iris authentication for specifying a user individual by using the iris.

In technologies of line-of-sight input, because the shifts of the radius of rotation of the eyeball and the line-of-sight during opening/closing of the pupil vary depending on users, line-of-sight characteristic data called calibration data, which is data for correcting the shifts, is necessary for each user. However, when an electronic apparatus is used by a plurality of persons, calibration data for the plurality of persons needs to be registered (acquired) in advance, and line-of-sight input is not accurately detected in some cases after a user using the electronic apparatus is changed.

Even when calibration data for a plurality of persons have been registered in advance, a user needs to select his/her registered content when using a function corresponding to line-of-sight input, which burdensome to the user. Unless a user remembers which of a plurality of pieces of registered calibration data is his/her own data, there is a burden that the user needs to acquire calibration data again.

Citation List

Patent Literature

PTL 2 Japanese Translation of PCT Application No. H08-504979

SUMMARY OF THE INVENTION

It is therefore an object of the present disclosure of technology to reduce a cumbersome process when a plurality of persons use line-of-sight input.

A first aspect of the present invention is an electronic apparatus including at least one memory and at least one processor which function as: control means for performing control to record, in recording means, characteristic data on a line-of-sight of; authentication means for authenticating the user; and detection means for detecting, in a case where characteristic data on a line-of-sight of the authenticated user is recorded in the recording means, a line-of-sight of the user by using the recorded characteristic data on the line-of-sight.

A second aspect of the present invention is a method for controlling an electronic apparatus, the method including: a control step for performing control to record, in recording means, characteristic data on a line-of-sight of; an authentication step for authenticating a user; and a detection step for detecting, in a case where characteristic data on a line-of-sight of the authenticated user is recorded in the recording means, a line-of-sight of the user by using the recorded characteristic data on the line-of-sight.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention are described in detail below with reference to the accompanying drawings.

Embodiment 1 Configuration of Imaging Apparatus

Referring toFIG.1, a configuration of an imaging apparatus100according to the present embodiment is described.FIG.1is a configuration diagram of the imaging apparatus100according to Embodiment 1. The imaging apparatus100is not limited to a camera, such as a digital camera, as long as line-of-sight input from a user can be received, and may be other electronic apparatus such as a smartphone and a personal computer.

The imaging apparatus100includes a lens unit101, an imaging element102, a light source unit103, an image processing circuit109, a recording unit110, a memory circuit111, a control circuit112, a display unit113, a detection unit114, and an iris authentication unit115. The imaging apparatus100further includes an operation unit116, a display unit117, line-of-sight imaging element118, and a bus120.

The lens unit101condenses incident light from a subject onto the imaging element102. For example, the lens unit101includes a plurality of lenses and an aperture.

The imaging element102acquires an image (image data) in which a subject appears on the basis of light condensed by the lens unit101. The light source unit103includes a light source for irradiating the subject with light.

The image processing circuit109(development processing unit) performs development processing, such as color matrix processing and gamma processing, on an image acquired from the imaging element102. Note that, in these pieces of processing, the image processing circuit109records the image in the memory circuit111as necessary. The image processing circuit109outputs the processed image to the display unit113, the display unit117, and the recording unit110.

The recording unit110records (stores) images and programs therein. The recording unit110records therein, for each user, iris data indicating characteristics of an iris and calibration data associated with the iris data. In the present embodiment, the calibration data is line-of-sight characteristic data used for correcting shift of the radius of rotation of the eyeball and line-of-sights during opening/closing of the pupil for each user and specifying a position of the viewpoint (viewpoint position). In other words, the calibration data is information that associates the direction of the eye of a user and the position (viewpoint position) that the user actually views.

The control circuit112executes control of a part of image processing of the image processing circuit109and driving of the imaging element102. The control circuit112executes the control of display of the display unit113and the display unit117, the control of reception of instructions from the operation unit116, and the control of passing of data to the memory circuit111and the recording unit110. Without being limited to the configuration in which the control circuit112controls the imaging element102and the image processing circuit109, the imaging element102or the image processing circuit109itself may include a control unit.

The detection unit114detects a line-of-sight (viewpoint position; viewing position; line-of-sight position) of a user on the basis of an image in which the eye (motion of eye) of the user appears. In the present embodiment, the line-of-sight (viewpoint position) of the user is detected in a state in which the user looks in the display unit117through the eyepiece finder. The detection unit114acquires calibration data for each user. Note that, in the present embodiment, a user can operate each functional unit in the imaging apparatus100depending on a viewpoint position. For example, when a viewpoint position on the display unit117is detected, the imaging apparatus100can execute autofocus (AF) for the detected position. Note that the detection unit114detects a viewpoint position of a user on the basis of an image in which the eye of the user appears that is acquired from the line-of-sight imaging element118.

The iris authentication unit115acquires iris data, which is information unique to a user (biometric data), and authenticates the user (performs iris authentication) to associate calibration data and iris data (user individual) with each other.

The operation unit116generates an operation signal corresponding to user operation. The operation unit116includes a touch panel, a dial, a 4-direction key, a shutter button, a power switch, a menu button, a play button, and the like.

The line-of-sight imaging element118acquires an image in which the eye of a user appears, which is an image used for detecting a line-of-sight of the user. In the present embodiment, the line-of-sight imaging element118includes IR pixels having sensitivity to infrared light. The bus120is a common path used for the functional units in the imaging apparatus100to exchange data.

Flow of Acquiring and Recording Calibration Data

Referring toFIG.2, the flow of processing from when the imaging apparatus100is started to when calibration data is acquired and the calibration data is recorded in the recording unit110is described.FIG.2illustrates functional units for executing processing related to the flow of acquiring and recording calibration data in the imaging apparatus100. Note that the light source unit103includes a light source201, a light source control unit202, and a prism205. The light source201applies infrared light (IR light). The light source control unit202controls the light source201to irradiate an eye200of a user with light. The prism205is disposed in front of the display unit117, and refracts light applied from the eye200of the user so as to enter the line-of-sight imaging element118.

First, the light source control unit202controls the light source201such that the eye200of the user is irradiated with infrared light from the light source201. Then, the infrared light reflected by the eye200of the user is refracted by the prism205to enter the line-of-sight imaging element118. The line-of-sight imaging element118has sensitivity to infrared light, and hence can acquire an image in which the eye200of the user appears.

When calibration data for the user authenticated by the iris authentication unit115is not recorded in the recording unit110, the data detection unit204included in the detection unit114acquires calibration data in accordance with the image of the eye200acquired by the line-of-sight imaging element118. Note that, when calibration data has been acquired in advance, the detection unit114can detect a position (viewpoint position) that the user views on the display unit117in accordance with the image of the eye200of the user acquired by the line-of-sight imaging element118.

At the time of start of the imaging apparatus100, the iris authentication unit115acquires iris data on the basis of an image from the line-of-sight imaging element118. A pattern of the iris part varies among persons, and hence an individual can be specified by treating information obtained by quantifying the characteristic amount of the pattern as iris data. The iris authentication unit115records iris data and calibration data in the recording unit110in relation to (in association with) each other. Note that, when calibration data to be associated with iris data is not recorded in the recording unit110, the data detection unit204acquires calibration data (executes calibration) as described above. Note that the iris authentication unit115may detect an iris pattern when an image is acquired from the line-of-sight imaging element118or when contact of eye (approach of eye) indicating that a user looks in an EVF as the display unit117is detected by an eyepiece detection unit (not shown), without being limited to the timing of the start of the imaging apparatus. Note that the eye-contact detection can be performed, for example, on the basis of a change in light amount of infrared light reflected by an object that is received by an eyepiece detection unit when the eyepiece detection unit irradiates the object with infrared rays.

Flow of Reading Calibration Data

FIG.3is a diagram illustrating a sequence of reading of calibration data with time (timings T310to T313).FIG.3illustrates processing for line-of-sight detection by the iris authentication unit115and the detection unit114together with the processing by the light source201and the line-of-sight imaging element118.

At the timing T310, the light source201irradiates (normally irradiates) the eye of the user with light, and the line-of-sight imaging element118acquires an image of the eye of the user. Note that, when the light amount is not sufficient in the normal application (application for line-of-sight detection) in the detection of the iris pattern, the light source201may increase the light amount of output of light to be applied. In other words, light having light amounts different between iris authentication and line-of-sight detection may be applied from the light source201.

At the timing T311, the iris authentication unit115acquires (extracts) an iris pattern from the acquired image of the eye.

At the timing T312, when iris data that matches the iris pattern extracted in the recording unit110is recorded in the recording unit110, the control circuit112acquires calibration data that has been recorded in the recording unit110in association with the iris data. The control circuit112sets (determines) the acquired calibration data as data used to detect a line-of-sight.

After the timing T313, on the basis of the set calibration data, the detection unit114acquires a viewpoint position of the user from the image acquired for line-of-sight detection (detects line-of-sight of user).

Processing for Setting Calibration Data

Referring toFIG.4, processing for setting calibration data as data used for line-of-sight detection is described below.FIG.4is a flowchart illustrating setting processing for calibration data. The processing in this flowchart starts when the power switch is turned on and the imaging apparatus100starts. Note that the processing in this flowchart is implemented when the control circuit112reads and executes a program stored in the recording unit110(storage medium).

In S401, the control circuit112controls the iris authentication unit115to detect an iris pattern of a user.

In S402, the control circuit112determines whether iris data (iris pattern data) that matches the iris pattern detected in S401is recorded in the recording unit110. When an iris pattern that matches the iris pattern detected in S401is recorded, the flow proceeds to S403. Otherwise, the flow proceeds to S404. In other words, in S402, a user is specified by an iris pattern (iris authentication), and it is determined whether the user executed calibration in the past and recorded calibration data in the recording unit110.

In S403, the control circuit112sets calibration data that is recorded in the recording unit110in association with the detected iris pattern as data used to detect a line-of-sight. In other words, in the present embodiment, in S402and S403, the control circuit112executes the iris authentication and sets calibration data that recorded in the recording unit110in association with the iris-authenticated user as data used for line-of-sight detection.

In S404, the control circuit112displays a selection screen for selecting whether to perform calibration on the display unit117. After that, the control circuit112determines whether a user who has checked the selection screen has requested (instructed) calibration. When the user requests calibration, the flow proceeds to S405. Otherwise, the flow proceeds to S407. Note that, without inquiring of the user whether to perform calibration, the flow may proceed to S405without exception, or proceed to S407without exception.

In S405, the control circuit112controls the detection unit114(data detection unit204) to newly acquire calibration data (to execute calibration).

In S406, the control circuit112sets (reflects) the calibration data acquired in S405as data used to detect a line-of-sight. The control circuit112records (saves) the iris data acquired in S401and the calibration data acquired in S405in the recording unit110in association with each other. In this manner, a user can use the calibration data acquired in S405from the next time.

In S407, the control circuit112sets standard data, which is general calibration data set in advance, as data used for line-of-sight detection. The standard data may be average data of calibration data for general persons (plurality of persons). Note that the standard data may be set at the time of shipping of the imaging apparatus100from the factory, or may be set by a user in advance. A plurality of pieces of standard data prepared depending on the sex, age, race, and height may be recorded in the recording unit110in advance, and in S407, a user may select data used for line-of-sight detection from among the plurality of pieces of standard data.

As described above, in Embodiment 1, when the imaging apparatus executes iris authentication (personal authentication) and calibration data associated with an iris-authenticated user is recorded in the recording unit, the imaging apparatus uses the calibration data for the detection of the line-of-sight of the user. In other words, a user who has used the imaging apparatus once is specified by acquiring iris data at the time of the start of the imaging apparatus, and hence calibration data on the user recorded in the imaging apparatus can be smoothly used. Even for a user who has never used the imaging apparatus, the user and calibration data can be easily recorded (registered) in association with each other by iris data acquired at the time of start of the imaging apparatus. Thus, even when a plurality of persons use the imaging apparatus, the function corresponding to line-of-sight input can be smoothly used.

In Embodiment 1, the light source applies infrared light for line-of-sight detection and iris data acquisition, but in Embodiment 2, infrared light is used for line-of-sight detection and visible light (RGB) is used for iris data acquisition. Note that a configuration of an imaging apparatus100according to Embodiment 2 is the same as the configuration of the imaging apparatus100according to Embodiment 1 as illustrated inFIG.1andFIG.2. In the following, only differences from Embodiment 1 are described, and descriptions of the same parts are omitted. Note that visible light is light in wavelengths which is visible to the eye of human, and is, for example, light having a wavelength from 0.3 µm to 0.7 µm. Infrared light is light whose wavelength is longer than that of red of visible light and which is invisible to the eye of human, and is, for example, light having a wavelength from 0.7 µm to 1 mm.

In the present embodiment, the line-of-sight imaging element118includes RGB pixels having sensitivity to visible light (RGB) in addition to IR pixels having sensitivity to infrared light. In the present embodiment, the display unit117operates also as a light source for visible light. As specific processing, the line-of-sight imaging element118receives reflected light of infrared light from the light source201by the IR pixels, and outputs the resultant to the detection unit114as an image. The line-of-sight imaging element118receives reflected light of visible light from the display unit117, which is an EVF, by the RGB pixels, and outputs the resultant to the iris authentication unit115as an image.

Referring toFIG.5, reading of calibration data is described below.FIG.5is a diagram illustrating a sequence of reading of calibration data in Embodiment 2. In the present embodiment, processing at timing T510is executed instead of the processing at the timing T310in the sequence illustrated inFIG.3according to Embodiment 1. In the present embodiment, at the timing T510, the display unit117applies visible light, and acquires reflected light of the visible light at the eye of the user, so that the line-of-sight imaging element118acquires an RGB image as an image of the eye of the user. Processing after the timing T311is the same as the processing according to Embodiment 1.

Note that, in the present embodiment, the imaging element for line-of-sight detection acquires both an image for line-of-sight detection and an image for iris authentication. However, the configuration is not limited thereto. A first imaging element may acquire an image for line-of-sight detection (receive infrared light), and a second imaging element different from the first imaging element may acquire an image for iris authentication (receive visible light). In the present embodiment, the configuration in which two light sources, that is, a light source for line-of-sight detection and a light source for iris authentication, are provided has been described. However, line-of-sight detection and iris authentication may be performed by a single light source. Alight source capable of emitting both infrared light and visible light may be employed such that light is applied while application timing is switched between iris authentication and line-of-sight detection.

As described above, in Embodiment 2, light having a wavelength different from that of light for detecting a line-of-sight is used to acquire iris data. Using different light beams for imaging can facilitate the change in resolution and exposure conditions during imaging between line-of-sight detection and iris authentication, and hence optimal conditions depending on usages can be set. It becomes easier to separate the functional unit for detecting a line-of-sight and the functional unit for performing iris authentication, and hence the present embodiment can be applied to various apparatus.

In Embodiment 3, the imaging apparatus100specifies a user (personal authentication; identification) by fingerprint authentication instead of iris authentication. In the following, descriptions of the same parts as in Embodiment 1 are omitted, and only differences are described in detail.

The imaging apparatus100according to the present embodiment includes a fingerprint authentication unit600instead of the iris authentication unit115in the configuration of the imaging apparatus100according to Embodiment 1. Note that, inFIG.6, the light source unit103, the line-of-sight imaging element118, and the display unit117illustrated inFIG.2are collectively illustrated as a line-of-sight detection block610.

Instead of the processing where the iris authentication unit115performs iris authentication (personal authentication) by using an image taken by the line-of-sight imaging element118, the fingerprint detection unit601detects a fingerprint pattern of a user and the fingerprint authentication unit600acquires fingerprint data to perform fingerprint authentication. The recording unit110records fingerprint data and calibration data therein in association with each other.

In other words, in the present embodiment, at the time of start of the imaging apparatus100, the control circuit112controls the fingerprint detection unit601to detect a fingerprint pattern of a user, and controls the fingerprint authentication unit600to acquire fingerprint data. Then, when fingerprint data that matches the acquired fingerprint data is recorded in the recording unit110, the control circuit112sets calibration data that is recorded in association with the acquired fingerprint data as data used for line-of-sight detection. Through such processing, the same effect as in Embodiment 1 can be obtained even when a fingerprint is used to perform personal authentication.

Note that, in the present embodiment, fingerprint authentication may be mounted to other electronic apparatus and personal computers that do not have the iris authentication function, such as smartphones, so that calibration data and fingerprint data are associated with each other.

Freely selected personal authentication may be used instead of iris authentication and fingerprint authentication. For example, face authentication may be performed by detecting the face of a user and acquiring face data. Personal authentication may be performed by biometric authentication such as a vein pattern. Any authentication method may be used as long as data for authentication (biometric data) and calibration data can be associated with each other.

As described above, in the present embodiment, an example where a user is specified by personal authentication other than iris authentication and calibration data corresponding to the user is read has been described. In this manner, calibration data can be associated with a user without depending on specific personal authentication. For example, in the case of the imaging apparatus according to Embodiment 1, iris authentication cannot be performed unless the eye of a user contacts (is close to) an eyepiece finder, but fingerprint authentication (personal authentication) can be performed when the finger of a user grips a grip. In this manner, by changing the method for personal authentication, an effect that the range of acquisition timing can be enlarged can be obtained.

According to the present invention, a cumbersome process when a plurality of persons use line-of-sight input can be reduced.

While the present invention has been described in detail on the basis of the preferred embodiments, the present invention is not limited to the specific embodiments, and various modes within the range not departing from the gist of the invention are also included in the present invention. Furthermore, each of the above-mentioned embodiments is merely illustrative of one embodiment of the present invention, and the embodiments can be combined as appropriate.

For example, face authentication may be used when the brightness of ambient light around the imaging apparatus100is equal to or more than predetermined brightness, and iris authentication (or fingerprint authentication) may be performed when the brightness of ambient light around the imaging apparatus100is less than (darker than) predetermined brightness. Note that, when the brightness of ambient light around the imaging apparatus100is less than predetermined brightness, a user may be notified by a notification unit such that the user makes the eye close to the eyepiece finder. In this manner, the use of a plurality of kinds of authentication enables users to be authenticated in many cases.

Other Embodiments

The present invention is not limited to the above-mentioned embodiments, and can be variously changed and modified without departing from the spirit and scope of the present invention. Thus, the following claims are attached in order to clarify the scope of the present invention.