Camera that uses light from plural light sources disposed on a device

In one aspect, a device includes a processor, a first light source accessible to the processor, a second light source different from the first light source that is accessible to the processor, a camera accessible to the processor, and storage accessible to the processor. The storage bears instructions executable by the processor to actuate the first light source to execute a first function using the camera and actuate the second light source to execute a second function different from the first function using the camera.

FIELD

The present application relates generally to cameras that use light from plural light sources disposed on a device.

BACKGROUND

As recognized herein, devices are often provided with a camera that has limited functionality. This limited functionality often prevents such a camera from being used for multiple purposes, which is undesirable for the increasingly complex devices that are coming to market.

SUMMARY

Accordingly, in one aspect a device includes a processor, a first light source accessible to the processor, a second light source different from the first light source that is accessible to the processor, a camera accessible to the processor, and storage accessible to the processor. The storage bears instructions executable by the processor to actuate the first light source to execute a first function using the camera and actuate the second light source to execute a second function different from the first function using the camera.

In another aspect, a method includes controlling a first light source disposed on a device at a first location to generate light, generating at least a first image at a camera disposed on the device based at least in part on at least some of the light from the first light source, and executing iris recognition at least in part using the at least first image. The method also includes controlling a second light source disposed on the device at a second location different from the first location to generate light, generating at least a second image at the camera based on at least some of the light from the second light source, and executing eye tracking on the at least second image.

In still another aspect, a device includes a processor and a camera module accessible to the processor. The camera module includes an imager, a first light source spaced a first distance away from the imager, and a second light source spaced a second distance away from the imager. The second distance is greater than the first distance.

DETAILED DESCRIPTION

This disclosure relates generally to device-based information. With respect to any computer systems discussed herein, a system may include server and client components, connected over a network such that data may be exchanged between the client and server components. The client components may include one or more computing devices including televisions (e.g., smart TVs. Internet-enabled TVs), computers such as desktops, laptops and tablet computers, so-called convertible devices (e.g., having a tablet configuration and laptop configuration), and other mobile devices including smart phones. These client devices may employ, as non-limiting examples, operating systems from Apple, Google, or Microsoft. A Unix or similar such as Linux operating system may be used. These operating systems can execute one or more browsers such as a browser made by Microsoft or Google or Mozilla or other browser program that can access web applications hosted by the Internet servers over a network such as the Internet, a local intranet, or a virtual private network.

As used herein, instructions refer to computer-implemented steps for processing information in the system. Instructions can be implemented in software, firmware or hardware; hence, illustrative components, blocks, modules, circuits, and steps are set forth in terms of their functionality.

A processor may be any conventional general purpose single- or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, and control lines and registers and shift registers. Moreover, any logical blocks, modules, and circuits described herein can be implemented or performed, in addition to a general purpose processor, in or by a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device such as an application specific integrated circuit (ASIC), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor can be implemented by a controller or state machine or a combination of computing devices.

Any software and/or applications described by way of flow charts and/or user interfaces herein can include various sub-routines, procedures, etc. It is to be understood that logic divulged as being executed by, e.g., a module can be redistributed to other software modules and/or combined together in a single module and/or made available in a shareable library.

“A system having one or more of A, B, and C” (likewise “a system having one or more of A, B, or C” and “a system having one or more of A, B, C”) includes systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.

Now specifically in reference toFIG. 1, it shows an example block diagram of an information handling system and/or computer system100. Note that in some embodiments the system100may be a desktop computer system, such as one of the ThinkCentre® or ThinkPad® series of personal computers sold by Lenovo (US) Inc. of Morrisville, N.C., or a workstation computer, such as the ThinkStation®, which are sold by Lenovo (US) Inc. of Morrisville, N.C.; however, as apparent from the description herein, a client device, a server or other machine in accordance with present principles may include other features or only some of the features of the system100. Also, the system100may be, e.g., a game console such as XBOX® or Playstation®.

As shown inFIG. 1, the system100includes a so-called chipset110. A chipset refers to a group of integrated circuits, or chips, that are designed to work together. Chipsets are usually marketed as a single product (e.g., consider chipsets marketed under the brands INTEL®, AMD®, etc.).

In the example ofFIG. 1, the chipset110has a particular architecture, which may vary to some extent depending on brand or manufacturer. The architecture of the chipset110includes a core and memory control group120and an I/O controller hub150that exchange information (e.g., data, signals, commands, etc.) via, for example, a direct management interface or direct media interface (DMI)142or a link controller144. In the example ofFIG. 1, the DMI142is a chip-to-chip interface (sometimes referred to as being a link between a “northbridge” and a “southbridge”).

The interfaces of the I/O hub controller150provide for communication with various devices, networks, etc. For example, the SATA interface151provides for reading, writing or reading and writing information on one or more drives180such as HDDs, SDDs or a combination thereof, but in any case the drives180are understood to be, e.g., tangible computer readable storage mediums that may not be transitory signals. The I/O hub controller150may also include an advanced host controller interface (AHCI) to support one or more drives180. The PCI-E interface152allows for wireless connections182to devices, networks, etc. The USB interface153provides for input devices184such as keyboards (KB), mice and various other devices (e.g., cameras, phones, storage, media players, etc.).

Still in reference toFIG. 1, a camera191is shown which is configured for gathering one or more images in accordance with present principles and providing input related thereto to the processor122. The camera191is understood to be able to sense infrared (IR) light and to generate images at least based on the IR light. Notwithstanding, the camera191may also be a thermal imaging camera, a digital camera such as a webcam, a three-dimensional (3D) camera, etc. The system100also includes a first light source193and a second light source195. The light sources193and195may be IR light emitting diodes (LEDs) in some embodiments, or still other types of IR light emitters in other embodiments.

In addition to the foregoing, the system100may also comprise a chip197made of a material such as silicon and having stored thereon one or more different types of images and/or templates associated with a particular user for secure authentication of the user outside of a guest operating system (GOS) executed by the system100, such as Windows, Linux, or Max OSX. In some embodiments, the chip197may be a hub storing plural types authentication templates, such as reference templates to perform fingerprint matching and iris matching, and hence authentication, using fingerprint and iris input from the user when the user seeks access to the system100.

Still further, the system100includes an embedded controller199, which will be described in more detail below. Additionally, though now shown for clarity, in some embodiments the system100may include a gyroscope for sensing and/or measuring the orientation of the system100and providing input related thereto to the processor122, an accelerometer for sensing acceleration and/or movement of the system100and providing input related thereto to the processor122, and an audio receiver/microphone providing input to the processor122based on, e.g., a user providing audible input to the microphone. Still further, and also not shown for clarity, the system100may include a GPS transceiver that is configured to receive geographic position information from at least one satellite and provide the information to the processor122. However, it is to be understood that another suitable position receiver other than a GPS receiver may be used in accordance with present principles to determine the location of the system100.

Before moving on toFIG. 2, it is to be understood that an example client device or other machine/computer may include fewer or more features than shown on the system100ofFIG. 1. In any case, it is to be understood at least based on the foregoing that the system100is configured to undertake present principles.

Turning now toFIG. 2, it shows example devices communicating over a network200such as the Internet in accordance with present principles. It is to be understood that each of the devices described in reference toFIG. 2may include at least some of the features, components, and/or elements of the system100described above. In any case,FIG. 2shows a notebook computer202, a desktop computer204, a wearable device206such as a smart watch, a smart television (TV)208, a smart phone210, a tablet computer212, and a server214such as an Internet server that may provide cloud storage accessible to the devices202-212. It is to be understood that the devices202-214are configured to communicate with each other over the network200to undertake present principles.

It is to also be understood in reference toFIG. 2that the block diagram of the smart phone210as shown therein is from a front perspective of the phone210when held upright, with the front of the phone210bearing a touch-enabled display. Furthermore, the phone210may include a camera216centrally disposed at a top portion of the phone210(e.g., at or near the top edge) relative to the upright orientation of the phone210shown. The phone210also includes a first light source218and a second light source220. The camera216and light sources218,220may be used in accordance with present principles as set for further below.

Before describingFIG. 3, it is to be understood that when executing eye tracking, it is desirable for a device such as the system100to actuate a light source thereon at a particular angle relative to its camera to create a red eye effect because this red eye effect is useful for performing eye tracking such as by tracking the movement of the user's eyes based on movement of the red eye effect as shown in plural images of the user's eyes taken relatively close in time to each other. Accordingly, it is desirable to juxtapose a light source to be used for eye tracking relatively close to the camera (such as substantially “on axis” with the camera) to create the red eye effect at the camera based on the path of light from the light source, reflected off the user's eye, and then to the camera.

Notwithstanding, it is to also be understood in accordance with present principles that for performing iris recognition, such a red eye effect is undesirable and can impede the ability of the camera to gather a detailed and accurate image of the user's iris for iris identification. Thus, it is also recognized herein that it is desirable to juxtapose a light source to be used for iris recognition father away from the camera than a light source for eye tracking to thus minimize or eliminate the potential for red eye effect at the camera based on the path of light from the farther away light source, reflected off the user's eye, and then to the camera.

Now referring toFIG. 3, it shows an example embodiment of a device300in accordance with present principles, with portions of the device300cut away for clarity. It is to be understood that the device300may include at least some of the components described above in reference to the system100. In any case, the device300includes a housing302bearing thereon a camera module304. The camera module304comprises a camera306, a first light source308and a second light source310. Note that the camera306, first light source308, and second light source310all face outward away from the housing302. Also note that the second light source310is spaced (e.g., at least laterally on the housing) a distance away from the camera306that is closer than a distance away that the first light source308is spaced from the camera306. As a point of reference and in some embodiments, the positioning of the camera306on the device300may correspond to the positioning of the camera216on the phone210described above. Furthermore, the positioning of the light source308may correspond to the positioning of the light source220on the phone210, while the positioning of the light source310may correspond to the positioning of the light source218on the phone210.

Describing the camera306is more detail, it may include an imager312for gathering images based on light that is sensed thereat, such as IR light, in accordance with present principles. The camera306also includes at least one lens314, and at least one IR bandpass filter316. The filter316may be configured to permit light in the IR band to pass therethrough from exterior to the device300to the imager312while disallowing light in at least one other band from passing therethrough. Furthermore, note that the camera306may include glass318(which may be made integral with the housing302) through which light may pass and which may protect the other portions of the camera306.

Also, note that an axis320is established along a direction of the camera306that faces outwardly away from the device300to gather at least one image based on light from outside the device300. The axis320may also be perpendicular to a plane established by the outward-facing surface of the glass318and/or IR bandpass filter316.

Still in reference toFIG. 3and describing the first light source308in more detail, it may include at least one IR light emitting diode (LED)322that emits light therefrom and through glass324, where the glass may protect the light source308(and may be made integral with the housing302). As may be appreciated fromFIG. 3, the LED322is disposed on the housing302at an angle beta relative to an axis326perpendicular to a plane established by the outward-facing surface of the glass324and/or a plane established by the surface of the housing302bearing the glass324.

Now describing the second light source310, it may include at least one IR LED328that emits light therefrom and through glass330, where the glass may protect the light source310(and may be made integral with the housing302).FIG. 3shows that the LED328is disposed on the housing302at an angle alpha relative to an axis332perpendicular to a plane established by the outward-facing surface of the glass330and/or a plane established by the surface of the housing302bearing the glass330. Note that the angle alpha is a smaller angle than the angle beta at which the light source308is oriented to emit light.

Accordingly, it may be appreciated that owing to the greater distance between the light source308and camera306than between the light source310and the camera306, and also due to the larger angle beta than the angle alpha, light from the source308may reflect off an eye334of a user and to the camera306for generation of an image without a red eye effect for performing iris recognition using image that is generated. It may also be appreciated that owing to the relatively smaller distance between the source310and camera306and lesser angle alpha, light from the source310may reflect off the eye334and to the camera306for generation of an image with a red eye effect for performing eye tracking using the image that is generated.

Before moving on to the description ofFIG. 4, it is to be understood in reference toFIG. 3that each of the light sources308and310may in some embodiments include sets of LEDs. It is to be further understood that in some embodiments, more than two light sources may be used and may be disposed at other locations on the device, such as, e.g., on the other side of the camera306relative to the perspective shown inFIG. 3and/or on the same side of the camera306as the light sources308and310are disposed as shown inFIG. 3(and optionally, disposed along the same line established by the light sources308and310). Moreover, in some embodiments there may be banks of light sources on each side of the camera306for use at different distances, angles, and/or fields of view of the user relative to the camera306.

Referring now toFIG. 4, it shows example logic that may be undertaken by a device such as the system100in accordance with present principles (referred to below as the “present device”). Beginning at block400, the logic actuates a first light source (such as the light source308described above) to emit light. The logic may do so at block400, e.g., responsive to a command from a user to power on the present device, responsive to a command from the user to authenticate the user for access to the present device, etc. The logic may also do so at block400using a guest operating system (GOS) such as Windows, Linux, or Max OSX, using a basic input/output system (BIOS), or using an embedded controller. It may be appreciated in accordance with present principles that using BIOS or an embedded controller may provide a secure path for authentication that is relatively less susceptible to being compromised by hacking and/or otherwise unauthorized manipulation.

In any case, from block400the logic moves to block402. At block402the logic actuates the camera to generate at least a first image at least in part using some of the light from the first light source such as may be reflected off a user's eye. The logic then proceeds to block404, where the logic executes iris recognition at least in part using the first image by, e.g., comparing the characteristics of the user's iris as shown in the first image to the characteristics of an iris shown in a reference image stored on the present device (e.g., on a pixel by pixel basis). Thereafter the logic proceeds to block406where, assuming the user's iris was recognized (e.g., based on a match of at least some iris characteristics to the reference image), the logic authenticates the user as being permitted to perform an operation at the present device.

In some embodiments, the logic may conclude at block406. However, in other embodiments such as where BIOS or an embedded controller is executing the present logic, the logic may optionally move to block408where the logic passes control of camera and/or of the first light source to a GOS starting up and/or executing on the present device so that the camera may be controlled by the GOS (and/or specific applications controlled by the GOS) for other purposes and/or for additional authentication.

Accordingly, the logic ofFIG. 5may be executed by a GOS on the present device once control of the camera and/or light source is passed thereto, and/or after block406ofFIG. 4. The logicFIG. 5begins at block500, where the logic actuates a second light source such as the light source310described above. The logic may do so at block500responsive to, e.g., a command to execute eye tracking received at the present device from a user, a command to execute eye tracking received from an application ran by the GOS which is seeking to use eye tracking, etc. Thus, note that an “eye tracking application” as described herein need not necessarily be an application dedicated only to eye tracking, but may be an application that uses eye tracking to perform other functions such as receiving user commands based on eye tracking, controlling movement of a cursor presented on a display of the present device to correspond to movement of the user's eyes across the display, using eye tracking as input for the user to play a game executing on the present device, determining information of interest to the user based on the user looking at a particular area of the display at which the information is presented, etc.

Regardless, and still in reference toFIG. 5, from block500the logic moves to block502where the logic actuates a camera on the present device to generate at least one second image, and in some embodiment plural images, based on at least in part using some of the light from the second light source such as may be reflected off a user's eye. The logic then proceeds to block504, where the logic executes eye tracking at least in part using the at least second image, such as at least in part by, e.g., tracking movement of the user's pupils as recognized based on movement of a red eye effect as shown in plural images generated at block502.

Continuing the detailed description in reference toFIG. 6, it shows example logic that may be undertaken by a device such as the system100in accordance with present principles (referred to below as the “present device”) for authenticating a user based on iris recognition using a “match on chip” process as disclosed herein. The logic ofFIG. 6may be executed by BIOS or an embedded controller and hence use a relatively secure path for authentication.

Beginning at block600, the logic receives a first image generated by a camera, such as the first image described above in reference toFIG. 4. The logic then moves to block602, where the logic accesses a chip with at least one reference image and/or template stored thereon. The chip may be made of a material such as silicon. In some embodiments, the chip may be a “hub” storing plural types images and/or templates for plural forms of authentication, such as reference templates to perform fingerprint matching, facial feature matching, voice matching, and iris matching, and hence authentication, respectively using fingerprints, facial features, a voice, and/or an iris of a user. The chip may also store information on gaze/eye tracking for authentication, such as may be the case where the user is to look at particular areas of the device (and in some embodiments, specifically the display) in a particular sequence.

In any case, the logic ofFIG. 6moves from block602to block604, where the logic compares the first image to at least one reference iris image and/or template to identify similarities and hence identify the user's iris as matching the iris image stored on the chip. Thus, at decision diamond606the logic determines whether a match exists. Responsive to a negative determination at diamond606, the logic moves to block608and denies access to the present device. However, responsive to an affirmative determination at diamond606, the logic moves to block610where the logic authenticates the user based on the match and then permits access to the device.

Before moving on to the description ofFIG. 7, it is to be understood in reference to the match on chip embodiment(s) disclosed herein that the reference images and/or templates may be “locked away” on this chip (and/or a host controller) such as for authentication (and also optionally, key exchange) at power on of the device. It is to also be understood that an application specific integrated circuit (ASIC) (e.g., on the chip) and/or an embedded controller which uses an encoded and/or dedicated link between the chip and the camera may be used (or if performing, e.g., fingerprint recognition, a finger print reader) to create a relatively secure path for authentication, although it is to also be understood that BIOS may access the chip to perform authentication. Furthermore, it is to be understood that the “back end” of the chip may connect to a system on a chip (SoC) and/or a CPU (e.g. the main processor of the present device) to pass control thereto upon successful authentication so that the CPU may boot the GOS and/or otherwise permit access to the present device.

Still further, in some embodiments the controller of the hub chip may mange which method(s) of authentication to use depending on an orientation of the device and/or other physical device configurations. Thus, if the device were a Lenovo Yoga convertible computer that has multiple configurations and/or modes of folding a panel bearing a display relative to a panel bearing a keyboard (e.g. tablet mode, tent mode, and clam shell mode), different authentication methods using the chip may be used. For example, in a tent mode, iris authentication may be used, in a clam shell mode, fingerprint authentication may be used, and in a tent mode, voice authentication may be used. Selection of which method to use during which physical device configuration may be made by a user e.g. using a UI presented on the display of the device.

Describing establishment of the reference data and/or templates on the chip, it is to be understood that a user may be prompted to create the reference data and/or templates, and/or provide the corresponding input (e.g. voice input, fingerprint input, iris input), when he or she starts the device for the first time and/or otherwise registers or enrolls himself or herself with the device. As but one example, a user may establish a fingerprint template by pressing their finger against a fingerprint sensor several times to create an adequate template for the fingerprint. In any case, that data is then saved to the chip (e.g., imprinted thereon) as the reference data and/or template. Also, it is to be understood that the chip itself may have encryption and/or other security protocols for security.

Now in reference toFIG. 7, it shows an example user interface (UI) presentable on a device such as the system100in accordance with present principles. The UI700includes at least a first setting702to configure how iris authentication is executed. Thus, a first check box704is presented that is selectable by a user to enable “secure” authentication at subsequent start ups of the device that does not involve use of a GOS. Also note that beneath the check box704for this secure authentication option is a radio button706selectable by a user to enable use of a software development kid (SDK) for the “secure” authentication, as well as a radio button708selectable by a user to enable use of a match on chip authentication for the “secure” authentication. SDKs for use in accordance with present principles will be described further below. In any case, note that a second check box710is also presented on the UI700that is selectable, in addition to or in lieu of selection of the box704, to enable iris authentication using a GOS and/or an application managed by the GOS for performing iris authentication (e.g., such as after an initial iris authentication using match on chip in response to a power on command).

Now describing SDKs in more detail, it is to be understood that SDKs and/or other software (e.g., an application programming interface (API)) developed to interface with an image signal processing (ISP) chip associated with a camera module, and/or to interface with the camera module another way, may be used by BIOS and/or a GOS in accordance with present principles, such as to perform iris identification and authentication. After iris authentication, the SDK may then be used to switch to activation of another diode than the one used for iris authentication to continue executing functions in accordance with present principles, such as to perform eye tracking. Even further, it is to be understood that the SDK may also be used to then switch back to iris authentication (e.g. during the same startup and/or login session for the device) to securely perform another iris authentication using a secure input/output camera path, an encrypted part of a processor, and/or match on chip, if desired.

It may now be appreciated that present principles provide for systems and methods of having one camera module with two IR diode sources of illumination to support multi-layered functionality using the camera. Furthermore, in some embodiments the camera module may be used via a secure path for authentication, and then the path may be switched to another camera I/O interface to the CPU for eye-tracking application use.

As those skilled in the art can appreciate, a BIOS may be a set of routines stored in read-only memory that enable a computer to start an operating system and to communicate with the various devices in a system, such as disk drives, keyboard, monitor, printer, and communications ports. In one embodiment, functions performed by BIOS may also be performed by other higher-level software application programs. In another embodiment, BIOS may be a Unified Extensible Firmware Interface (UEFI), which assists in control handoff of a computer system to an operating system during a pre-boot environment (e.g., after the computer system is powered on, but before the operating system starts).

Before concluding, it is to be understood that although a software application for undertaking present principles may be vended with a device such as the system100, present principles apply in instances where such an application is downloaded from a server to a device over a network such as the Internet. Furthermore, present principles apply in instances where such an application is included on a computer readable storage medium that is being vended and/or provided, where the computer readable storage medium is not a transitory signal and/or a signal per se.

While the particular CAMERA THAT USES LIGHT FROM PLURAL LIGHT SOURCES DISPOSED ON A DEVICE is herein shown and described in detail, it is to be understood that the subject matter which is encompassed by the present application is limited only by the claims.