Time of flight sensing for providing security and power savings in electronic devices

An electronic device includes a time-of-flight sensor configured to sense a distance between the electronic device and at least one object proximate the electronic device. Processing circuitry is coupled to the time-of-flight sensor and controls access to the electronic device based on the sensed distance. The electronic device may include a digital camera that the processing circuitry controls to perform facial or iris recognition utilizing the sensed distance from the time-of-flight sensor.

BACKGROUND

Technical Field

The present disclosure relates generally to time of flight sensing and more specifically to the utilization of time of flight sensing to provide security and power savings for electronic devices.

Description of the Related Art

Current mobile electronic devices such as laptop computers, tablet computers and smart phones increasingly provide a variety of different ways of controlling access to the electronic device in addition to conventional password access control. Many such devices, for example, now utilize iris recognition, facial recognition, or both, to authenticate a user and either provide or deny access of the user to the electronic device. Current iris and facial recognition systems typically utilize an iterative process with the electronic device providing feedback to the user so that the user may properly position his or her face to allow the recognition system to operate properly. For example, the electronic device displays user feedback as to the position of the user's face relative to the electronic device and provides the user an indication when the position of the user's face relative to the electronic device, including the distance from the electronic device, is proper so that the recognition system may begin capturing images to authenticate the user. As a result, current iris and facial recognition systems may result in a relatively slow and cumbersome process of authenticating the user due to the time required to properly position the users face relative to the electronic device and the subsequent computation to process captured images once the positioning is proper. This is particularly true in low light conditions during which a camera of the recognition system captures the images required for recognition at a lower rate (i.e., fewer frames per second).

In addition to security relating to providing or denying access to mobile electronic devices, security concerns may also arise due to the portable nature of such mobile electronic devices and the different environments in which these devices are utilized. For example, mobile electronic devices are commonly utilized in public settings such as in a coffee shop, a restaurant, a semi-public workshare type location, and so on. In these settings privacy concerns may also arise in relation to the content that a user of the mobile electronic device is viewing on a visual display of the device. The visual display of a mobile electronic device may in these settings be capable of being seen by a number of people seated at an adjacent table or otherwise proximate the user and his or her mobile electronic device. Moreover, the location of the setting or environment in which the user is utilizing his or her mobile electronic device may result in a higher likelihood that people proximate the user in the environment may be from competitor companies, such where the user is in a coffee shop in Silicon Valley or a city such as Seattle with a large number of high-tech and startup companies.

Yet another issue that arises for users of mobile electronic devices relates to the security of the device itself when in a public setting or environment. A laptop computer, for example, typically locks when a display of the computer is closed to prevent an unauthorized person from physically taking the computer and thereby gaining access to the computer. A person in such a public environment certainly does not want his or her mobile laptop computer to be stolen but may be much more concerned about a thief gaining access to the contents of that laptop computer than the computer itself. A sensor, such as a Hall sensor, is utilized in many laptop computers to sense the opening and closing of the display. Such sensors are inexpensive but may not reliably detect the closing of the display of the laptop computer. As a result, closing the display may not result in locking or preventing access to the computer until a person provides the required authentication information. In addition to locking the laptop computer closing the display also typically turns OFF the computer or places the computer in a low-power mode of operation upon detecting the closing of the display. Thus, if the sensor does not reliably detect the closing of the display the computer may not be turned off or placed in a low-power mode of operation, which results in unwanted power consumption and reduced battery life when the computer is under battery power.

There is a need for improving the security of mobile electronic devices and reducing the power consumption of such devices.

BRIEF SUMMARY

In one embodiment of the present disclosure, an electronic device includes a time-of-flight sensor configured to sense a distance between the electronic device and at least one object proximate the electronic device. Processing circuitry is coupled to the time-of-flight sensor and controls access to the electronic device based on the sensed distance. The electronic device may include a digital camera that the processing circuitry controls to perform facial or iris recognition utilizing the sensed distance from the time-of-flight sensor.

DETAILED DESCRIPTION

FIG. 1is a functional block diagram of an electronic device100including a time-of-flight (TOF) sensor102that detects a distance DTOFbetween the electronic device and a person104according to one embodiment of the present disclosure. In one embodiment, the detected distance DTOFis utilized in controlling a digital camera106in the electronic device100as part of authenticating the person104attempting to gain access to the electronic device, such as through facial or iris recognition, as will be explained in more detail below. In some embodiments, the distance DTOFdetected by the TOF sensor102is utilized in detecting closure of a lid of the electronic device100where the device is a laptop computer, and in locking access to the laptop computer in response to detecting closure of the lid along with placing the laptop computer in a low power mode of operation, as will be described in more detail below with reference toFIGS. 5A, 51B and 6. In still further embodiments, the TOF sensor102detects respective distances DTOFto multiple objects such as multiple people that are proximate the electronic device100and in this way provides privacy-related security for a user of the electronic device in public environments, as will be described in more detail below with reference toFIGS. 6-8.

In the following description, certain details are set forth in conjunction with the described embodiments to provide a sufficient understanding of the present disclosure. One skilled in the art will appreciate, however, that the subject matter of the present disclosure may be practiced without these particular details. Furthermore, one skilled in the art will appreciate that the example embodiments described below do not limit the scope of the present disclosure to the disclosed embodiments, and will also understand that various modifications, equivalents, and combinations of the disclosed embodiments and components of such embodiments are within the scope of the present disclosure. Embodiments including fewer than all the components of any of the respective described embodiments may also be within the scope of the present disclosure although not expressly described in detail below. Finally, the operation of well-known components and/or processes have not been shown or described in detail below to avoid unnecessarily obscuring the present disclosure.

The electronic device100in the example embodiment ofFIG. 1includes processing circuitry108that controls the overall operation of the electronic device and also executes applications or “apps”110that provide specific functionality for a user of the electronic device. The electronic device100may be any type of electronic device, such as a smart phone, a tablet computer, a laptop computer, or other type of mobile electronic device, and may also be a non-mobile type of device such as a desktop computer. Some aspects of the present disclosure are particularly advantageous for use in mobile electronic devices in public or non-private settings, and thus in the embodiments ofFIGS. 4A, 4B and 5-8described below the electronic device100is assumed to be a laptop computer.

InFIG. 1, a power management subsystem112of the electronic device100is coupled to the processing circuitry108, and would typically include a battery for powering the electronic device and also control circuitry for controlling power-related operating modes of the device such as charging of the battery, power-savings modes to extend batter life, and so on. The electronic device100further includes a visual display114such as a liquid crystal display (LCD) or a touch screen including a suitable touch visual display and a touch panel (not shown) attached to or formed as an integral part of the touch visual display. In operation, the where the visual display114is a touch screen, the touch screen senses touches of a user of the electronic device100and provides sensed touch information to the processing circuitry108to thereby allow the user to interface with and control the operation of the electronic device. The processing circuitry108also controls the touch screen114to display desired visual content on the touch visual display portion of the touch screen.

The electronic device1000further includes data storage or memory116coupled to the processing circuitry108for storing and retrieving data including the apps110and other software executing on the processing circuitry and utilized by the electronic device100during operation. Examples of typical types of memory116include solid state memory such as DRAM, SRAM and FLASH, solid state drives (SSDs), and could also include any other types of memory suited to the desired functionality of the electronic device1000including phase-change memory (PCM), digital video disks (DVDs), compact disk read-only (CD-ROMs), compact disk read-write (CD-RW) memories, magnetic tape, hard and floppy magnetic disks, tape cassettes, and so on. Input devices118are coupled to the processing circuitry108and may include a keypad, whether implemented through the visual display114where the display is a touch screen or separately, a pressure sensor, accelerometer, microphone, keyboard, mouse, and other suitable input devices. Output devices120are also coupled to the processing circuitry108and may include, for example, audio output devices such as a speaker, printers, vibration devices, and so on. The input devices118and output devices120collectively may include other types of typical communications ports for the electronic device100, such as USB ports, HDMI ports, and so on. The electronic device100further includes communications subsystems122coupled to the processing circuitry108and which may include Wi-Fi, GPS, cellular and Bluetooth subsystems for providing the device with the corresponding communications functionality. The specific type and number of input devices118, output devices120, communications subsystems122, and even the specific functionality of the power management subsystem112will of course depend on the specific type of the electronic device100.

The processing circuitry108controls the overall operation of the electronic device100including authenticating the person104to determine whether that person is an authorized user that should be granted access to the electronic device or an unauthorized person to which access should be denied. In operation of authenticating the person104, the TOF sensor102senses the distance DTOFbetween the electronic device100and the person104attempting to gain access to the electronic device. This may be initiated, for example, when the person104powers on the electronic device100in an attempt to gain access to the device. As will be described in more detail below with reference toFIG. 2, in sensing the distance DTOFto the person104the TOF sensor102transmits an optical pulse signal and then detects a time between transmission of this optical pulse signal and receipt of a portion of this transmitted optical pulse signal reflected off the person in the form of a returned optical pulse signal. The person104must be positioned within a field of view FOVTOFof the TOF sensor102for the sensor to properly illuminate the person with the transmitted optical pulse signal and sense the corresponding returned optical pulse signal reflected off the person.

The distance DTOFdetected by the TOF sensor102is supplied to the processing circuitry108and is utilized by the processing circuitry in authenticating the person104. The processing circuitry108utilizes the detected distance DTOFin determining whether and when to activate the digital camera106to capture an image of the face or iris of the person104, as will be described in more detail below. As with the TOF sensor102, the person104must be positioned within a field of view FOVCAMof the camera106to enable the camera to properly capture an image of the face or Iris of the person being authenticated. The camera106provides the captured image of the person104to the processing circuitry108which, in turn, utilizes the captured image and the sensed distance D TOF in authenticating the user, as will also be described in more detail below.

If authenticated, the processing circuitry108will grant the person104access to the electronic device100. Conversely, if the processing circuitry100does not authenticate the person104, meaning that person is not an authorized user of the electronic device100, the processing circuitry denies the person access to the electronic device. The processing circuitry108would also in this situation typically provide some sort of feedback to the person104on the visual display114regarding this determination, such as displaying a message to the person like “Access denied. User not authenticated.” Other embodiments of the present disclosure utilizing the detected distance D TOF from the TOF sensor102will be described in more detail below.

Before describing embodiments of the present disclosure in more detail, the TOF sensor102will first be discussed in more detail with reference toFIG. 2, which is a more detailed functional block diagram of the TOF sensor102ofFIG. 1according to one embodiment of the present disclosure. In the embodiment ofFIG. 2, the TOF sensor102104includes a light source200, which is, for example, a laser diode such as a vertical-cavity surface-emitting laser (VCSEL) for generating the transmitted optical pulse signal designated as202in the figure. The transmitted optical pulse signal202is transmitted into the field of view FOVTOFthrough a projection lens204. The reflected or returned optical pulse signal is designated as206in the figure and corresponds to a portion of the transmitted optical pulse signal202that is reflected off the person104back to the TOF sensor102. The returned optical pulse signal206is received through a reflection lens208in the TOF sensor102. The lens208directs the reflected optical pulse signal206to range estimation circuitry210for estimating the imaging distance DTOFbetween TOF sensor102and the projection surface104, as previously discussed generally with reference toFIG. 1. The range estimation circuitry210includes a target single-photon avalanche diode (SPAD) array212, which receives the returned optical pulse signal206via the lens208. The target SPAD array212typically includes large number of SPAD cells (not shown inFIG. 2), each cell including a SPAD for sensing a photon of the reflected optical pulse signal206. In some embodiments of the TOF sensor102, the lens208directs reflected optical pulse signals206from certain spatial zones within the field of view FOVTOFto certain groups of SPAD cells or zones of SPAD cells in the target SPAD array212, as will be described in more detail below with respect to alternative embodiments of the present disclosure.

Each SPAD cell in the SPAD array212will provide an output pulse or SPAD event when a photon in the form of the returned optical pulse signal206is detected by that cell in the target SPAD array212. A delay detection circuit214determines a delay time between the transmitted optical pulse signal202and a SPAD event from the SPAD array212, which corresponds to the return of the reflected optical pulse signal206to the SPAD array. In this way, by detecting these SPAD events an arrival time of the return or reflected optical pulse signal206can be estimated. The delay detection circuit214determines the time of flight based upon the difference between the transmission time of the transmitted optical pulse signal202and the arrival time of the returned optical pulse signal206as sensed by the SPAD array212. From the determined time of flight, the delay detection circuit214generates a detected distance signal DDTOFindicating the detected distance DTOFbetween the person104and the TOF sensor102.

A reference SPAD array216senses the transmission of the transmitted optical pulse signal202generated by the light source200. The reference SPAD array216receives an internal reflection218from the lens204of a portion of the transmitted optical pulse signal202upon transmission of the transmitted optical pulse signal from the light source200. The reference SPAD array216effectively receives the internal reflection218of the transmitted optical pulse signal202at the same time the transmitted optical pulse signal is transmitted. In response to this received internal reflection218, the reference SPAD array216generates a corresponding SPAD event indicating the transmission of the transmitted optical pulse signal202. The delay detection circuit214includes suitable circuitry, such as time-to-digital converters or time-to-analog converters, to determine a time or time-of-flight TOF between the transmission of the transmitted optical pulse signal202and receipt of the reflected optical pulse signal208. The delay detection circuit214then utilizes this determined time-of-flight TOF to determine the distance DTOFbetween the person104and the TOF sensor201, which is assumed to be the same as the distance between the person and the electronic device100. The range estimation circuit210further includes a laser modulation circuit220that drives the light source200. The delay detection circuit214generates a laser control signal LC that is applied to the laser modulation circuit220to control activation of the laser200and thereby control generation of the transmitted optical pulse signal202.

FIG. 3Ais a flowchart showing initialization of a facial/iris recognition process for the electronic device100ofFIG. 1utilizing a distance DTOFsensed by the TOF sensor102according to one embodiment of the present disclosure. This process will now be described with reference toFIGS. 1 and 3A. In the embodiment ofFIG. 3A, the process starts in step300A and then proceeds to step302A in which a user of the electronic device100captures through the digital camera106an image or images of the user's face or iris depending on whether the recognition process is utilizing facial or iris recognition. The process then proceeds to step304A and the TOF sensor102senses a capture distance DTOF_CAPat which the image or images of step302A were captured. The capture distance DTOF_CAPthus indicates the distance between the person104and the electronic device100when the image or images of step302A were captured. From step304A, the process proceeds to step306A and the processing circuitry108determines a recognition window RW based on a resolution of the digital camera106and optics components of this digital camera.

The recognition window RW indicates a range of permissible distances around the capture distance DTOF_CAPat which the person104may be positioned from the electronic device100when the camera106is capturing images to authenticate the person, as will be described in more detail with reference toFIG. 3B. The recognition window RW is defined by a minimum distance DMINand a maximum distance DMAXat which the person104may be positioned from the electronic device100when the camera106is capturing images for authentication purposes. These minimum and maximum distances DMINand DMAXthus define a range or window around the capture distance DTOF_CAPat which the initial images of step302A were captured, namely (DMIN<DTOF_CAP<DMAX). Ideally future images captured by the digital camera106for authenticating the person104would be captured at the same distance as the initial images captured in step302A, namely at the capture distance DTOF_CAP. In reality, however, these future images captured by the camera106for authentication purposes will be captured at a distance at least slightly different from the capture distance DTOF_CAP. The recognition window RW defined by the minimum and maximum distances DMINand DMAXthus defines a permissible range around the capture distance DTOF_CAPat which these future images may be captured and utilized by the processing circuitry108to reliably authenticate the person104.

After determination of the recognition window RW in step306A, the process proceeds to step308A and the processing circuitry108processes the initial image or images captured in step302A to generate recognition parameters that will be utilized by the processing circuitry in authenticating the person104using subsequently captured images from the camera106, as will be described in more detail below with reference toFIG. 3B. These recognition parameters include the capture distance DTOF_CAPalong with the minimum and maximum distances DMINand DMAXdefining the recognition window RW. In step308A these recognition parameters are saved by the processing circuitry108in the memory116ofFIG. 1, and then from step308A the initialization process proceeds to step310A and terminates.

FIG. 3Bis a flowchart showing a facial/iris recognition process for the electronic device100ofFIG. 1after the initialization process ofFIG. 3Ahas been performed. The recognition process ofFIG. 3Bstarts at step300B and proceeds to step302B and the TOF sensor102again senses the distance DTOFbetween the person104and the electronic device100. The processing circuitry108may periodically activate the TOF sensor102to periodically sense the distance DTOFor may wait until a person104attempts to access the electronic device100before activating the TOF sensor.

From step302B, the process proceeds to step304B and the processing circuitry108determines whether the DDTOFsignal from the TOF sensor102indicating the detected distance DTOFto the person104is within the recognition range RW, namely (DMIN<DTOF_CAP<DMAX). If the determination in step304B is negative, the process goes to step306B and the processing circuitry108may either determine no person104is present proximate the electronic device100or may provide visual feedback to the person through the visual display114to have that person adjust his or her distance from the electronic device. When the detected distance DTOFto the person104is very large or otherwise indicates no person is present proximate the electronic device100, the processing circuitry108in step306B may then place the electronic device100in a low-power mode of operation. The processing circuitry108delays a certain period of time before returning to step302B and again activating the TOF sensor102to sense whether a person104is present proximate the electronic device. Alternatively, if the detected distance DTOFindicates a person104is present even though step304B has determined the person is not within the permissible recognition range RW, the processing circuitry108may provide visual feedback to the person based upon the detected distance. For example, where the detected distance DTOFis less than the minimum distance DMINthe processing circuitry provides a message on the visual display114instructing the person104to move back away from the electronic device100in an attempt to increase the detected distance so that the person is positioned within the permissible recognition range RW. Conversely, where the detected distance DTOFis greater than the minimum distance DMAXthe processing circuitry108provides a message on the visual display114instructing the person104to move towards from the electronic device100. From the step306B, the process proceeds back to step302B.

When the determination in step304B is positive, this indicates the person104is positioned within the permissible recognition range RW from the electronic device100. In this situation, the process proceeds from step304B to step308B and the processing circuitry108adjusts operating characteristics of the digital camera106based upon the detected distance DTOFof the person104in anticipation of capturing images of the person as part of the authentication process. The processing circuitry108may, for example, adjust the zoom of the digital camera106based upon the detected distance DTOFrelative to the capture distance DTOF_CAPat which initialization images of an authorized user were previously captured as discussed above with reference toFIG. 3A. This is done so that the image or images about to be captured by the digital camera106have characteristics that allow them to be reliably compared to the initially captured images from the initialization process ofFIG. 3A.

From step308B the process proceeds to step3106in which the processing circuitry108may once again provide visual feedback to the person104through the visual display114if conditions in the environment containing the electronic device100and the person104necessitate further adjustment of the position of the person. For example, high or low levels of ambient light in a room containing the electronic device100and person104may require further adjustment of the position of the person for the digital camera106to capture sufficient images of the person. Thus, the step3106may include the processing circuitry108again activating the TOF sensor102to sense the detected distance DTOFof the person104and provide feedback to the person through the visual display114to properly position the person for the capture of images by the digital camera106.

Once the person104is properly positioned in step3106, the process proceeds to step312B and the processing circuitry108controls the digital camera106to capture an image or images of the person. The processing circuitry108then utilizes these captured images to make a determination as to whether the person104has been authenticated, meaning the person is the authorized user corresponding to the person who's images were captured during the initialization process ofFIG. 3A. From step312B, the process proceeds to step314B and the processing circuitry108proceeds based on whether the person104has been authenticated or not. When the user or person104has been authenticated, the determination in step314B is positive and the process proceeds to step316B in which the processing circuitry grants the user access to the electronic device100since the person has been authenticated. The process then proceeds to step318B and terminates. When the determination in step314B is negative, which indicates the person104has not been authenticated and is thus not an authorized user of the electronic device100, the process proceeds to step320B and the processing circuitry108denies the person access to the electronic device. The processing circuitry108also in step320B would typically provide the person104with suitable visual feedback on the visual display114letting the person know that they are not an authorized user of the electronic device100. A suitable message such as “access denied” could be provided on the visual display114in such a situation. From step320B the process again proceeds to step318B and terminates.

FIG. 4is a diagram illustrating a nominal ocular hazard distance associated with the light source200in the TOF sensor102ofFIG. 2. As previously mentioned, the light source200is typically a laser such as a laser diode and thus in the situation where facial or iris recognition is being implemented, particularly iris recognition, the TOF sensor102may be providing transmitted optical pulse signals202that illuminate an eye of the person104being authenticated. Depending upon the power of the laser200this could, of course, be a dangerous situation, with the transmitted optical pulse signals202possibly damaging the eye of the person104. A distance known as the Nominal Ocular Hazard Distance (NOHD) is the minimum distance at which it is safe to view a given laser beam with the human eye. The radiance of a laser beam (power per unit area) falls below and accessible admission limit AEL at the NOHD and thus will not damage the human eye at the NOHD from the source of the laser beam and greater distances.

FIG. 4illustrates an example of the NOHD for the laser200of the TOF sensor102. In the figure, the person104is at a safe distance from the electronic device100because the distance DTOFbetween the person104and the electronic device100is greater than the NOHD. As illustrated in the figure, a hazard zone HZ corresponding to distance is less than the NOHD exists and in this hazard zone illumination of the eye of the person104by the laser200could cause damage to the eye. Thus, in one embodiment of the processes ofFIGS. 3A and 3B, the processing circuitry108also provides feedback to the person104if the detected distance DTOFfrom the TOF sensor102indicates the person is in the hazard zone HZ. This could be visual feedback on the visual display114, or audible feedback through a speaker that is one of the output devices120, or both.

FIG. 5Ais a perspective view of a laptop computer500including a TOF sensor502within a lid504of the laptop computer for detecting closure of the lid to lock out access to the laptop computer and implement a power saving mode of operation according to another embodiment of the present disclosure.FIG. 5Bis a partial cross-sectional view of the laptop computer500ofFIG. 5Aillustrating a distance DTOF_LIDbetween the lid504and a base506of the laptop computer500. The TOF sensor502senses the distance DTOF_LIDand this distance is utilized in detecting closure of the lid504of the laptop computer500, as will now be described in more detail. In the embodiment ofFIGS. 5A and 5B, the laptop computer500is one example of the electronic device100with the TOF sensor502corresponding to the TOF sensor102ofFIG. 1. Instead of sensing the distance DTOFbetween a person104and the electronic device100as does the TOF sensor102, however, the TOF sensor502senses the distance DTOF_LIDbetween the lid504and base506of the laptop computer500as previously mentioned. The TOF sensor502is shown positioned in a center top portion of the lid504opposite the base506but may be positioned in other locations on the lid or even on the base in other embodiments.

The sensed distance DTOF_LIDis utilized in detecting the closure of the lid504and blocking access to the laptop computer500upon detecting a closure, and to place the laptop computer in a low-power mode of operation, as will now be described in more detail with reference to the flowchart ofFIG. 6. The distance DTOF_LIDwill obviously become smaller as the lid504is being closed and will be at a minimum when the lid has been closed. A bi-directional arrow508inFIG. 5Arepresents the opening and closing of the lid504as the lid is rotated about an axis at the edge of the lid that is attached to the base506. InFIG. 5B, and arrow510represents the direction of the opening of the lid504while an arrow512represents the direction of the closing of the lid. Thus, as seen inFIG. 5Bthe distance DTOF_LIDbetween the lid504and the base506decreases as the lid is being closed and thus rotated in the direction512towards the base. The lid504is closed when the lid is rotated in the direction512as far as possible to be in contact with the base506, as will be appreciated by those skilled in the art, and when in this position the distance DTOF_LIDsensed by the TOF sensor502will be a minimum.

FIG. 6is a flowchart illustrating a process implemented in the laptop computer500to detect closure of the lid504and enter a power saving mode of operation according to another embodiment of the present disclosure. As previously mentioned, the laptop computer500is one example of the electronic device100ofFIG. 1and thus the laptop computer includes processing circuitry corresponding to the processing circuitry108of the electronic device, with this processing circuitry of the laptop computer operating in combination with the TOF sensor502to implement the process ofFIG. 6. The process ofFIG. 6starts in step600and proceeds to step602in which the TOF sensor502senses the distance DTOF_LIDbetween the lid504and the base506.

From step602the process proceeds to step604and determines whether the distance DTOF_LIDis less than a first threshold DTH1. This first threshold DTH1is a lower distance threshold to compensate for nonlinearities in the distance DTOF_LIDby the TOF sensor502as this sensed distance become smaller. For example, where the TOF sensor502includes a VCSEL as the light source200nonlinearities in the distance sensed by the sensor may occur once the distance is approximately 15 millimeters or smaller. These nonlinearities in the detected distance may be improved by lowering the power of the optical signal generated by the VCSEL as the distance sensed by the TOF sensor502become smaller. As a result, when the sensed distance DTOF_LIDis less than the first threshold DTH1(DTH1=15 mm, e.g.), the process proceeds to step606and determines whether the light source200of the TOF sensor502, which is assumed to be a VCSEL in the current example, is already operating in a low-power mode LPM. If the determination in step606is negative the process proceeds to step608and places the VCSEL200in the TOF sensor502into the low-power mode LPM to improve the linearity of the sensed distance DTOF_LID.

From step608the process proceeds to step610and determines whether the sensed distance DTOF_LIDis less than a second threshold DTH2that is smaller than the first threshold DTH1. The second threshold DTH2has a value corresponding to the lid504being closed. Thus, when the determination in step610is negative, indicating that the sensed distance DTOF_LIDis not less than the second threshold DTH2and thus the lid504is not closed, the process goes back to step602and the TOF sensor502once again detects the distance DTOF_LID. Conversely, when the determination in step610is positive this indicates the sensed distance DTOF_LIDis less than the second threshold DTH2and thus the lid504has been closed. In this situation, the process proceeds from step610to step612and the laptop computer500is placed into a low-power mode LPM of operation.

As previously discussed, conventional laptop computers typically utilize a Hall-effect sensor placed in the lid or base of a laptop computer to detect closure of the lid and these sensors, while inexpensive, may not reliably detect closure of the lid of a laptop computer. The TOF sensor502can more reliably detects closure of the lid504to thereby more reliably lock and prevent access to the laptop computer500until a person provides required authentication information to access the computer. In addition, because the TOF sensor502more reliably detects closure of the lid504, the laptop computer500is more reliably placed in the low-power mode LPM of operation, reducing unwanted power consumption by the laptop computer and thereby extending battery life of a battery in the laptop computer.

FIGS. 7A and 7Bare graphs illustrating operation of the TOF sensor102ofFIG. 2in detecting multiple objects within a field of view of the TOF sensor102to thereby provide privacy-related security for a user of the electronic device100in public environments according to yet another embodiment of the present disclosure. The graphs ofFIGS. 7A and 7Bare signal diagrams showing a number of counts along a vertical axis and time bins along a horizontal axis. The number of counts indicates a number of SPAD events that have been detected in each bin, as will be described in more detail below. These figures illustrate operation of a histogram based ranging technique by the TOF sensor102ofFIGS. 1 and 2according to this embodiment of the present disclosure. This histogram based ranging technique allows the TOF sensor102to sense or detect multiple objects within the FOVTOFof the TOF sensor, and detection of these multiple objects is used in providing privacy-related security in a public environment for the electronic device100, as will now be described in more detail.

In this histogram based ranging technique, more than one SPAD event is detected each cycle of operation, where a transmitted optical pulse signal202is transmitted each cycle. SPAD events are detected by the target SPAD array212and reference SPAD array216, where a SPAD event is an output pulse provided by the array indicating detection of a photon. Each SPAD array212and216typically includes a plurality of cells (not shown inFIG. 2). Each cell in the SPAD arrays212and216will provide an output pulse or SPAD event when a photon is received in the form of the returned optical pulse signal206for target SPAD array212and internal reflection218of the transmitted optical pulse signal202for the reference SPAD array216. By monitoring these SPAD events an arrival time of the optical signal216or218that generated the pulse can be estimated. Each detected SPAD event during each cycle is allocated to a particular bin, where the bin is a time period in which the SPAD event was detected. Thus, each cycle is divided into a plurality of bins and a SPAD event detected or not for each bin during each cycle. Detected SPAD events are summed for each bin over multiple cycles to thereby form a histogram in time as shown inFIG. 8for the received or detected SPAD events.

FIGS. 7A and 7Billustrate this concept over a cycle. Multiple cells in each of the SPAD arrays212and216may detect SPAD events in each bin, with the count of each bin indicating the number of such SPAD events detected in each bin over a cycle.FIG. 7Billustrates this concept for the internal reflection718of the transmitted optical pulse signal202as detected by the reference SPAD array216. The sensed counts (i.e., detected number of SPAD events) for each of the bins shows a peak700at about bin2with this peak being indicative of the transmitted optical pulse signal202being transmitted.FIG. 7Aillustrates this concept for the reflected optical pulse signal206, with there being two peaks702and704at approximately bins3and9. These two peaks702and704(i.e., detected number of SPAD events) indicate the occurrence of a relatively large number of SPAD events in the bins3and9, which indicates reflected optical pulse signals206reflecting off a first object causing the peak at bin3and reflected optical pulse signals reflecting off a second object at greater distance than the first object causing the peak at bin9. A valley706formed by a lower number of counts between the two peaks702and704indicates no additional detected objects between the first and second objects. Thus, the TOF sensor102is detecting two objects, such as two people, within the FOVTOFof the sensor in the example ofFIGS. 7A and 7B. The two peaks702and704inFIG. 7Aare shifted to the right relative to the peak700ofFIG. 7Bdue to the time-of-flight of the transmitted optical pulse signal202in propagating from the TOF sensor102to the two objects within the FOVTOFbut at different distances from the TOF sensor.

FIG. 8illustrates a histogram generated by TOF sensor102over multiple cycles. The height of the rectangles for each of the bins along the horizontal axis represents the count indicating the number of SPAD events that have been detected for that particular bin over multiple cycles of the TOF sensor102. As seen in the histogram ofFIG. 8, two peaks800and802are again present, corresponding to the two peaks702and704in the single cycle illustrated inFIG. 7A. From the histogram ofFIG. 8, either the TOF sensor102itself or the processing circuitry108(FIG. 1) then determines a distance DTOFto each of the first and second objects in the FOVTOFof the TOF sensor. The processing circuitry108then monitors the multiple detected objects over time to provide the privacy-related security for a user of the electronic device100in public environments, as will be described in more detail below with reference toFIG. 9.

The TOF sensor102may also detect SPAD events in the target SPAD array212generated by background or ambient light in the environment containing the electronic device100. This ambient light is not indicative of the distance between the electronic device100and the object within the of the TOF sensor102, and the TOF sensor utilizes this detected ambient light (e.g., a histogram generated when no transmitted optical pulse signals202are transmitted) to adjust detected SPAD events over cycles of operation. Thus, the histogram output by the TOF sensor102ofFIG. 8would be adjusted based on this background or ambient light histogram to provide more reliable distancing information. A TOF sensor that generates histograms as discussed above for the TOF sensor102is discussed in more detail in U.S. patent application Ser. No. 14/734,138, which is incorporated herein by reference in its entirety.

FIG. 9is a flowchart illustrating operation of the electronic device100ofFIG. 1in a providing a user of the electronic device with privacy-related security in a public environment based upon detection of multiple objects by the TOF sensor102as discussed with reference toFIGS. 7A, 7B and 8according to another embodiment of the present disclosure. InFIG. 9, the process starts in step900and proceeds to step902in which the TOF sensor102senses the distance an object or multiple objects within the field of view FOVTOFof the sensor and proceeds to step904and generates a histogram as discussed above with reference toFIG. 8. From step904the process proceeds to step906and the processing circuitry108(FIG. 1) determines whether the histogram generated in step904indicates several objects are present within the field of view FOVTOFof the TOF sensor. The process ofFIG. 9is only required when multiple objects, and more specifically multiple persons, are present proximate the electronic device100within the field of view FOVTOFof the TOF sensor102. As a result, if the determination is step906is negative, meaning the processing circuitry108has determined multiple objects are not present, then the process returns to step902and again executes steps902-906to continue monitoring for the detection of multiple objects.

When the determination in step906is positive, the processing circuitry108has processed the histogram from step904and determined that multiple objects are present within the field of view FOVTOFof the TOF sensor102. In this situation, the process proceeds from step906to step908and starts a timer to start timing a period over which multiple histograms generated by the TOF sensor904are provided for processing to the processing circuitry108. The processing circuitry108utilizes these histograms to determine whether one of the additional objects detected in step906is approaching the electronic device100and thereby warranting the processing circuitry perform additional processing to provide security for the authorized user of the electronic device100. The timer is started in step908and in step910the process determines whether the time period being timed by the timer has lapsed or expired. During this time period being timed by the timer the processing circuitry108receives additional histograms generated by the TOF sensor102. Once the period of the timer has expired, the determination in step910is positive and the process proceeds to step912.

In step912, the processing circuitry108processes the histograms provided by the TOF sensor102over the time period timed by the timer in steps908and910to monitor the detected objects over the time period to determine whether any of the detected objects are approaching the electronic device100. For example, a person walking up from behind the authorized user while the user is viewing the visual display114of the electronic device100would typically be unnoticed by the authorized user and thus could present a security threat. In such a situation, the processing circuitry108would detect that this person is approaching the electronic device100and the process would then proceed to step914to take additional appropriate action, as will now be described in more detail.

When the determination in step912is positive, the process proceeds to step914and the processing circuitry108turns ON or activates the digital camera106to start capturing images for facial recognition of the multiple detected objects. One of the objects detected by the TOF sensor102is presumably an authorized user of the electronic device100since the process ofFIG. 9is executed after an authorized user has gained access to and is utilizing the electronic device. Thus, the object corresponding to the authorized user may be ignored. The distance of the authorized user from the electronic device100would typically not vary much over time and thus the processing circuitry108would not select such an object for facial recognition in step914. Alternatively, as part of step914processing circuitry108could again process digital images from the camera106to verify that a particular detected object is the authorized user. In still another embodiment, the processing circuitry108could simply perform facial recognition on all of the multiple detected objects in step914to determine whether each of these objects is a person.

From step914the process proceeds to step916and the processing circuitry108determines whether any of the multiple objects detected by the TOF sensor102are people based on the facial recognition from step914. If a detected object is not a person, then there is presumably not a concern about such an object even if that object is approaching the electronic device100. For example, if the authorized user is using the electronic device100at an outdoor café and a vehicle is pulling into a parking spot behind the authorized user then such an object may be detected by the TOF sensor102but would not be of concern for security purposes since the object is not a person. This determination is made for each of the detected objects of interest in step916. If none of the detected objects of interest, namely the detected objects approaching the electronic device100as determined in step912, is a person then there is presumably no security risk for the authorized user and the process proceeds back to step902to keep periodically monitoring the environment in which the authorized user is utilizing the electronic device.

If the determination in step916is positive, then this indicates that at least one of the detected objects of interest is a person and this person or persons may therefore present a security risk to the authorized user. A person or persons walking up from behind the authorized user and viewing what is being displayed on the visual display114of the electronic device100fall into this category. As a result, when the determination in step916is positive the process proceeds to step918and the authorized user is in some way warned about the presence of a potential security risk. The processing circuitry108could, for example, provide text or an icon on the visual display114to warn the authorized user about such a situation. In this way, the authorized user would be made aware of the situation and may then determine whether any action need be taken. This would typically be preferable to the electronic device100automatically taking action in such situations since the specific circumstances may not warrant such action. The detected person or persons approaching the electronic device100from behind the authorized user could, of course, be known to the authorized user such as in a situation where the authorized user is showing this person or persons what is being displayed on the visual display114. In this situation, it would not be desirable to automatically turn OFF the visual display114or electronic device100since there is no security risk. In other embodiments, however, as part of warning the authorized user in step918the processing circuitry108could automatically turn off the visual display114, could power down the electronic device100, or could log out the authorized user.

While in the present description embodiments are described including the TOF sensor102including SPAD arrays, the principles of the circuits and methods described herein for calculating a distance to objects could be applied to arrays formed of other types of photon detection devices.