English:
[0002] Electronic devices, such as laptops, cellular telephones, tablet devices, and other types of personal electronic devices typically provide a level of login security in order to access the device. For example, many devices allow users to setup passwords that are required in order to use the device. Other types of authentication that may be used are biometrics, such as a fingerprint, or facial recognition. Once a user is authenticated, applications and other data on the device are accessible.
[0003] One drawback in this type of authentication is that once a user is authenticated, the device may be accessible to anyone until the user logs out, or the secure session expires. Thus, once a user initiates a session on a device, the device is left vulnerable to other unauthorized users.
[0004] In one embodiment, a method for managing a secure session is disclosed. The method includes detecting a login event at an electronic device using a first login method to initiate a secure session, capturing an initial image at a same time as the login event, capturing initial sensor data at the same time as the login event, monitoring for changes in the sensor data during the secure session, maintaining the secure session based on the login sensor data and the monitored changes in the sensor data, and during the secure session, permitting access to the electronic device using reidentification.
[0005] In another embodiment, the method may be embodied in computer executable program code and stored in a non-transitory storage device. In yet another embodiment, the method may be implemented by an electronic device having image capture capabilities.
[0006] FIG. 1 shows, in block diagram form, a simplified block diagram of an electronic device according to one or more embodiments.
[0007] FIG. 2 shows, in flow chart form, a method for generating and monitoring a secure session in accordance with one or more embodiments.
[0008] FIG. 3 shows, in flow chart form, a method for generating and monitoring a secure session using a confidence score in accordance with one or more embodiments.
[0009] FIG. 4 shows, in flow chart form, a method for person reidentification in accordance with one or more embodiments.
[0010] FIG. 5 shows, in block diagram form, a simplified multifunctional device according to one or more embodiments.
[0012] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of this description, some of this disclosure's drawings represent structures and devices in block diagram form in order to avoid obscuring the disclosed subject matter. In this context, it should be understood that references to numbered drawing elements without associated identifiers (e.g., 100) refer to all instances of the drawing element with identifiers (e.g., ΙΟΟσ and 100b). Further, as part of this description, some of this disclosure's drawings may be provided in the form of a flow diagram. The boxes in any particular flow diagram are presented in a particular order. However, it should be understood that the particular flow of any flow diagram is used only to exemplify one embodiment. In other embodiments, any of the various components depicted in the flow diagram may be deleted, or the components may be performed in a different order, or even concurrently. In addition, other embodiments may include additional steps not depicted as part of the flow diagram. The language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Reference in this disclosure to "one embodiment" or to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention, and multiple references to "one embodiment" or to "an embodiment" should not be understood as necessarily all referring to the same embodiment or to different embodiments.
[0013] It should be appreciated that in the development of any actual implementation (as in any development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system and business-related constraints), and that these goals will vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art of image capture having the benefit of this disclosure.
[0015] Electronic Device 100 may include a central processing unit (CPU) 130. Processor 130 may be a system-on-chip such as those found in mobile devices and include one or more dedicated graphics processing units (GPUs). Electronic Device 100 may also include a memory 140 and storage 150. Memory 140 and storage 150 may each include one or more different types of memory, which may be used for performing device functions in conjunction with CPU 130. For example, memory 140 and storage 150 may include cache, ROM, and/or RAM. Memory 140 and storage 150 may store various programming modules during execution, including security module 155, reidentification module 160, and environmental detection module 165. Electronic device 100 may also include a camera 120. Camera 120 may include an image sensor, a lens stack, and other components that may be used to capture images. In one or more embodiments, the camera is part of the user device, such as the Electronic Device 100, and may be front-facing such that the camera is able to capture an image of a user in front of a screen.
[0016] In one or more embodiments, reidentification module 160 is configured to identify one or more users by an image. An image may include, for example, video, live photos, and 3D captures. In one or more embodiments, reidentification module 160 obtains images captured, for example, by camera 120 and verifies the identity of one or more people in the image. Reidentification module 160 may, for example, compare facial features in a captured image with facial features previously determined to belong to a particular user. For example, reidentification module 160 may identify facial features in an image captured by camera 120 and compare the identified facial features with facial features previously determined to belong to an authorized user. In one or more embodiments, a face signature or a face feature vector may be securely stored for facial recognition, such as in a secure memory within system 100. In one or more embodiments, associations between facial features or other facial recognition data and authorized users may be stored, for example, in authorized user store 170. In one or more embodiments, reidentification module 160 may use an image to identify a user in any additional manner.
[0017] In one or more embodiments, authorized user store 170 may be stored in storage 150. Storage 150 may include any storage media accessible by a computer during use to provide instructions and/or data to the computer, and may include multiple instances of a physical medium as if they were a single physical medium. For example, a machine-readable storage medium may include storage media such as magnetic or optical media, e.g., disk (fixed or removable), tape, CD-ROM, DVD-ROM, CD-R, CD- RW, DVD-R, DVD-RW, or Blu-Ray. Storage media may further include volatile or non-volatile memory media such as RAM (e.g., synchronous dynamic RAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM, low- power DDR (LPDDR2, etc.) SDRAM, RAMBUS DRAM (RDRAM) (RAMBUS is a registered trademark of Rambus Inc.), static RAM (SRAM)), ROM, nonvolatile memory (e.g., Flash memory) accessible via a peripheral interface such as the USB interface, etc. Storage media may include micro-electromechanical systems (MEMS), as well as storage media accessible via a communication medium such as a network and/or a wireless link.
[0018] In one or more embodiments, authorized user store 170 may include data used to determine that a particular authentication feature belongs to a user. In one or more embodiments, authorized user store 170 may store authentication data in the form of a tree, a table, a database, or any other kind of data structure. Although authorized user store 170 is depicted as part of Electronic Device 100, in one or more embodiments, authentication data may additionally or alternatively be stored in a data store in a network device, such as a remote system or network storage, connected to the Electronic Device 100 across a network.
[0019] Environmental detection module 165 may process environmental factors or other sensor data obtained by one or more sensors within or operatively connected to Electronic Device 100. For example, in one or more embodiments, the environmental detection module 165 may consider sensor data from sensors that are part of electronic device 100, such as sensors 175. Environmental detection module 165 may also consider sensor data from sensors in devices external to electronic device 100, such as sensors 185 in external device 180. For example, environmental detection module 165 may consider sensor data from user devices connected to electronic device 100, such as smart watches, mobile phones, activity trackers, and the like. Sensors 175 or 185 may also include other cameras, such as a depth camera, an infrared camera, or another sensing device with a 2D sensor array.
[0020] In one or more embodiments, the environmental detection module 165 may monitor environmental conditions such as ambient light, location information, time of day, and other environmental factors related to the electronic device 100. In one or more embodiments, the environmental detection module 165 may capture sensor data at the time a user securely logs in to the electronic device 100, such as a login event. In one or more embodiments, the environmental detection module 165 may store the captured data for the environmental factors, for example, as associated with the authorized user in authorized user store 170.
[0021] In one or more embodiments, the security module 155 may support user authentication processes. For example, security module 155 may manage user authentication for a device, such as Electronic Device 100. In one or more embodiments, security module 155 may manage password authentication, biometric authentication, voice authentication, and the like.
[0022] In addition, security module 155 may also manage authentication through person reidentification. That is, in one or more embodiments, security module 155 may generate, maintain, or restart a secure session based on authentication in accordance with this disclosure. The security module 155 may manage an initial secure login, for example, by a user using, for example, a password or other login means. In one or more embodiments, the security module 155 may trigger camera 120 to capture an initial image of the user entering login information, and may trigger environmental detection module 165 to capture sensor data related to the environment or context of the electronic device. For purposes of clarity, the term "image" as used below may correspond to any type of data that contributes to facial recognition, such as a 2D image, feature vectors, facial models, video images, and the like. In one or more embodiments, during the secure session, the security module 155 may dynamically maintain the secure session by monitoring for changes in the sensor data. The sensor data may be continuously monitored, or may be occasionally or periodically monitoring the sensor data. In one or more embodiments, the security module 155 may maintain the secure session by allowing a user to access the device using reidentification when it is determined that reidentification could present a positive match based on the monitored sensor data. That is, if a login image is stored, for example in authorized user store 170, as associated with sensor data that is substantially similar to the monitored sensor data, then reidentification may be used. For example, a positive reidentification may be possible if the ambient lighting is similar to ambient lighting data stored in association with a previous login image in authorized user store 170. In one or more embodiments, the stored login image used for reidentification may be stored as part of a current secure session, or in association with a previous secure session.
[0023] In one or more embodiments, the security module 155 may calculate a confidence score based on the sensor data. For example, in one or more embodiments, the security module 155 may calculate the confidence score based on sensor data detected at the time of the secure login and, in one or more embodiments, sensor data or information stored in authorized user store 170 as associated with previous captured images. Thus, for example, if ambient light data indicates that the lighting is significantly different than at the time of the secure login, then the security module may calculate a lower confidence score than if the ambient light remains similar. Thus, in one or more embodiments, the confidence score indicates a likelihood of a positive identification of a user using reidentification, for example using as facial recognition, based on the conditions in which the initial image was captured and the current environmental conditions. In one or more embodiments, the confidence score may also indicate a likelihood of a positive identification of a user using facial recognition based on the conditions in which other images have been captured and stored, for example, in authorized user store 170.
[0024] In one or more embodiments during or after the secure session, the security module may capture an updated image using camera 120. The security module 155 may use reidentification to identify the user. For example, the reidentification module 160 may compare facial features of the updated image to the facial features of the login image to determine that the authenticated user is gazing at the electronic device 100. In response to determining that the authenticated user is the user in the updated image, the security module 155 may maintain or start the secure session.
[0025] Turning to FIG. 2, an example method for generating and maintaining a secure session is depicted in the form of a flow chart. It should be understood that the various components of the flow chart described below may be performed in a different order or simultaneously, and some components may even be omitted in one or more embodiments. The flow chart begins at 205, when a login event is detected. In one or more embodiments, the detected login event may be an authentication of a user logging into a device using a login method such as a password, biometric information, and the like.
[0026] The method continues at 210 where the security module opportunistically enrolls the user and generates a secure session. In one or more embodiments, the user may already be enrolled, and additional data regarding the user may be stored during the opportunistic enrollment. As an example, a user may enter login information into a laptop or other personal device in order to gain access to the device's functionality. In one or more embodiments, beginning the secure session includes, at 215, capturing an initial image. For example, the device may include a front facing camera (i.e., a camera facing away from a display device from which an initial login screen may be seen) which captures an image of the user as the user is logging into the system, or immediately after the user is logging into the system. In one or more embodiments, the security module, for example, may trigger the front facing camera to capture an image at or near the time of the secure login. In doing so, the camera may capture an image of a user who is authenticated at the secure login.
[0027] In one or more embodiments, generating a secure session may also include initiating capturing initial sensor data at 220. In one or more embodiments, the environmental detection module, for example, may capture initial sensor data at or near the time of the secure login. The initial sensor data may include, for example, location information determined using a GPS device, time information, network connectivity information, ambient light, image data, audio data, temperature data, barometer data, altimeter data, gyroscope data, and the like. The sensor data may be captured from sensors that are part of the electronic device, or are communicably coupled to the electronic device. The initial login sensor data may be stored such that sensor data monitored during the secure session may be compared against the initial sensor data. In addition, the initial sensor data may be stored to be used in comparing sensor data in a future secure session.
[0028] The method continues at 225, and the sensor data is continuously monitored. In one or more embodiments, the sensor data may be monitored occasionally or periodically through the secure session. The flowchart continues at 230 and the secure session is maintained based on a comparison of the initial sensor data to the monitored sensor data. In one or more embodiments, at 235, maintaining the secure session allows a user to bypass other login techniques and use reidentification to access the device.
[0029] FIG. 3 shows, in flow chart form, a method for generating and monitoring a secure session using a confidence score in accordance with one or more embodiments. FIG. 3 includes some of the same actions included in FIG. 2, including detecting (205) a login event, beginning (210) a secure session, including capturing (215) an initial image and capturing (220) initial sensor data, and monitoring (225) for changes in the sensor data.
[0030] FIG. 3 differs from FIG. 2 in that following for changes in the sensor data, flowchart continues at 335 and security module calculates a confidence score based on the login sensor data and the monitored sensor data. In one or more embodiments, the security module may calculate a confidence score regarding the authorization of a current user based on the monitored sensor data. For example, if the difference between the login sensor data and monitored sensor data exceeds a predetermined threshold, the security module may calculate a lower confidence score than if the monitored sensor data more closely matched the login sensor data.
[0031] The flowchart continues at 340 where a determination is made regarding whether the confidence score is an acceptable value. For example, the security module may determine that the calculated confidence score does not meet an acceptable value threshold, whether that threshold is predefined, or dynamically defined based on monitored values. The acceptable value threshold may be defined, for example, based on a percentage or other comparison to an initial calculated confidence value, or based on particular measured parameter from the sensor data. For example, in one or more embodiments, the acceptable value threshold may be defined as a function of one or more types of sensor data, or may be different based on the types or values of sensor data monitored.
[0032] If, at 340 it is determined that the confidence score is an acceptable value, then the security module continues to 235 and the security module permits access to the device using reidentification. Permitting access using reidentification may include active steps, such as those depicted in blocks 345-360. Particularly, in one or more embodiments, maintaining the secure session may include at 345, detecting that a user is attempting to access the device. In one more embodiments, the user may actively interface with the device, or the device may sense that the user has moved the device. The flowchart continues at 350, and the camera 120 obtains a current image. In one or more embodiments, the current image may be captured as the user is gazing at the camera or is in view of the camera such that the user may be identified. At 355, reidentification is triggered using the updated image. In one or more embodiments, the reidentification module 160 may compare the captured image to the original login image to authenticate the user. In one or more embodiments, the reidentification module 160 may alternatively, or additionally, compare the captured image to other stored images, which are associated with similar sensor data. The flow chart continues at 360, and the security module 155 allows the user to access the device based on reidentification and the current image.
[0033] Returning to decision block 340, if it is determined that the confidence score is no longer an acceptable value, then the flowchart continues to 365 and the security module ends the secured session. According to one or more embodiments, the secure session may be ended based on the monitored sensor data regardless of logout features associated with the initial method of the login event at block 205. For example, if the initial login method indicates a timeout feature that causes the session to end after a particular amount of time. In one or more embodiments, the security module may override the timeout feature if the confidence score remains an acceptable value based on environmental factors. In one or more embodiments, automatic logout features of the initial login method may be taken into consideration when calculating a confidence score. For example, if the initial login method dictates that the user should be logged out and the session should end based on a certain set of conditions, the confidence score may be lower when those conditions are met. In one or more embodiments of the invention, the logout features of the initial login method will override the sensor data and the secure session will end regardless of the confidence score. In other embodiments, if the confidence score determined in accordance with 335 is less than another value, the secure session may be terminated before the scheduled timeout feature. That is, in some embodiments multiple thresholds may be utilized.
[0034] In one or more embodiments, when the secure session is ended, the user may be required to use an additional authentication method other than reidentification. For example, when the user attempts to login again, the user may be required to use a password or biometrics such as a fingerprint, in order to be authenticated. However, in one or more embodiments, when the user logs in using the additional methods, the user may also be enrolled along with the sensor data captured at the time of the login, thereby generating a new secure session.
[0035] For purposes of an example, a user may log into her device in a local coffee shop using a password. At the time the user is authenticated using the password, or substantially around the time the user is authenticated using the password, an image of the user may be captured, for example, by a front facing camera on her device. As long as the user stays in the same location (i.e., the coffee shop), the user can maintain the secure session, merely by facial recognition. According to one or more embodiments, once the user leaves the location, the confidence score will drop, perhaps to an unacceptable value, where after the secure session will end. That is, in one or more embodiments, when the user remains in a same location, reidentification methods will more likely find a positive match of the user than when the user leaves the location because sensor data that affect the reidentification will remain similar.
[0036] As another example, a user may log into her device in an office. Over the next few hours, the user may access the device simply by appearing in updated images captured by a camera connected to the device. If the user leaves her office, the ambient light may change and cause the confidence score to drop, perhaps to an unacceptable value, where after the secure session will end.
[0037] The above examples are merely intended to provide the reader with a better understanding of embodiments of the inventive subject matter and are not intended to limit any one or more embodiments of the invention.
[0038] Referring now to FIG. 4, a flow chart depicts an example method of starting a secure session using person reidentification. In one or more embodiments, after a confidence score is no longer an acceptable value, a user may restart the secure session by gazing at the camera, or merely appearing in front of the camera when an updated image is captured. Over time, a user may be authenticated in a number of environments. In one or more embodiments, authorized user store 170 may include a number of user images and corresponding sensor data. Thus, for example, if a user uses their computer at their office during the day and then brings it home, reidentification may allow the security module 155 to authenticate the user using only reidentification. The reidentification may be based on previously stored environmental data and corresponding images, even though the best match for the environmental data is not from the most recent secure session. It should be understood that the example shown in FIG. 4 is depicted merely for purposes of clarity for the reader, and is not intended to limit the disclosure.
[0039] The method begins at 405 where the current sensor data is captured. In one or more embodiments, the sensor data may come from sensors on the device the user is attempting to access, or may be communicably coupled to the device. The flowchart continues at 410 and the current sensor data is compared to stored sensor data associated with previous sessions. In one or more embodiments, the current sensor data may be compared to previously captured sensor data stored in authorized user store 170 as associated with previous secure sessions for authorized users.
[0040] The flowchart continues at 415 and a determination is made regarding whether a positive identification is likely. In one or more embodiments, the determination may be based on a confidence value as described above. Further, in one or more embodiments, determining whether a positive identification is likely includes determining whether facial recognition might be successful based on the current sensor data and sensor data captured coincident with previous login images for previous secure sessions.
[0041] If a determination is made at 415 that a positive identification is not likely, then the flowchart continues at 420 and the user is re-enrolled. That is, in one or more embodiments, the user must utilize a second login method, such as a password or biometric identification, and a current image and sensor data may be captured and stored.
[0042] Returning to decision block 415, if it is determined that a positive identification is likely, then the flowchart continues at 425, then the secure session begins using reidentification. In one or more embodiments, the reidentification module 160 captures a current image and compares the current image to stored images associated with sensor data similar to current sensor data. The reidentification module may, for example, compare facial features in the current image to facial features of the stored images. In one or more embodiments, the captured image and current sensor data is stored in authorized user store and may be used for future reidentification of the user.
[0043] Referring now to FIG. 5, a simplified functional block diagram of illustrative multifunction electronic device 500 is shown according to one embodiment. Multifunction electronic device 500 may include processor 505, display 510, user interface 515, graphics hardware 520, device sensors 525 [e.g., proximity sensor/ambient light sensor, accelerometer and/or gyroscope), microphone 530, audio codec(s) 535, speaker(s) 540, communications circuitry 545, digital image capture unit 550 video codec(s) 555, memory 560, storage device 565, and communications bus 570. Multifunction electronic device 500 may be, for example, a digital camera or a personal electronic device such as a personal digital assistant (PDA), personal music player, mobile telephone, or a tablet computer. In some embodiments, multifunction electronic device 500 corresponds to Electronic Device 100.
[0044] Processor 505 may execute instructions necessary to carry out or control the operation of many functions performed by device 500 [e.g., such as the generation and/or processing of images in accordance with this disclosure). Processor 505 may, for instance, drive display 510 and receive user input from user interface 515. User interface 515 may allow a user to interact with device 500. For example, user interface 515 can take a variety of forms, such as a button, keypad, dial, a click wheel, keyboard, display screen and/or a touch screen . Processor 505 may also, for example, be a system-on-chip such as those fou nd in mobile devices and incl ude a ded icated graph ics processing u nit (G PU). Processor 505 may be based on reduced instruction-set computer ( RISC) or complex instruction-set computer (CISC) architectures or any other suitable architecture and may include one or more processing cores. Graph ics hardware 520 may be special pu rpose computational hardware for processing graphics and/or assisting processor 505 to process graphics information. I n one embod iment, graph ics hardware 520 may include a programmable graph ics processing un it (G PU).
[0045] Sensor and camera circu itry 550 may capture still and video images that may be processed, at least in part, by video codec(s) 555 and/or processor 505 and/or graph ics hardware 520, and/or a dedicated image processing u nit incorporated within circuitry 550. Images so captured may be stored in memory 560 and/or storage 565. Memory 560 may include one or more different types of med ia used by processor 505 and graphics hardware 520 to perform device fu nctions. For example, memory 560 may include memory cache, read-only memory (ROM), and/or random access memory (RAM). Storage 565 may store med ia (e.g., aud io, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data. Storage 565 may include one more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). Memory 560 and storage 565 may be used to tangibly retain computer program instructions or code organized into one or more modules and written in any desired computer programming language. When executed by, for example, processor 505 such computer program code may implement one or more of the methods described herein.
[0046] In practice, the environmental detection module may monitor low-power sensor data, such as sensor data that requires less power to monitor than the reidentification feature, such as facial recognition. Thus, utilizing reidentification only when it is determined that reidentification will likely be successful based on compared sensor data may provide power and other resource savings. In one or more embodiments, reidentification may occur rapidly, for example greater than 30 frames per second, for a better user experience. In one or more embodiments, the camera may capture multiple initial images of the same user so that facial recognition becomes more efficient over time. For example, during the secure session, because the security module calculates a high confidence score, the security module may occasionally capture one or more additional images of the user and store the images, or facial features from the images, in the authorized user store 170. As another example, each time a user uses the same login information, the security module 155 may capture those initial images and store the additional login images in the authorized user store 170 such that the security module 155 can associate particular login information with a set of images that include the user associated with that login information. Thus, the reidentification module 160 may rely on the entire set of images to authenticate a user using facial recognition, according to one or more embodiments. Any method of reidentification allowing for the identification of a same user in multiple images may be utilized. Finally, variations of the above-described embodiments may be used in combination with each other.
[0047] Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein."

中文:
[0002]电子设备(诸如膝上型电脑、蜂窝电话、平板设备,以及其他类型的个人电子设备)通常提供一定级别的登录安全性,以便访问该设备。例如,许多设备允许用户设置需要的密码,以便使用该设备。可使用的其他类型的认证是生物识别,诸如指纹或面部识别。一旦用户被认证,设备上的应用程序和其他数据就为可访问的。
[0003]该类型的认证中的一个缺点是一旦用户被认证,对于任何人该设备都可为可访问的,直到用户注销,或安全会话结束。因此,一旦用户在设备上发起会话,设备就易受其他未授权用户的攻击。
[0004]在一个实施方案中,公开了一种用于管理安全会话的方法。该方法包括使用第一登录方法在电子设备处检测登录事件以发起安全会话;在与登录事件相同的时间捕获初始图像;在与登录事件相同的时间捕获初始传感器数据;在安全会话期间监测传感器数据的改变;基于登录传感器数据和传感器数据中所监测的改变来维持安全会话;以及在安全会话期间,准许使用重新识别访问电子设备。
[0005]在另一个实施方案中,该方法可体现在计算机可执行程序代码中且存储在非暂态存储设备中。在另一个实施方案中,该方法可由具有图像捕获能力的电子设备来实施。
[0006]图1以框图形式示出了根据一个或多个实施方案的电子设备的简化框图。
[0007]图2以流程图形式示出了根据一个或多个实施方案的用于生成和监测安全会话的方法。
[0008]图3以流程图形式示出根据一个或多个实施方案用于使用置信度评分生成和监测安全会话的方法。
[0009]图4以流程图形式示出根据一个或多个实施方案的用于人员重新识别的方法。
[0010]图5以框图形式示出了根据一个或多个实施方案的简化的多功能设备。
[0012]在以下描述中,为了解释的目的,阐述了很多具体细节,以便提供对发明构思的彻底理解。作为该描述的一部分,本公开的附图中的一些附图以框图形式表示结构和设备,以便避免模糊本发明所公开的主题。在这个背景下,应当理解,对没有相关联的标识符(例如,100)的编号绘图元素的引用是指具有标识符(例如,100a和100b)的绘图元素的所有实例。另外,作为该描述的一部分,本公开的附图中的一些附图可以流程图的形式提供。任何特定流程图中的框以特定次序呈现。然而,应当理解,任何流程图的具体流动仅用于举例说明一个实施方案。在其他实施方案中,流程图中描绘的各种部件中的任一个可被删除,或者可以不同的次序或甚至同时执行部件。此外,其他实施方案可包括未被描绘为流程图的一部分的附加步骤。出于可读性和指导性目的,已经主要选择本公开中使用的语言,并且该语言可未被选择用于描写或界定本发明的主题。在本公开中对“一个实施方案”或“实施方案”的引用意指结合该实施方案所描述的特定特征、结构或特性包括在本发明的至少一个实施方案中,并且对“一个实施方案”或“实施方案”的多个引用不应被理解为必然地全部指代相同的实施方案或指代不同的实施方案。
[0013]应当了解,在任何实际具体实施的开发中(如在任何开发项目中),必须要做出很多决策以实现开发者的特定目标(例如,符合系统和业务相关的约束条件),并且这些目标将在一个具体实施和另一个具体实施之间变化。还应当了解,此类开发工作可能是复杂且费时的,但尽管如此,对于受益于本公开的图像捕获领域的普通技术人员而言,将仍然是常规任务。
[0015]电子设备100可包括中央处理单元(CPU)130。处理器130可为片上系统诸如在移动设备中发现的这些片上系统,并且包括一个或多个专用图形处理单元(GPU)。电子设备100还可包括存储器140和存储装置150。存储器140和存储装置150可各自包括一种或多种不同类型的存储器,一种或多种不同类型的存储器可用于结合CPU 130执行设备功能。例如,存储器140和存储装置150可包括高速缓存、ROM和/或RAM。存储器140和存储装置150可在执行期间存储各种编程模块,包括安全模块155、重新识别模块160,以及环境检测模块165。电子设备100还可包括相机120。相机120可包括图像传感器、透镜叠堆,以及可用于捕获图像的其他部件。在一个或多个实施方案中,相机是用户设备(诸如电子设备100)的一部分,并且可为前向的,使得相机能够在屏幕的前面捕获用户的图像。
[0016]在一个或多个实施方案中,重新识别模块160被配置为通过图像识别一个或多个用户。图像可包括例如视频、现场照片和3D捕获。在一个或多个实施方案中,重新识别模块160例如通过相机120获得捕获的图像,并且验证图像中的一个或多个人员的身份。重新识别模块160可例如将所捕获的图像中的面部特征与先前确定为属于特定用户的面部特征进行比较。例如,重新识别模块160可识别由相机120捕获的图像中的面部特征部,并且将所识别的面部特征与先前确定为属于授权用户的面部特征进行比较。在一个或多个实施方案中,面部特征图或面部特征向量可安全地存储诸如在系统100内的安全存储器中用于面部识别。在一个或多个实施方案中,面部特征或其他面部识别数据与授权用户之间的关联可例如存储在授权用户存储170中。在一个或多个实施方案中,重新识别模块160可使用图像来以任何附加方式识别用户。
[0017]在一个或多个实施方案中,授权用户存储170可存储在存储装置150中。存储装置150可包括在使用期间由计算机可访问的以向计算机提供指令和/或数据的任何存储介质,并且可包括物理介质的多个实例,好像它们是单个物理介质那样。例如,机器可读存储介质可包括存储介质诸如磁性或光学介质,例如,盘(固定或可拆卸)、带、CD-ROM、DVD-ROM、CD-R、CD-RW、DVD-R、DVD-RW或蓝光。存储介质还可包括易失性或非易失性存储器介质诸如RAM(例如,同步动态RAM(SDRAM)、双数据速率(DDR、DDR2、DDR3等)SDRAM、低功率DDR(LPDDR2等)SDRAM、RAMBUS DRAM(RDRAM)(RAMBUS是Rambus公司的注册商标)、静态RAM(SRAM))、ROM、经由外围接口诸如USB接口等可访问的非易失性存储器(例如,闪速存储器)等。存储介质可包括微机电系统(MEMS),以及经由通信介质诸如网络和/或无线链路可访问的存储介质。
[0018]在一个或多个实施方案中,授权用户存储170可包括用于确定特定认证特征属于用户的数据。在一个或多个实施方案中,授权用户存储170可以树、表、数据库,或任何其他种类的数据结构的形式存储认证数据。虽然授权用户存储170被描绘为电子设备100的一部分,但是在一个或多个实施方案中,认证数据可附加地或另选地存储在跨越网络连接到电子设备100的网络设备(诸如远程系统或网络存储装置)的数据存储中。
[0019]环境检测模块165可处理由电子设备100内的或操作地连接到电子设备100的一个或多个传感器获得的环境因素或其他传感器数据。例如,在一个或多个实施方案中,环境检测模块165可考虑来自为电子设备100的一部分的传感器(诸如传感器175)的传感器数据。环境检测模块165还可考虑来自电子设备100外部的设备中的传感器(诸如外部设备180中的传感器185)的传感器数据。例如,环境检测模块165可考虑来自连接到电子设备100的用户设备的传感器数据,用户设备诸如智能手表、移动电话、活动追踪器等。传感器175或传感器185还可包括其他相机,诸如深度相机、红外相机或具有2D传感器阵列的另一个感测设备。
[0020]在一个或多个实施方案中,环境检测模块165可监测环境条件诸如环境光线、位置信息、当日时间,以及与电子设备100相关的其他环境因素。在一个或多个实施方案中,环境检测模块165可在用户安全地登录到电子设备100中(诸如登录事件)的时候捕获传感器数据。在一个或多个实施方案中,环境检测模块165可将环境因素的所捕获的数据(例如,如与授权用户相关联的)存储在授权用户存储170中。
[0021]在一个或多个实施方案中,安全模块155可支持用户认证过程。例如,安全模块155可管理设备诸如电子设备100的用户认证。在一个或多个实施方案中,安全模块155可管理密码认证、生物特征认证、语音认证等。
[0022]此外,安全模块155还可通过人员重新识别来管理认证。也就是说,在一个或多个实施方案中,安全模块155可根据本公开基于认证来生成、维持或重新启动安全会话。安全模块155可例如由用户使用例如密码或其他登录手段来管理初始安全登录。在一个或多个实施方案中,安全模块155可触发相机120捕获用户输入登录信息的初始图像,并且可触发环境检测模块165来捕获与电子设备的环境或背景有关的传感器数据。为清楚起见,如下面所使用的术语“图像”可对应于有助于面部识别的任何类型的数据,诸如2D图像、特征向量、面部模型、视频图像等。在一个或多个实施方案中,在安全会话期间,安全模块155可通过监测传感器数据中的改变来动态维持安全会话。传感器数据可被连续监测,或者可偶尔或周期性地监测传感器数据。在一个或多个实施方案中,当确定重新识别可基于所监测的传感器数据呈现肯定的匹配时,安全模块155可通过允许用户使用重新识别访问设备,维持安全会话。也就是说,如果将登录图像(如与基本上类似于所监测的传感器数据的传感器数据相关联的)存储在例如授权用户存储170中,则可使用重新识别。例如,如果环境光照度类似于存储为与授权用户存储170中的先前登录图像相关联的环境光照度数据,则肯定的重新识别可为可能的。在一个或多个实施方案中,用于重新识别的所存储的登录图像可被存储为当前安全会话的一部分,或与先前安全会话相关联。
[0023]在一个或多个实施方案中,安全模块155可基于传感器数据计算置信度评分。例如,在一个或多个实施方案中,安全模块155可基于在安全登录时检测的传感器数据,计算置信度评分,并且在一个或多个实施方案中,安全模块155可基于存储在授权用户存储170中的如与先前捕获的图像相关联的传感器数据或信息,计算置信度评分。因此,例如,如果环境光线数据指示光照度显著不同于安全登录时的光照度,则安全模块可计算比环境光线保持类似的低的置信度评分。因此,在一个或多个实施方案中,基于其中捕获初始图像的条件和当前环境条件,置信度评分指示使用重新识别(例如,使用面部识别)用户的肯定的识别的可能性。在一个或多个实施方案中,基于其中已捕获其他图像且将其他图像存储例如在授权用户存储170中的条件,置信度评分还可指示使用面部识别用户的肯定的识别的可能性。
[0024]在一个或多个实施方案中,在安全会话期间或之后,安全模块可使用相机120捕获更新的图像。安全模块155可使用重新识别来识别用户。例如,重新识别模块160可将更新的图像的面部特征与登录图像的面部特征进行比较,以确定被认证的用户正在凝视电子设备100。响应于确定被认证的用户是更新的图像中的用户,安全模块155可维持或启动安全会话。
[0025]转到图2,以流程图的形式描绘了用于生成和维持安全会话的实施例方法。应当理解,可以不同的次序或同时执行下面描述的流程图的各种部件,并且可甚至在一个或多个实施方案中省略一些部件。当检测到登录事件时,流程图开始于205。在一个或多个实施方案中,所检测的登录事件可为使用登录方法诸如密码、生物特征信息等登录到设备中的用户的认证。
[0026]该方法在210处继续,在210中,安全模块适时地对用户进行登记,并且生成安全会话。在一个或多个实施方案中,用户可以已经登记,并且可在适时登记期间存储关于用户的附加数据。例如,用户可将登录信息输入到膝上型电脑或其他个人设备中,以便取得对设备的功能的访问。在一个或多个实施方案中,开始安全会话包括在215捕获初始图像。例如,设备可包括前向相机(即,远离从其可看到初始登录屏幕的显示设备的相机),当用户登录到系统中时,该前向相机捕获用户的图像,或者在用户登录到系统中之后,该前向相机立即捕获用户的图像。在一个或多个实施方案中,安全模块例如可触发前向相机以在安全登录时或接近安全登录时捕获图像。这样做,相机可捕获在安全登录时被认证的用户的图像。
[0027]在一个或多个实施方案中,生成安全会话还可包括在220处发起捕获初始传感器数据。在一个或多个实施方案中,环境检测模块例如可在安全登录时或接近安全登录时捕获初始传感器数据。初始传感器数据可包括例如使用GPS设备确定的位置信息、时间信息、网络连接性信息、环境光线、图像数据、音频数据、温度数据、气压计数据、高度计数据、陀螺仪数据等。可从为电子设备的一部分的,或可通信地耦接到电子设备的传感器捕获传感器数据。可存储初始登录传感器数据,使得在安全会话期间监测的传感器数据可与初始传感器数据进行比较。此外,可存储初始传感器数据以用于比较在未来安全会话中的传感器数据。
[0028]该方法在225处继续,并且连续监测传感器数据。在一个或多个实施方案中,可通过安全会话偶尔或周期性地监测传感器数据。流程图在230处继续,并且基于初始传感器数据与所监测的传感器数据的比较,维持安全会话。在一个或多个实施方案中,在235处,维持安全会话允许用户绕过其他登录技术,并且使用重新识别来访问设备。
[0029]图3以流程图形式示出根据一个或多个实施方案用于使用置信度评分生成和监测安全会话的方法。图3包括在图2中包括的相同动作中的一些动作,该一些动作包括检测(205)登录事件,开始(210)安全会话,包括捕获(215)初始图像且捕获(220)初始传感器数据,以及监测(225)传感器数据中的改变。
[0030]图3与图2的不同之处在于在传感器数据中的改变之后,流程图在335处继续,并且安全模块基于登录传感器数据和所监测的传感器数据计算置信度评分。在一个或多个实施方案中,安全模块可基于所监测的传感器数据计算关于当前用户的授权的置信度评分。例如,如果登录传感器数据和所监测的传感器数据之间的差值超过预先确定的阈值,则安全模块可计算与如果所监测的传感器数据更紧密地匹配登录传感器数据相比更低的置信度评分。
[0031]流程图在340处继续,在340处,关于置信度评分是否为可接受的值做出确定。例如,安全模块可确定所计算的置信度评分不满足可接受的值阈值,无论该阈值是预定义的,还是基于所监测的值动态地定义的。可例如基于与初始计算的置信度值的百分比或其他比较,或基于来自传感器数据的特定测量的参数定义可接受的值阈值。例如,在一个或多个实施方案中,可接受的值阈值可被定义为一种或多种类型的传感器数据的函数,或者可基于所监测的传感器数据的类型或值而不同。
[0032]如果,在340处确定置信度评分是可接受的值,则安全模块继续到235,并且安全模块准许使用重新识别访问该设备。准许使用重新识别访问可包括活动步骤,诸如框345-360中所描绘的那些步骤。具体地,在一个或多个实施方案中,维持安全会话可包括在345处,检测用户正试图访问该设备。在一个或多个实施方案中,用户可主动地与设备交互,或者设备可感测用户已移动设备。流程图在350处继续,并且相机120获得当前图像。在一个或多个实施方案中,当用户凝视相机或在相机的视野中时可捕获当前图像,使得可识别用户。在355处,使用更新的图像触发重新识别。在一个或多个实施方案中,重新识别模块160可将所捕获的图像与原始登录图像进行比较以认证用户。在一个或多个实施方案中,重新识别模块160可另选地或附加地将所捕获的图像与其他存储的图像进行比较,该其他存储图像与类似的传感器数据相关联。流程图在360处继续,并且安全模块155允许用户基于重新识别和当前图像来访问设备。
[0033]返回到决策框340,如果确定置信度评分不再是可接受的值,则流程图继续到365,并且安全模块终结安全的会话。根据一个或多个实施方案,不管与在框205处的登录事件的初始方法相关联的注销特征,可基于所监测的传感器数据,终结安全会话。例如,如果初始登录方法指示在特定量时间之后致使会话终结的超时特征。在一个或多个实施方案中,如果置信度评分基于环境因素保持可接受的值,则安全模块可覆写超时特征。在一个或多个实施方案中,当计算置信度评分时,可考虑初始登录方法的自动注销特征。例如,如果初始登录方法指示用户应注销且会话应基于某组条件终结,则当满足这些条件时,置信度评分可较低。在本发明的一个或多个实施方案中,不管置信度评分,初始登录方法的注销特征将覆写传感器数据,并且安全会话将终结。在其他实施方案中,如果根据335所确定的置信度评分小于另一个值,则可在预定的超时特征之前终止安全会话。也就是说,在一些实施方案中可利用多个阈值。
[0034]在一个或多个实施方案中,当安全会话终结时,用户可需要使用除重新识别之外的附加的认证方法。例如,当用户试图再次登录时,用户可需要使用密码或生物识别诸如指纹,以便被认证。然而,在一个或多个实施方案中,当用户使用附加方法登录时,用户也可与在登录时捕获的传感器数据一起登记,从而生成新的安全会话。
[0035]为了实施例的目的,用户可使用密码在本地咖啡店中登录到她的设备中。在用户使用密码被认证时,或基本上围绕用户使用密码被认证的时间,可例如通过她的设备上的前向相机来捕获该用户的图像。只要用户停留在相同位置(即,咖啡店)中,用户就可仅通过面部识别来维持安全会话。根据一个或多个实施方案,一旦用户离开该位置,置信度评分就将下降,可能下降到不可接受的值,其后安全会话将终结。也就是说,在一个或多个实施方案中,与当用户离开该位置时相比,当用户保持处于相同位置时,重新识别方法将更可能发现用户的肯定的匹配,因为影响重新识别的传感器数据将保持类似。
[0036]又如,用户可在办公室中登录到她的设备中。在接下来的几小时内,用户可简单地通过出现在由连接到设备的相机捕获的更新的图像中来访问设备。如果用户离开她的办公室,则环境光线可改变,并且致使置信度评分下降,可能下降到不可接受的值,其后安全会话将终结。
[0037]上面的实施例仅旨在为读者提供对本发明主题的实施方案的更好理解,并且不旨在限制本发明的任何一个或多个实施方案。
[0038]现在参考图4,流程图描绘了使用人员重新识别启动安全会话的实施例方法。在一个或多个实施方案中,在置信度评分不再为可接受值之后,当捕获更新的图像时,用户可通过凝视相机,或仅出现在相机前面,重新启动安全会话。随时间推移,用户可在多个环境中被认证。在一个或多个实施方案中,授权用户存储170可包括多个用户图像和对应的传感器数据。因此,例如,如果用户在白天期间在其办公室处使用它们的计算机且然后将计算机带回家,则重新识别可允许安全模块155仅使用重新识别来认证用户。重新识别可基于先前存储的环境数据和对应的图像,即使环境数据的最佳匹配不是来自最近的安全会话。应当理解,为了读者清楚的目的,仅描绘了图4所示的实施例,并且图4所示的实施例不旨在限制本公开。
[0039]该方法开始于405处,在405处,捕获当前传感器数据。在一个或多个实施方案中,传感器数据可来自用户试图访问的设备上的或可通信地耦接到设备的传感器。流程图在410处继续,并且将当前传感器数据和与先前会话相关联的所存储的传感器数据进行比较。在一个或多个实施方案中,可将当前传感器数据与存储在授权用户存储170中的如与授权用户的先前安全会话相关联的先前捕获的传感器数据进行比较。
[0040]流程图在415处继续,并且做出关于肯定的识别是否是很可能的确定。在一个或多个实施方案中,确定可基于如上所述的置信度值。另外,在一个或多个实施方案中,确定肯定的识别是否是很可能的包括基于当前传感器数据和与先前安全会话的先前登录图像一致地捕获的传感器数据,确定面部识别是否可能是成功的。
[0041]如果在415处做出肯定的识别不太可能的确定,则流程图在420处继续,并且用户被重新登记。也就是说,在一个或多个实施方案中,用户必须利用第二登录方法诸如密码或生物特征识别,并且可捕获和存储当前图像和传感器数据。
[0042]返回到决策框415,如果确定肯定的识别是很可能的,则流程图在425处继续,然后安全会话使用重新识别开始。在一个或多个实施方案中,重新识别模块160捕获当前图像,并且将当前图像和与类似于当前传感器数据的传感器数据相关联的存储的图像进行比较。重新识别模块可例如将当前图像中的面部特征与所存储的图像的面部特征进行比较。在一个或多个实施方案中,所捕获的图像和当前传感器数据存储在授权用户存储中,并且可用于用户的未来重新识别。
[0043]现在参考图5,示出了根据一个实施方案的例示性多功能电子设备500的简化功能框图。多功能电子设备500可包括处理器505、显示器510、用户界面515、图形硬件520、设备传感器525(例如,接近传感器/环境光线传感器、加速度计和/或陀螺仪)、麦克风530、(多个)音频编解码器535、(多个)扬声器540、通信电路545、数字图像捕获单元550、(多个)视频编解码器555、存储器560、存储设备565和通信总线570。多功能电子设备500可为例如数字相机或个人电子设备诸如个人数字助理(PDA)、个人音乐播放器、移动电话或平板电脑。在一些实施方案中,多功能电子设备500对应于电子设备100。
[0044]处理器505可执行必要的指令以完成或控制由设备500执行的许多功能的操作(例如,诸如根据本公开的图像的生成和/或处理)。处理器505可例如驱动显示器510,并且可从用户界面515接收用户输入。用户界面515可允许用户与设备500交互。例如,用户界面515可采取多种形式,诸如按钮、小键盘、转盘、点击式触摸转盘、键盘、显示屏,和/或触摸屏。处理器505也可例如为片上系统,诸如在移动设备中发现的那些片上系统,并且包括专用图形处理单元(GPU)。处理器505可基于精简指令集计算机(RISC)架构或复杂指令集计算机(CISC)架构或任何其他合适的架构,并且可包括一个或多个处理内核。图形硬件520可为用于处理图形的专用计算硬件和/或用于处理图形信息的辅助处理器505。在一个实施方案中,图形硬件520可包括可编程图形处理单元(GPU)。
[0045]传感器和相机电路550可捕获可至少部分地由以下设备来处理的静态图像和视频图像:(多个)视频编解码器555和/或处理器505和/或图形硬件520,和/或并入在电路550内的专用图像处理单元。所以,所捕获的图像可被存储在存储器560和/或存储装置565中。存储器560可包括由处理器505和图形硬件520使用的一种或多种不同类型的介质以执行设备功能。例如,存储器560可包括存储器高速缓存、只读存储器(ROM),和/或随机存取存储器(RAM)。存储装置565可存储媒体(例如,音频文件、图像文件和视频文件)、计算机程序指令或软件、偏好信息、设备配置文件信息以及任何其他合适的数据。存储装置565可包括一个或多个非暂态存储介质,包括例如磁盘(固定盘、软盘和可移除盘)和磁带、光学介质(诸如CD-ROM和数字视频光盘(DVD)),以及半导体存储器设备(诸如电可编程只读存储器(EPROM)和电可擦除可编程只读存储器(EEPROM))。存储器560和存储装置565可用于有形地保存被组织成一个或多个模块且以任何期望的计算机编程语言编写的计算机程序指令或代码。当由例如处理器505执行时,此类计算机程序代码可实施本文所述的方法中的一种或多种。
[0046]在实践中,环境检测模块可监测低功率传感器数据,诸如比重新识别特征(诸如面部识别)需要更少监测功率的传感器数据。因此,仅当基于比较的传感器数据确定重新识别将可能为成功时,利用重新识别可提供功率和其他资源节省。在一个或多个实施方案中,为了更好的用户体验,可快速发生重新识别(例如,每秒大于30帧)。在一个或多个实施方案中,相机可捕获相同用户的多个初始图像,使得面部识别随时间变得更有效。例如,在安全会话期间,由于安全模块计算高置信度评分,所以安全模块可偶尔捕获用户的一个或多个附加图像,并且将图像,或来自图像的面部特征存储在授权用户存储170中。又如,每次用户使用相同的登录信息,安全模块155可捕获那些初始图像,并且将附加登录图像存储在授权用户存储170中,使得安全模块155可使特定的登录信息与包括与该登录信息相关联的用户的一组图像相关联。因此,根据一个或多个实施方案,重新识别模块160可依赖于整组图像以使用面部识别认证用户。可利用允许在多个图像中识别相同用户的任何重新识别方法。最后,可彼此结合地使用上述实施方案的变型。
[0047]在回顾上面的描述时,许多其他实施方案对于本领域的技术人员而言将是显而易见的。因而应当参考所附权利要求以及此类权利要求被赋予的等同形式的完整范围来确定本发明的范围。在所附权利要求中,术语“包括(including)”和“其中(in which)”被用作相应术语“包括(comprising)”和“其中(wherein)”的简易的英语等同形式。