Patent ID: 12197550

DETAILED DESCRIPTION

The present disclosure includes apparatuses and methods related to generating a real-time image including an attempted user using an image sensor, comparing the real-time image to a reference image of an authorized user stored in a memory resource using a processing resource coupled to the image sensor, identifying an obstructed portion of the attempted user in the real-time image based at least in part on comparing the real-time image to the reference image, comparing an unobstructed portion of the attempted user in the real-time image to a portion of the authorized user in the reference image corresponding to the unobstructed portion of the attempted user in the real-time image, providing access to a computing device in response to the unobstructed portion of the attempted user in the real-time image matching the portion of the authorized user in the reference image, and rejecting access to the computing device in response to the unobstructed portion of the attempted user in the real-time image being different from the portion of the authorized user in the reference image.

Current facial recognition settings and/or software cannot identify an attempted user with a partially obstructed face. The attempted user's face can be partially obstructed due to eyewear and/or a mask, for example. Often an attempted user is unable to open a computing device because facial recognition settings and/or software cannot identify the attempted user with a partially obstructed face. This can inconvenience or endanger the attempted user by forcing them to remove their eyewear and/or mask, use their finger to enter a passcode or for fingerprint recognition, or use another computing device.

Embodiments of the present disclosure can enable an attempted user with a partially obstructed face to access a computing device by allowing an authorized user to customize security settings of the computing device. The security settings can determine circumstances in which the computing device can be accessed and/or a type of access an attempted user can have. For example, an attempted user can be rejected from using a computing device or receive limited or unlimited access to the computing device based on an unobstructed portion of the attempted user's face, a secondary authentication method, a location of the computing device, the present time, the present day of the week, and/or the present date.

As used herein, “a number of” something can refer to one or more of such things. A “plurality” of something intends two or more. The figures herein follow a numbering convention in which the first digit or digits correspond to the drawing figure number and the remaining digits identify an element or component in the drawing. Similar elements or components between different figures may be identified by the use of similar digits. For example, reference numeral102may reference element “2” inFIG.1Aand/orFIG.1B, and a similar element may be referenced as202inFIG.2. In some instances, a plurality of similar, but functionally and/or structurally distinguishable, elements or components in the same figure or in different figures may be referenced sequentially with the same element number. As will be appreciated, elements shown in the various embodiments herein can be added, exchanged, and/or eliminated so as to provide a number of additional embodiments of the present disclosure. In addition, the proportion and the relative scale of the elements provided in the figures are intended to illustrate various embodiments of the present disclosure and are not to be used in a limiting sense.

FIGS.1A and1Billustrate examples of a computing device100for generating an image of an attempted user104in accordance with a number of embodiments of the present disclosure. The computing device100can be, but is not limited to, a smartphone, a wearable device, a tablet, a laptop, a desktop computer, a smart assistant device, or any combination thereof. The computing device100can further include an image sensor102(e.g., a camera) to generate the image (e.g., a photograph and/or video) of an attempted user104.

The face of attempted user104can include an obstructed portion106and an unobstructed portion108. The obstructed portion106can be covered and/or altered while the unobstructed portion108can be exposed and/or unchanged. The face of the attempted user104can be obstructed by a mask, as illustrated inFIG.1Aor eyewear, as illustrated inFIG.1B. However, a face of an attempted user104can also be obstructed by other items or personal features, such as, scarves, hats, costumes, bandanas, bangs, facial hair, and/or makeup, for example.

In a number of embodiments, the computing device100can identify the obstructed portion106of the face of the attempted user104by comparing a real-time image captured by the image sensor102to a reference image stored in memory of the computing device100. In response to identifying the obstructed portion106, the computing device100can compare the unobstructed portion108to a portion of the reference image stored in the memory corresponding to the unobstructed portion108.

Since the unobstructed portion108inFIG.1Aincludes the eyes, eyebrows, ears, and hairline of the user104, the computing device100can focus a comparison on these features. For example, the computing device100can measure the distance between the eyes of the attempted user104by determining the number of pixels between the eyes of the attempted user104.

InFIG.1B, the unobstructed portion108includes the mouth, nose, ears, chin, and hairline of the attempted user104, accordingly the computing device100can focus a comparison on these features. For example, the computing device100can compare the color value of a pixel of the lips of the attempted user104in the real-time image to the color value of a pixel of the lips of the authorized user in the stored reference image. If the color values do not match because the attempted user104is wearing lipstick, for example, the computing device100may compare the ear size of the attempted user104in the real-time image to the ear size of the authorized user in the stored reference image.

FIG.2illustrates an example of a computing device200for generating a real-time image of an attempted user in accordance with a number of embodiments of the present disclosure. The computing device200can include an image sensor202to generate the real-time image of the attempted user. Computing device200can correspond to computing device100inFIG.1and image sensor202can correspond to image sensor102inFIG.1. In a number of embodiments, the computing device200can further include a memory resource210, a processing resource212, a microphone214, a geographic positioning system (GPS)216, and/or a user interface217.

The memory resource (e.g., memory)210can include volatile and/or non-volatile memory, for instance, DRAM and/or NAND. The memory210can be coupled to the processing resource212(e.g., processor) and can store an artificial intelligence (AI) model211. The memory210can be any type of storage medium that can be accessed by the processing resource212to perform various examples of the present disclosure. For example, the memory210can be a non-transitory computer readable medium having computer readable instructions (e.g., computer program instructions) stored thereon that are executable by the processing resource212to generate a real-time image including an attempted user using the image sensor202, compare the real-time image to a reference image of an authorized user stored in the memory210using the processing resource212coupled to the image sensor202, identify an obstructed portion of the attempted user in the real-time image based at least in part on comparing the real-time image to the reference image, compare an unobstructed portion of the attempted user in the real-time image to a portion of the authorized user in the reference image corresponding to the unobstructed portion of the attempted user in the real-time image, provide access to the computing device200in response to the unobstructed portion of the attempted user in the real-time image matching the portion of the authorized user in the reference image, and reject access to the computing device200in response to the unobstructed portion of the attempted user in the real-time image being different from the portion of the authorized user in the reference image.

The processing resource212can determine a location of the computing device200and compare the unobstructed portion of the attempted user in the real-time image to the portion of the authorized user in the reference image corresponding to the unobstructed portion of the attempted user in the real-time image based at least in part on the location of the computing device200. In some examples, the location of the computing device200can be determined using the GPS216, a Wi-Fi network, a Bluetooth connection, and/or image recognition. The computing device200can be connected to a particular Wi-Fi network or a Bluetooth connection, which can have a known location. For example, the computing device200can be connected to the authorized user's home Wi-Fi network or the computing device200can have a Bluetooth connection with the authorized user's vehicle. Both of these locations could have particular security settings. In some locations, the computing device200may require only partial facial recognition by allowing the attempted user access to the computing device200when a portion of the attempted user's face is obstructed and/or access to the computing device may be limited based on the location of the computing device. For example, an attempted user may be able to make phone calls and receive and transmit text messages but may not be able to make purchases at a location of a store without using full facial recognition.

In some examples, the computing device200may not be able to determine a location of the computing device200and/or an authorized user may not provide access for the computing device200to determine the location of the computing device200. The computing device200, in these circumstances, may determine a security setting based on whether the computing device200is in a less secure and/or public location. The computing device200can determine whether the computing device200is in a less secure and/or public location based on the background of the real-time image. For example, the computing device200may determine the background includes retail shelves and determine the computing device200is located within a store. Accordingly, the computing device200can initiate security settings for a less secure and/or public location.

In a number of embodiments, the processing resource212can determine a time, day of the week, and/or date and compare the unobstructed portion of the attempted user in the real-time image to a portion of the authorized user in the reference image corresponding to the unobstructed portion of the attempted user in the real-time image based at least in part on the time, the day of the week, and/or the date. The time, day of the week, and/or date can be determined from the operating system of the computing device and/or an application on the computing device, for instance, a calendar application.

The authorized user can set and/or create security settings that allow access to the computing device200even with an obstructed portion of the attempted user. In a number of embodiments, the authorized user may allow access to the computing device200with an obstructed portion of the real-time image and/or allow limited access to the computing device200based on the time, day of the week, and/or the date.

In some examples, another authentication method can be requested in response to the computing device200identifying the obstructed portion of the attempted user. The computing device200can receive real-time audio from the attempted user via the microphone214and compare the real-time audio to reference audio of the authorized user stored in the memory210. If the real-time audio matches the reference audio in memory, access can be provided to the computing device200.

In a number of embodiments, the computing device200can generate the user interface217. The user interface217can be a graphical user interface (GUI) that can provide and/or receive information to and/or from an attempted user and/or an authorized user of the computing device200. The user interface217can be shown on a display of the computing device200. In some examples, the user interface217can display a prompt to set a security setting based on the location of the computing device200and/or the time, day of the week, and/or date. The security setting can determine circumstances in which the computing device200can be accessed and/or a type of access an attempted user can have.

The computing device200can perform a facial recognition operation prior to receiving and/or implementing a security setting. For example, the image sensor202can capture another real-time image including another attempted user and the processing resource212can compare the other real-time image to the reference image and receive a security setting for a current location of the computing device200and/or the current time, day of the week, and/or date in response to the other attempted user in the other real-time image matching the authorized user in the reference image.

The computing device200can include hardware, software, and/or firmware that is configured to perform operations (e.g., logic operations, among other operations) associated with AI operations using the AI model211. In some examples, the AI model211can be trained on and/or external to the computing device200. Data from an authorized user or sample data can be used to train the AI model211. The AI model211can receive a present image, a present timestamp, a present location, or present audio each time the processing resource212provides access to the computing device200to train the AI model211to identify when the authorized user would like access to be granted. For example, the AI model211can output a security setting in response to performing an AI operation on the present image, the present timestamp, the present location, and/or the present audio.

In a number of embodiments, the image sensor202can generate another real-time image including the attempted user after a particular period of time has passed since providing access to the attempted user. The image sensor202can generate real-time images periodically to ensure the attempted user is an authorized user even after the attempted user has accessed the computing device200. The processing resource212can compare the other real-time image to the reference image. If the unobstructed portion of the attempted user in the other real-time image matches the portion of the authorized user in the reference image, the processing resource212can continue providing access to the computing device200. If the unobstructed portion of the attempted user in the other real-time image is different from the portion of the authorized user in the reference image, the processing resource212can reject access to the computing device200. In some examples, the processing resource212can lock the computing device200, alert a backup device of a security breach, transmit a message to alert the authorized user of a security breach, sign out of accounts on the computing device200, store a picture and/or fingerprint of the attempted user, and/or transmit the picture and/or fingerprint of the attempted user.

FIG.3illustrates an example of a flow diagram for providing access to a computing device (e.g., computing device100inFIG.1and/or computing device200inFIG.2) in accordance with a number of embodiments of the present disclosure. An image sensor (e.g., image sensor102inFIG.1and/or image sensor202inFIG.2) of the computing device can generate a real-time image of an attempted user at block318.

A memory (e.g., memory210inFIG.2) of the computing device can store a reference image of an authorized user at block320. The reference image can be taken of the authorized user during setup of the computing device and/or when setting security settings. In some examples, the authorized user can capture the reference image including their face without any obstructions. The reference image can be used by the computing device for full or partial facial recognition.

The real-time image and the reference image can be compared at block322during a facial recognition operation. In a number of embodiments, a facial recognition operation can include comparing pixel values of the real-time image to pixel values of the reference image. The computing device can determine a portion of the attempted user in the real-time image is obstructed at block324in response to a particular number of pixels in the real-time image being different from a corresponding number of pixels in the reference image. For example, if a significant percentage of pixel values are green in the real-time image and the corresponding pixels in the reference image are tan, the computing device can determine a portion of the attempted user in the real-time image is obstructed.

If the computing device determines there is no obstructed portion in the real-time image at block324, the computing device can determine whether the real-time image and the reference image match at block326. The real-time image and the reference image can match if a threshold number of pixel values are equal to pixel values of corresponding pixels in the reference image. The computing device can provide access to the attempted user at block334if the real-time image and the reference image match.

If the real-time and reference images do not match, the attempted user can enter a code at block328. The computing device can determine whether the code matches at block330, and the computing device can provide access to the attempted user at block334if the code matches, or the computing device can reject access to the attempted user at block332if the code does not match.

In response to the computing device determining a portion of the real-time image is obstructed at block324, the computing device can compare the unobstructed portion of the real-time image to a portion of the reference image corresponding to the unobstructed portion of the real-time image at block336. If the attempted user is wearing a hat that covers the attempted user's hair, hairline, eyebrows, and/or ears in the real-time image, the computing device can compare the attempted user's eyes, nose, lips, and/or chin to the authorized user's eyes, nose, lips, and/or chin. For example, the computing device can compare the attempted user's nose to the authorized user's nose by determining the distance between nostrils of the attempted user and the authorized user using the number of pixels and the size of the pixels in the real-time image and the reference image.

If the computing device determines the unobstructed portion of the real-time image matches the portion of the reference image corresponding to the unobstructed portion of the real-time image at block338, the computing device can provide access to the attempted user at block334. If the computing device determines the unobstructed portion of the real-time image does not match the portion of the reference image corresponding to the unobstructed portion of the real-time image at block338, the attempted user can enter a code at block328. If the code matches at block330, the computing device can provide access to the attempted user at block334. If the code does not match at block330, the computing device can reject access to the attempted user at block332.

FIG.4illustrates an example of a flow diagram for providing access to a computing device (e.g., computing device100inFIG.1and/or computing device200inFIG.2) in accordance with a number of embodiments of the present disclosure. An image sensor (e.g. image sensor102inFIG.1and/or image sensor202inFIG.2) of the computing device can generate a real-time image of an attempted user at block440.

A memory (e.g., memory210inFIG.2) of the computing device can store a reference image of an authorized user at block442. The reference image can be taken of the authorized user during setup of the computing device and/or when setting security settings. In some examples, the authorized user can capture the reference image including their face without any obstructions. The reference image can be used by the computing device for full and/or partial facial recognition.

The real-time image and the reference image can be compared at block444during a facial recognition operation. In a number of embodiments, a facial recognition operation can include comparing pixel values of the real-time image to pixel values of the reference image. The computing device can determine a portion of the attempted user in the real-time image is obstructed at block446in response to a particular number of pixels in the real-time image being different from a corresponding number of pixels in the reference image.

If the computing device determines there is no obstructed portion in the real-time image at block446, the computing device can determine whether the real-time and reference images match at block448. The real-time and reference images can match if a threshold number of pixel values of the real-time image are equal to pixel values of corresponding pixels in the reference image. The computing device can provide access to the attempted user at block456if the real-time and reference images match.

If the real-time and reference images do not match, the attempted user can enter a code at block450. The computing device can determine whether the code matches at block452and the computing device can provide access to the attempted user at block456, if the code matches or the computing device can reject access to the attempted user at block454, if the code does not match.

In response to the computing device determining a portion of the real-time image is obstructed at block446, the computing device can determine a location of the computing device at block458. In a number of embodiments, the location of the computing device can be determined using GPS (e.g., GPS216inFIG.2), a Wi-Fi network, a Bluetooth connection, and/or image recognition. In some examples, the computing device can determine the computing device is at the authorized user's workplace using image recognition. For example, the real-time image may include a cubicle in the background that has been in the background of previous real-time images including previous attempted users that have matched with the authorized user.

The authorized user can set and/or create security settings that allow access to the computing device even with an obstructed portion of the real-time image. In a number of embodiments, the authorized user may allow access to the computing device with an obstructed portion of the real-time image if the computing device is in an approved location as determined at block460. An approved location can include the authorized user's cubicle, for example.

If the computing device determines the computing device is in an approved location at block460, the computing device can compare the unobstructed portion of the real-time image to a portion of the reference image corresponding to the unobstructed portion of the real-time image at block462.

If the computing device determines the unobstructed portion of the real-time image matches the portion of the reference image corresponding to the unobstructed portion of the real-time image at block464, the computing device can provide access to the attempted user at block456. If the computing device determines the computing device is not at an approved location at block460or the unobstructed portion of the real-time image does not match the portion of the reference image corresponding to the unobstructed portion of the real-time image at block464, the attempted user can enter a code at block450. If the code matches at block452, the computing device can provide access to the attempted user at block456. If the code does not match at block452, the computing device can reject access to the attempted user at block454.

FIG.5illustrates an example of a flow diagram for providing access to a computing device (e.g., computing device100inFIG.1and/or computing device200inFIG.2) in accordance with a number of embodiments of the present disclosure. An image sensor (e.g. image sensor102inFIG.1and/or image sensor202inFIG.2) of the computing device can generate a real-time image of an attempted user at block566.

A memory (e.g., memory210inFIG.2) of the computing device can store a reference image of an authorized user at block568. The reference image can be taken of the authorized user during setup of the computing device and/or when setting security settings. In some examples, the authorized user can capture the reference image including their face without any obstructions. The reference image can be used by the computing device for full and/or partial facial recognition.

The real-time image and the reference image can be compared at block570during a facial recognition operation. In a number of embodiments, a facial recognition operation can include comparing pixel values of the real-time image to pixel values of the reference image. The computing device can determine a portion of the attempted user in the real-time image is obstructed at block572in response to a particular number of pixels in the real-time image being different from a corresponding number of pixels in the reference image.

If the computing device determines there is no obstructed portion in the real-time image at block572, the computing device can determine whether the real-time and reference images match at block574. The real-time and reference images can match if a threshold number of pixel values in the real-time image are equal to pixel values of corresponding pixels in the reference image. The computing device can provide access to the attempted user at block582if the real-time and reference images match.

If the real-time and reference images do not match, the attempted user can enter a code at block576. The computing device can determine whether the code matches at block578, and the computing device can provide access to the attempted user at block582if the code matches, or the computing device can reject access to the attempted user at block580if the code does not match.

In response to the computing device determining a portion of the real-time image is obstructed at block572, the computing device can determine the time, day of the week, and/or date at block584. In a number of embodiments, the time, day of the week, and/or date can be determined from the operating system of the computing device or an application on the computing device.

The authorized user can set and/or create security settings that allow access to the computing device even with an obstructed portion of the real-time image. In a number of embodiments, the authorized user may allow access to the computing device with an obstructed portion of the real-time image if the time, day of the week, and/or date is an approved time, day of the week, and/or date as determined at block586. An approved time could be the authorized user's work hours from 8:00 a.m. to 5:00 p.m., for example.

If the computing device determines the time, day of the week, and/or the date is an approved time, day of the week, and/or date at block586, the computing device can compare the unobstructed portion of the real-time image to a portion of the reference image corresponding to the unobstructed portion of the real-time image at block588.

If the computing device determines the unobstructed portion of the real-time image matches the portion of the reference image corresponding to the unobstructed portion of the real-time image at block590, the computing device can provide access to the attempted user at block582. If the computing device determines the computing device is not at an approved time, day of the week, and/or date at block586or the unobstructed portion of the real-time image does not match the portion of the reference image corresponding to the unobstructed portion of the real-time image at block590, the attempted user can enter a code at block576. If the code matches at block578, the computing device can provide access to the attempted user at block582. If the code does not match at block578, the computing device can reject access to the attempted user at block580.

FIG.6is a flow diagram of a method691for providing access to a computing device in accordance with a number of embodiments of the present disclosure. At block692, the method691can include generating a real-time image including an attempted user using an image sensor. The image sensor can be a camera included in the computing device, for example.

At block693, the method691can include comparing the real-time image to a reference image of an authorized user stored in a memory resource using a processing resource coupled to the image sensor. The reference image can be taken of the authorized user during setup of the computing device and/or when setting security settings. The comparison can include comparing pixel values of the real-time image to pixel values of the reference image.

At block694, the method691can include identifying an obstructed portion of the attempted user in the real-time image based at least in part on comparing the real-time image to the reference image. The computing device can determine a portion of the attempted user in the real-time image is obstructed in response to a particular number of pixels in the real-time image being different from a corresponding number of pixels in the reference image.

The method691can further include receiving a request for another authentication method from the computing device in response to identifying the obstructed portion of the attempted user in the real-time image. In a number of embodiments, receiving real-time audio from the attempted user via a microphone and comparing the real-time audio to reference audio from the authorized user stored in the memory resource can be included in method691as another authentication method. Method691can further include providing access to the computing device in response to the real-time audio from the attempted user matching the reference audio from the authorized user, wherein the access to the computing device is limited access or unlimited access.

At block695, the method691can include comparing an unobstructed portion of the attempted user in the real-time image to a portion of the authorized user in the reference image corresponding to the unobstructed portion of the attempted user in the real-time image. For example, if the attempted user's face is obstructed by eyewear, the unobstructed portion could include a user's lips. Accordingly, the lips of the attempted user could be compared to the lips of the authorized user.

At block696, the method691can include providing access to a computing device in response to the unobstructed portion of the attempted user in the real-time image matching the portion of the authorized user in the reference image. The method691can further include providing limited access to the computing device by restricting access to a number of applications on the computing device, restricting purchases using the computing device, restricting access to confidential information on the computing device, and/or restricting access to settings of the computing device.

At block697, the method691can include rejecting access to the computing device in response to the unobstructed portion of the attempted user in the real-time image being different from the portion of the authorized user in the reference image. Rejecting access to the computing device could prevent the attempted user from performing operations on the computing device. For example, the attempted user may be unable to open the computing device, make phone calls, send text messages, and/or connect to the internet.

Although specific embodiments have been illustrated and described herein, those of ordinary skill in the art will appreciate that an arrangement calculated to achieve the same results can be substituted for the specific embodiments shown. This disclosure is intended to cover adaptations or variations of one or more embodiments of the present disclosure. It is to be understood that the above description has been made in an illustrative fashion, and not a restrictive one. Combination of the above embodiments, and other embodiments not specifically described herein will be apparent to those of skill in the art upon reviewing the above description. The scope of the one or more embodiments of the present disclosure includes other applications in which the above structures and methods are used. Therefore, the scope of one or more embodiments of the present disclosure should be determined with reference to the appended claims, along with the full range of equivalents to which such claims are entitled.

In the foregoing Detailed Description, some features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the disclosed embodiments of the present disclosure have to use more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.