Patent ID: 12254687

DETAILED DESCRIPTION

Hardware components of a facial recognition system can include an infrared (IR) camera, a flood illuminator, a proximity sensor, an ambient light sensor, a front camera, a dot projector, and a processor. For example, the optical components can be included in a so-called “TrueDepth strip” or any other suitable location of a mobile device.FIG.1is an image representing components of a facial recognition system including a camera10, a dot projector12, and an IR camera14. Although shown in a left-to-right arrangement, the components of the facial recognition system and those ofFIG.1can be arranged in any suitable orientation with respect to each other and other components of the mobile device. The components of the facial recognition system are typically adjacent to the electronic display of the mobile device. It has been found that the dot projector12and the IR camera14must be functional for the facial recognition system to work in certain mobile devices. If one of these two components are defective the facial recognition system will not work properly.

Failures of a facial recognition system of a mobile device can be caused by a defect of the dot projector or IR camera. It is assumed that software associated with the facial recognition system in any particular mobile device is working properly and not otherwise corrupted or nonoperational. Thus, testing of the hardware components becomes a surrogate for testing the entire facial recognition system.

Verification of the operation of the dot projector12and the IR camera14of a mobile device can be automated by putting the mobile device into a certain mode where the dot projector12and the IR camera14should be turned on and then verifying that these components are, in fact, turned on. If these components do not turn on when they should be turned on, the facial recognition system will not operate.

Although not required, preferably a mobile device under test can be mounted in an automated test apparatus. Optionally, testing can be performed manually or semi-automated.FIG.2is a block diagram of a test system200used to verify operation of the dot projector12and the IR camera14of a mobile device20under test, according to one embodiment.FIG.2shows that the mobile device20under test can be driven by a mobile device driver30to put the mobile device20into a test mode for testing. A camera40, for example a digital camera with sensitivity into the infrared range, driven by a computer50can be used to capture and store digital images of the mobile device20during testing. Optionally, the computer50can be used to control the mobile device driver30. Optionally, the mobile device driver30, the camera40, and the computer50can be components of an automated test system used to functionally test the mobile device20. Optionally, the test system can include an input connection configured to operatively connect to the mobile device20.

The computer50can include a processor and a memory for storage. The computer can be a stand-alone central processing unit or a decentralized network of computers.

Optionally, the memory can be separate from the computer50. The memory can include storage of an application to control the camera40, an application for an operator to interface with the computer50or test system200, and images captured by the camera40or previously stored, such as a predetermined test image.

In an embodiment for a certain model or family of mobile devices, a mobile device can be put into the test mode by engaging the facial recognition system/function of the mobile device, e.g., unlocking the mobile device, if locked, opening a camera application in the mobile device, initiating the front camera (i.e., the camera used to set-up and verify facial recognition), and, for some mobile devices, switching the camera application to portrait mode. In this mode, the dot projector and the IR camera should be on. Optionally, the processor can transmit, via the input connection to the mobile device20, instructions causing the mobile device20to engage the facial recognition system.

Because the dot projector typically works by projecting thousands of beams or dots in the infrared spectrum, outside the visible spectrum, a naked eye cannot be used to verify if such a dot projector is working. However, a sensor sensitive in the infrared spectrum can be used to determine if the dot projector is emitting infrared radiation. Such an infrared sensor can include the digital camera40with a CCD sensor sensitive in the spectrum in which the dot projector emits.

FIGS.3and4are examples of digital images of mobile devices captured while testing hardware components of a facial recognition system according to an embodiment of the present disclosure. The digital images ofFIGS.3and4were taken in a dark chamber using a testing set-up like that shown inFIG.2. Because the images were taken in a dark ambient environment of mobile devices with a dark case, the outline of the mobile devices is not visible and is represented by the dotted rectangle. For simplicity of the figures, most areas of the electronic displays have been blocked by a gray rectangle.

Prior to capturing the digital images ofFIGS.3and4, the mobile devices were put into a test mode in which the dot protector was expected to be on. At this point, IR radiation should be emitted from the dot projector and the IR camera. After put into the test mode, the images ofFIGS.3and4of the mobile devices were captured by a digital camera.

FIG.3shows a mobile device with the dot projector on and emitting IR radiation within the yellow circle. The IR camera is emitting IR radiation shown within the green circle. Thus, this mobile device would be considered as passing this test.FIG.4shows a mobile device with the dot projector not emitting IR radiation while the IR camera is emitting IR radiation shown within the green circle. Thus, this mobile device would be considered as failing this test because the dot projector is not projecting.

In an embodiment, a test method is described with respect to the flowchart provided inFIG.5. This test method can be fully automated and performed using a hardware set up like that shown inFIG.2. In step S1, a mobile device under test is put into a test mode. The test mode can be any combination of user actions or commands that are meant to power on the dot projector and the IR camera of the mobile device. For example, the test mode can be a set-up mode of a facial recognition application. In one aspect, the mobile device is put into the test mode by opening the front camera and switching to portrait mode. The mobile device can be put into the test mode by a user or an automated mobile device driver. Optionally, instructions can be transmitted to the mobile device causing the mobile device to be put into the test mode by engaging the facial recognition system using software commands or an application.

In step S2, a captured image, e.g., an image or a digital image of the mobile device under test, is obtained with a camera or a digital camera or sensor with sensitivity into the IR range covering the wavelengths emitted by the dot projector and the IR camera. In an embodiment, the image or digital image of the mobile device under test includes a first portion and a second portion. In a further embodiment, the first portion receives IR illumination from the dot projector. In a still further embodiment, the second portion receives IR illumination from the IR camera.

In step S3, the digital image captured in step S2is compared with a predetermined test image, such as, for example an image or digital image previously taken of a like known-good mobile device, one in which it is known that the dot projector and the IR camera are working and turned on. In an embodiment, the predetermined test image includes a first portion and a second portion. In an embodiment, the first portion of the predetermined test image corresponds to the first portion of the captured image. In a further embodiment, the first portion of the predetermined test image receives IR illumination from the dot projector of the like known-good mobile device. In an embodiment, the second portion of the predetermined test image corresponds to the second portion of the captured image. In a still further embodiment, the second portion of the predetermined test image receives IR illumination from the IR camera of the like known-good mobile device.

In an embodiment, the comparison of the captured image with the predetermined test image includes comparing a luminance value of a pixel or group of pixels of the captured image with a luminance value of a corresponding pixel or group of pixels of the predetermined test image. The image of the known-good mobile device will show that the dot projector and the IR camera are emitting IR radiation, for example, like that shown inFIG.3. Digital data of pixels of the captured image and the known-good image can be characterized in terms of x-y position and representative luminance value. This allows for a comparison of the two images in the digital domain which can be automated.

Based on the results of step S3, the operating status of the dot projector and the IR camera is determined and the test of the facial recognition hardware components of the mobile device under test is passed or failed. If the dot projector and the IR camera are emitting IR radiation, then the mobile device passes the test. The mobile device will fail the test if either of the dot projector and the IR camera are not emitting IR radiation.

The above-described embodiments of the present invention can be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, with one or more processors in an integrated circuit component. Though, a processor may be implemented using circuitry in any suitable format.

Additionally, or alternatively, the above-described embodiments can be implemented as a non-transitory computer readable storage medium embodied thereon a program executable by a processor for performing a method of various embodiments. Such a non-transitory, machine-readable medium can have stored thereon a plurality of executable instructions to perform methods described herein.

It should be understood that the foregoing description is only illustrative of the present invention. Various alternatives and modifications can be devised by those skilled in the art without departing from the present invention. Accordingly, the present invention is intended to embrace all such alternatives, modifications, and variances that fall within the scope of the appended claims.