BIOMETRIC SERVICE EVALUATION ARCHITECTURE FOR A VEHICLE

A biometric evaluation system for a vehicle includes a vision sensor configured to detect a biometric quality of a user. The biometric evaluation system also includes an operational system for the vehicle. At least one processor is in communication with the vision sensor. The processor is configured to execute a first classification algorithm that performs a first biometric validation based on the biometric quality to estimate a state of the user. The first classification algorithm requires a first service latency. In response to an outcome of the first biometric validation, the at least one processor is further configured to execute a second classification algorithm that performs a second biometric validation to confirm the state of the user. The second classification algorithm requires a second service latency greater than the first service latency. The processor is further configured to communicate a signal to modify operation of the vehicle.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to a biometric service evaluation architecture and, more specifically, to computationally efficient systems and methods for performing biometric service evaluation in a vehicle environment.

BACKGROUND OF THE DISCLOSURE

Detecting the status of a driver of a vehicle, such as an identity or a liveliness level, typically requires time-consuming active steps that may require significant effort or computational capacity to verify biometric requirements. Conventional computing architectures may not be properly divided into fast, low-latency operations and lengthy, high-latency operations. As a result, there is a need for a less computationally heavy and time-optimized architecture for validating a state of a user of the vehicle.

SUMMARY OF THE DISCLOSURE

According to a first aspect of the present disclosure, a biometric evaluation system for a vehicle includes a vision sensor configured to detect at least one biometric quality of a user. The biometric evaluation system also includes an operational system for the vehicle. At least one processor is coupled with a memory and is in communication with the vision sensor. The processor is configured to execute a first classification algorithm that performs a first biometric validation based on the at least one biometric quality to estimate a state of the user. The first classification algorithm requires a first service latency. In response to an outcome of the first biometric validation, the at least one processor is further configured to execute a second classification algorithm that performs a second biometric validation based on the at least one biometric quality to confirm the state of the user. The second classification algorithm requires a second service latency greater than the first service latency. The processor is further configured to communicate a signal to the operational system to modify operation of the vehicle based on at least one of the first and the second biometric validations.

Embodiments of the first aspect of the present disclosure can include any one or a combination of the following features:the first and second service latencies include first and second response times, respectively;the first response time is less than the second response time;the at least one processor includes a local processor and an edge processor in communication with the local processor, wherein the local processor is configured to execute the first classification algorithm and the edge processor is configured to execute the second classification algorithm;the outcome of the first classification algorithm is one of a pass condition and an unsuccessful condition, and wherein the at least one processor is configured to execute the second classification algorithm in response to the unsuccessful condition;the first classification algorithm is biased toward the unsuccessful condition;wherein the operational system includes an electromechanical output device selectively energized based on the signal;the operational system is a powertrain system and the communication of the signal is performed in response to an unsuccessful condition of each of the first and second biometric validations;the outcome of the first classification algorithm is one of a pass condition and an unsuccessful condition, and wherein the at least one processor is configured to execute the second classification algorithm in response to the pass condition;the outcome of the first classification algorithm and an outcome of the second classification algorithm are each one of a multi-state output corresponding to an emotional state of the user;the first classification algorithm performs facial landmark analysis based on image data captured by the vision sensor and the second classification processes the image data in a neural network;the at least one processor is further configured to recursively track an output of the first biometric validation applied to the user to determine a false negative rate of the first biometric validation for the user, and modify the first biometric validation based on the false negative rate;modifying the first biometric validation includes bypassing execution of the first classification algorithm;modifying the first biometric validation includes performing an alternative biometric validation based on the at least one biometric quality in lieu of performing the first biometric validation; andthe state is one or more of a liveliness level of the user, an identity of the user, an inebriated state of the user, and an engagement level of the user.

According to a second aspect of the present disclosure, a biometric evaluation system includes a vision sensor configured to detect at least one biometric quality of a user. An electromechanical output device is selectively energized based on an evaluation of the at least one biometric quality of the user. A first processor is coupled with a memory and is in communication with the vision sensor. The first processor is configured to execute a first classification algorithm that performs a first biometric validation based on the at least one biometric quality to estimate a state of the user. The first classification algorithm requires a first response time. A second processor is in communication with the first processor and is configured to, in response to an outcome of the first biometric validation, execute a second classification algorithm that performs a second biometric validation based on the at least one biometric quality to confirm the state of the user. The second classification algorithm requires a second response time. The second response time is longer than the first response time. An operational system that is in communication with the first and second processors is configured to limit actuation of the electromechanical output device in response to not passing of both the first and second biometric validations.

Embodiments of the second aspect of the present disclosure can include any one or a combination of the following features:the at least one processor is further configured to recursively track the outcome of the first biometric validation applied to the user to determine a false negative rate of the first biometric validation for the user, and modify the first biometric validation based on the false negative rate;modifying the first biometric validation includes bypassing execution of the first classification algorithm; andthe electromechanical output device is configured to control a security operation for a building.

According to a third aspect of the present disclosure, a method to evaluate a biometric quality of a driver of a vehicle includes capturing at least one image of the driver via a vision sensor. The method further includes executing, via at least one processor that is in communication with the vision sensor, a first classification algorithm that performs a first biometric validation based on the biometric quality to estimate a state of the driver. The first classification algorithm requires a first processing power level. The method further includes based on the estimation of the state of the driver, executing, at the at least one processor, a second classification algorithm that performs a second biometric validation based on the biometric quality to confirm the state of the driver. The second classification algorithm requires a second processing power level. The second processing power level is greater than the first processing power level. The method further includes communicating an instruction to adjust an operational system of the vehicle in response to the state of the driver.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In general, the present disclosure provides for an algorithm architecture for computations and analysis of biometric services. The present architecture first evaluates a relatively simple rapid test via a relatively low-power, lightweight method of analyzing a biometric quality. If the first evaluation results in an unsuccessful result, a more complex evaluation may be performed that may require more computational power, more electrical power, and/or a greater degree of user involvement. In other examples, a service latency for the rapid test may differ from the service latency of the complex test. Thus, the algorithm architecture of the present disclosure may provide for increased efficiency in time and/or power requirements to identify a driver status, such as an inebriated state, a distracted state, a liveliness level, or the like, as will be further discussed herein.

Referring generally toFIGS.1-8, reference numeral10generally designates a biometric evaluation system. Although illustrated and described throughout the disclosure as being incorporated with a vehicle12, it is contemplated that the system10of the present disclosure may be incorporated with a stationary structure, such as a building, or any other structure that implements biometric evaluation and/or validation. For example, the system10may be employed on a mobile device (e.g., a smartphone or tablet) or with a building security system for authenticating users to enter an area or limit access to the area (e.g., a security door). Thus, and as will be described further, the system10may incorporate an electromechanical device, such as an electromagnetic lock/bolt, that may be operable as a security feature for a structure or device.

The biometric evaluation system10of the present disclosure is configured to monitor a biometric quality of a user14, analyze the biometric quality, and determine a state of the user14based on the biometric quality. The state of the user14may be an identity of the user14, an emotional state of the user14, a mental state of the user14, an inebriated state of the user14, or the like. For example, as depicted inFIG.1, a vision sensor16may be employed adjacent to an exterior18of the vehicle12in order to capture an image of the user14to determine an identity of the user14. Based on the identity of the user14, the biometric evaluation system10of the present disclosure may communicate instructions to unlock or open a door20to allow access to an interior22of the vehicle12. As will be clear in the foregoing examples, other biometric services may be evaluated by the present system10and other vehicle operations may be controlled in response to such evaluations.

Referring now toFIGS.2A and2B, the vision sensor16may additionally, or alternatively, be incorporated into the interior22of the vehicle12, such as a vehicle cabin. As depicted, the vision sensor16may be an image sensor, or imaging device, configured to capture an image of the interior22and, more particularly, an image of the user14. It is contemplated that “user” may refer to a driver of the vehicle12or a non-operator passenger in the vehicle12. With particular reference toFIG.2B, the vision sensor16may be configured to capture image data corresponding to various physical features, such as facial and body features, in order to estimate and/or determine the state of the user14. For example, the vision sensor16may capture images of eyes24, a nose26, a forehead area28, and the like of a face of the user14. Further, the vision sensor16may capture images of upper and lower body portions30,32of the user14(FIG.1) in order to determine a body pose of the user14. Based on the physical features of the user14, the biometric evaluation system10of the present disclosure may estimate or determine the state of the user14by, for example, comparing the one or more biometric qualities of the user14captured in the image data to target or expected image data.

For example, if the biometric evaluation system10is performing an identification function, the biometric evaluation system10may process images of the eyes24of the user14(e.g., the driver) to scan an iris34of the eye24with light (e.g., infrared (IR) light, near-infrared (NIR) light, visible light) to compare a reflected pattern of light to a unique pattern of light specific to the user14. In some implementations, thermal imaging may be performed by the system10using IR or NIR light and a thermal imager. Other identification functions may be performed based on features, such as eye color, facial geometries, or other feature extraction routines employed by the system10. In another example, the biometric evaluation system10is configured to perform an inebriation detection function. In this example, images of the eyes24may be processed in order to determine a gaze direction36, a pupil dilation, a color of the eyes24(e.g., a bloodshot color), or an inability to track moving objects captured by the vision sensor16(e.g., a passing vehicle, a passing pedestrian, and the like). As will be described further herein, unsuccessful completion of initial tests for these biometric services may result in a more computationally heavy function being executed by the system10in order to verify or validate the test result.

One exemplary test for detecting a non-operating state of the driver includes operation of an illumination assembly38to generate a light40in the cabin of the vehicle12. As exemplarily depicted inFIG.2A, the illumination assembly38may include a light guide42extending adjacent to a dashboard assembly44or in an interface area46of the cabin. It is contemplated that the illumination assembly38may be positioned near the vision sensor16or at a front of the vehicle12to allow the gaze direction36to be tracked. It is further contemplated that a human-machine interface (HMI48) in the vehicle12may be operable to perform the tests expected by the illumination system38.

The illumination assembly38may include a driver circuit50(FIG.3) and a plurality of light sources52behind the light guide42. The driver circuit50may control illumination of each of the light sources52to cause the light40to move along the light guide42between left and right directions. In the example test, the vision sensor16may capture movement of the eyes24of the user14to track a focus of the user14. For example, LEDs may be mounted to a portion of the vehicle12interior22(e.g., the dashboard assembly44, the HMI48, etc.) for outputting a light pattern in a generally horizontal orientation. Such light patterns may be projected by the illumination assembly38during an inebriation, or drunkenness, test to check horizontal eye gaze nystagmus conditions of the user14. In particular, the illumination assembly38may be selectively activated to cause a sequential lighting condition from left to right and right to left with audible instructions or visual instructions communicated to the user14to instruct the user14to track the light40. During such test, the vision sensor16, such as an imager, may capture the gaze direction36and/or pupil/iris qualities, and one or more processors54,56,58(FIG.3) may process such image data to identify a nystagmus condition. It is contemplated that this example is nonlimiting and that other devices configured to detect or verify the state of the driver as determined based on the vision sensor16may be employed. For example, a breathalyzer may be employed in conjunction with a second, stronger test for validating an alcohol consumption level or inebriation state of the user14upon a negative result of a first, less involved test or in tandem with a first less computationally heavy test.

Referring now more particularly toFIG.3, the biometric evaluation system10of the present disclosure may include one or more processors54,56,58in communication with one or more of the vision sensing devices16previously described. By way of example, the one or more processors54,56,58may include a first processor54and a second processor56, with the first processor54configured to perform initial estimation testing, and the second processor56executing verification algorithms to validate the estimation. In other examples, the initial estimation testing and the validation algorithm are performed on a common processor (e.g., the first processor54). In general, the first processor54may be operable as an edge computing device for relatively low-power operations, and the second processor56may be a computing device for processing data from various subsystems of the vehicle12for relatively high-power operations, such as a local processor coupled with the vehicle12and interposing communication between various edge-computing devices and vehicle operational systems68.

The low-power operations may correspond to low computational or electrical power levels, such as shorter/lower computational latency and/or response time, lower electrical power consumption, or the like. Similarly, the high-power operations may correspond to high computational or electrical power levels such as longer/higher computational latency and/or response times, higher electrical power consumption, or the like. In general, the power levels for each operation differ in timing, and the system may therefore be utilized to optimize when to employ either algorithm, and which test to perform to enhance the overall biometric operational experience. It is contemplated that the service latency, which may refer to the time from initial frame acquisition to classification output, may be less for the tests having low computational power level requirements compared to the tests service latency for the high computational power level requirements. Thus, although the processing time (e.g., the time to process one frame or many frames) may be the same in either test, the overall service latency may nonetheless be different amongst the rapid and complex tests. In some examples, the single-frame processing time may exceed the multi-frame processing time, but the overall service latency for the single-frame method may be faster due to only requiring capturing a single frame. In some examples, a single processor is employed to perform both algorithms. One or both of the first and second processors54,56may include or be in communication with an artificial intelligence engine60. The data captured via the vision sensors16may be processed in the artificial intelligence engine60to train machine learning models62for prediction of driver state estimation. In addition, or in the alternative, the data captured may be processed in a neural network64to identify the physical features of the occupant14.

A local database66(e.g., a memory) may be in communication with the one or more processors54,56,58and/or the artificial intelligence engine60and be configured to store historical data related to one or more users14of the vehicle12. For example, the local database66may store identification data (e.g., fingerprint data, iris34identification data, name data, etc.) of users14of the vehicle12. As will be further described in relation to the proceeding figures, the database may further store historical data related to success rates, rates of unsuccessful testing, efficacy rates, or efficiency data specific to particular tests for driver state estimation. For example, the local database66may store a running average, median, or other statistical quality related to how many false negatives of a given test (e.g., an emotional state algorithm) were inaccurate relative to a second, stronger, more computationally heavy, test (e.g., 3D regression modeling).

With continued reference toFIG.3, the one or more processors54,56,58may be in communication with one or more operational systems68of the vehicle12. In general, the operational systems68may be restricted or otherwise limited in response to the one or more processors54,56,58determining a negative condition of a user state, or for determining a particular state of the driver consistent with being unfit for operation of the vehicle12. Accordingly, the operational systems68may include one or more powertrain subsystems for controlling a movement of the vehicle12, a gear position of the vehicle12, or the like. In some examples, the powertrain system70includes an ignition system72, an engine control system74, a transmission system76, a steering system78, a throttle system80, a brake system82, and/or any other system configured to transmit mechanical propulsion or direction for the vehicle12or provide operation of the vehicle12. In general, the operational systems68may be configured to control an electro-mechanical device, such as a motor, a solenoid, a valve, or the like, associated with the particular operational system. In this way, the electromechanical device may be selectively energized based on the evaluation by the software. In a non-limiting example, determination of an inebriated state of the user14by the present dual algorithm may result in a pull-over instruction communicated by the one or more processors54,56,58to the operational system68. In response to receiving the pull-over instruction, the operational system68may control the steering system78, the transmission system76, and/or the engine control system74to control the vehicle12off of the road or adjacent a side of the road. In other examples, the state of the driver may be determined prior to operation of the vehicle12from a stationary position, such as an inebriation state test of the driver prior to changing gears from park to another gear.

Other hardware may be provided in the vehicle12for validating the state of the user14. For example, the previously described illumination system38may be employed for generating a light corresponding to a target location for the gaze direction36of the user14. In some examples, one or more microphones84or other audio recording devices may be employed for capturing audio data related to an emotional or mental state of the user14. A breathalyzer mechanism86may further be included to verify an inebriation state of the user14.

Still referring toFIG.3, the biometric evaluation system10may include a network88that is configured to provide communication between the various systems within the vehicle12, as well as to provide communication between the systems of the vehicle12and remote devices, such as a server90. The server90may incorporate the remote processing device58and a remote database92in communication with the remote processing device. In some examples, the remote database92stores cohort data applicable across a plurality of biometric evaluation systems in order to accumulate data across a diversity of users. Such data may be employed to promote or demote particular modes of biometric testing based on false negative/positive rates, as will be discussed further herein. Other remote devices include cloud computing devices such as the cloud computing processor58in communication with the server90. In some examples, the various operational systems68of the vehicle12may be in communication with the network88via a communication module94. The communication module94and the network88may be configured for wired or wireless communication. For example, the communication may include one or more protocols, such as Wi-Fi, Ethernet, Bluetooth®, Ultra-Wideband (UWB), Zigbee®, 5G, 4G, 3G, or any other shortwave or longwave radio communication protocol. The communication module94may provide communication between the vision devices as well as the additional hardware (e.g., breathalyzer mechanism86, illumination system38), as well as the one or more processors54,56,58, the operational systems68, and/or one or more mobile devices96associated with the user14.

As will further be described herein, it is contemplated that the cloud computing device, alternatively referred to as a remote processor58, may be employed for the more computationally heavy algorithm execution of the present disclosure, and the local or edge processors may be employed to perform initial, computationally light algorithms. For example, the rapid-precise architecture for analyzing the biometric quality of the user14may include using a rapid test to determine a driver engagement level, via the one or more local processors and, in response to determining that the user14(e.g., the driver) is not engaged with control of the vehicle12(e.g., steering, gas control, braking), the remote processor58may execute the high-power algorithm to verify the estimated engagement level via one or more of the remote processing devices. As previously described, in other examples, the rapid testing may be performed on an edge-computing device (e.g., the first processor54), and the heavy algorithm may be executed on a local processing device (e.g., the second processor56). In still other examples, both the heavy and light computational algorithms may be performed on a common processor local to the vehicle12. In this way, biometric profiled may be stored locally, the system10may not rely on cloud-connection, and service latency may be reduced.

Referring now generally toFIGS.4-6, various implementations of the present software architecture are employed to achieve a more computationally efficient outcome for determining driver states. With particular reference toFIG.4, a method400for two-tiered local biometric evaluation includes executing, at one or more processing devices (e.g., the on-vehicle processors54,56or the remote processor58), a first classification algorithm for estimating a state of the driver at step402. Execution of the first classification algorithm may include performing a first biometric validation based on at least one biometric quality captured via the vision sensor16at step404. At step406, the first classification algorithm estimates, or determines, a state of the user14in a pass/no-pass paradigm. It is contemplated that the first classification algorithm may be configured to be biased toward false negatives (e.g., biased toward determining an inebriated, intoxicated, distracted, or otherwise “negative” state). For example, for inebriation verification, the first classification algorithm may process the vision signals (e.g., vision data) of the eyes24of the user14to determine a level of pupil dilation, an eye color (e.g., bloodshot eyes24), and/or an inability to focus/track passing objects based on a limited number of frames, such as a single frame of the image data. In these examples, if any level of pupil dilation beyond a limited threshold (e.g., 50% of an average pupil dilation in daylight, any determination of red in the eyes24, or any missed shift in eye gaze based on a lighting pattern of the illumination device38) may result in an unsuccessful biometric validation. Thus, the first classification algorithm may serve as an initial check that is relatively low in time constraints relative to a second classification algorithm described below. In general, the first classification algorithm may have a first power requirement and the second classification may have a second power requirement greater than the first power requirement.

The biometric validation method400further includes executing a second classification algorithm that is performed in response to an outcome of the first biometric validation at step408. Execution of the second classification algorithm may include performing a second biometric validation based on at least one biometric quality of the user14to confirm the state of the user14at step410. At step412, the second classification algorithm is configured to validate or alter an estimation of the state of the user14. If the second classification algorithm confirms a no-pass condition of the biometric service being tested, the method proceeds to step412of communicating an instruction to control at least one of the operational systems68of the vehicle12to modify or adjust operation of the vehicle12at step414. For example, determination of an inebriated state may result in limiting enabling of the ignition subsystem to limit operation of the vehicle12. If either the first classification algorithm or the second classification algorithm results in an output of a pass condition (e.g., determination that the driver is sober or not inebriated), then the method proceeds to communication of a validation system to the operational system68to allow operation of one or more of the operational systems68of the vehicle12(e.g., the powertrain system70, the ignition system72, or the like) at step416. It is contemplated that validation checks may be performed by other systems within the vehicle12to confirm the detected state of the user14, and that, in some examples, the testing performed by the system is not intended to replace other confirmed methods for determining a state of the driver.

It is contemplated that the architecture described in relation to method400may be employed for clearly-defined successful or unsuccessful conditions. For example, biometric services such as inebriation verification, driver engagement, facial recognition, face liveliness, and the like may be validated using the method400. In addition, it is contemplated that an unsuccessful condition may include an inconclusive condition in which the first classification algorithm results in an unknown or unpredicted result. For example, if the first classification algorithm is unable to confirm a particular state (e.g., identity, sobriety), then such result may be treated as a negative outcome and the second classification algorithm will be executed. It is further contemplated that other pass/no-pass tests may be employed in the method400for determining user states based on biometric qualities of the user14, such as fatigue verification and other biometric services previously described.

Turning now more particularly toFIG.5, a method500for multistate estimation may be employed via the processors54,56,58. The software architecture employed via method500is similar to the software architecture employed for method400but may be configured to validate a non-binary classification for a state of the user14. For example, determining an agitated state of the user14, a fatigued state of the user14, a calm state of the user14, an enraged state of the user14, or the like, may be incorporated into the algorithms of the method500, and therefore may result in a multitude of states predictable based on the biometric quality. In general, the steps of executing the first classification algorithm, including performing the first biometric validation based on at least one biometric quality and estimating a state of the user14(e.g., steps402-406) may be employed in the present method500via steps502-506. Similarly, the steps of executing the second classification algorithm based on the outcome of the first biometric validation, as well as the sub-steps of the second classification algorithm (e.g., steps408-412) may further be performed by the method500, via steps508-512. Different from method400, the method500for multistate estimation may include reporting, or communicating, a confidence level for a state of the driver determined based on the vision signal between the first and second classification algorithms. For example, the first classification algorithm may be configured to estimate one of 5 to 10 states of the driver, such as those previously described, in the form of an emotional state, or a stress level. The degree of confidence may, as previously described, be non-binary (e.g., 25%, 50%, etc.). Based on the confidence level exceeding a predetermined threshold that may be biased toward a false negative determination, the method500may proceed to execution of the second classification algorithm508or may proceed to determination of the emotional/stress level state in a pass condition. Thus, the unsuccessful state previously described with relation to method400may rather correspond to an insufficient degree of certainty to meet a predetermined confidence threshold. In some examples, the confidence threshold is between 75% and 100%, though other verification ranges may be utilized.

By way of example, the first biometric validation may include body-pose analysis for target positions100and/or landmark analysis of various facial features of the driver, such as raised eyebrows, an open mouth102, various wrinkles formed along the forehead, or other landmark features that will be described further in relation toFIG.7. Based on this landmark analysis, the first classification algorithm may determine that the driver is one of excited, angry, or the like. In general, the confidence level may be determined based on target points for the various landmarks of the face, such that a comparison between the determined landmarks to the target landmarks may allow the processors54,56,58of the present disclosure to predict the state of the user14.

Execution of the second classification algorithm may include spatiotemporal 3D regression modeling performed in one or more neural networks64, or identified via processing of the facial features captured in the vision signals in the trained machine learning models62previously described in relation toFIG.3. Other verification algorithms may also or alternatively be employed, such as multimodal fusion techniques. For example, the second classification algorithm may incorporate audio data related to verbal communication of the user14, including, for example, a volume, an intensity, an enunciation, or the like, of words from the user14. In other examples, further processing of an increased number of image frames (or video data), for example, use of additional hardware (e.g., the microphones84), or the like may be employed to validate the state of the driver.

Still referring toFIG.5, following an output of the second classification algorithm or a high confidence output of the first classification algorithm may result in communicating an instruction to control the operational systems68of the vehicle12depending on the state determined by the method500. For example, at step514, the signal communicated to control the operational systems68may be particular to the specific state identified. In some examples, determination of a fatigued state may result in control of the vehicle12off of a road to a parking area, including slowing the speed of the vehicle12or the like. In other examples, determination of an agitated, or enraged state (e.g., due to road rage), may result in limiting the speed or motion. It is contemplated that communication between the processors54,56,58of the present disclosure and the various operational systems68described herein may be overridden or responsive to detection of various conditions of the user14.

Referring toFIG.6, a redundancy method for determining the state of the user14includes performing the first and second classification algorithms as previously described with respect toFIGS.4and5in parallel, such that an intermediate step of determining agreement between the light and heavy algorithms is provided. For example, the method600may include executing the first classification algorithm at step602and executing the second classification algorithm in parallel with the execution of the first classification algorithm at step604. The outputs of both steps602and604may be compared to one another at step606. For example, in a binary system, such as the system previously described with respect to method400, a pass/no-pass of the rapid algorithm may be compared to a pass/no-pass of the heavy algorithm at step606and, if both outputs agree, the method600may communicate an instruction to control various subsystems of the vehicle12based on the verified state at step608. In the event that the light and heavy algorithms do not agree, the method600employed by the various processors54,56,58of the system10may include reporting an inconclusive validation, which may result in a negative condition resulting in limiting operations of the vehicle12, at step610. The method600employed by the biometric evaluation system10of the present disclosure may apply differently depending on that specific biometric service employed by the system, and may particularly depend on a rating level for the system.

Referring now toFIG.7, an exemplary image of the user14captured via the vision sensor16demonstrates various target positions98determined by the processors54,56,58of the present disclosure in order to provide estimations and/or validations of biometric qualities. For example, binary state estimations, as well as multistate estimations, may be performed based on various body positions (e.g., body pose), facial feature prediction (e.g., landmark analysis), as well as the previously described audio feedback analysis. As depicted, a target pose100corresponding to a non-distracted state of the user14may be compared with an actual pose of the user14. As shown, the user14may have a hunched or agitated pose or a hands-off of the wheel pose that may be determined by the one or more processors54,56,58based on edge detection analysis performed on the vision signal. Based on a comparison between the target pose100and the actual pose, the biometric evaluation system10may determine that the user14is distracted, fatigued, or the like. The pose identification algorithm may be performed in the second classification algorithm following the estimation of a distracted state based on landmark analysis of the face.

With continued reference toFIG.7, the first classification algorithm may analyze various data points on the face, such as the eyes24, the forehead area28, the mouth102, etc., and compare such features to target features associated with a non-distracted state, non-fatigued state, or another target state of the user14. In the example presented above, the first classification algorithm may determine that the driver is distracted based on gaze estimation corresponding to pupil alignment with a particular area. Wrinkles identified in the vision signal, or the image data, may be incorporated into the first classification algorithms to determine a level of distraction, an emotional state, a stress level state, or the like. As illustrated, the one or more processors54,56,58may identify tears, the open mouth102, head wrinkles, stretched skin along the edges of the mouth102, or the like, and, while not determining a particular emotional state, may determine a non-approved state of the user14for operation of the vehicle12or motion of the vehicle12.

In the example illustrated, the first classification algorithm performed under the emotional state detection method500previously described may be configured to detect the various features of the face, via the landmark analysis (e.g., pupil direction, detection of wrinkles, or the like), and estimate a stressed state of the user14. The second classification algorithm of the method500may then be configured to perform 3D regression modeling to determine the existence of the tears or the open mouth102to determine that the user14is shouting or otherwise expressing verbal communication. This second biometric evaluation may be performed in one of the machine learning models62or neural networks64previously described to determine these particular spatiotemporal aspects. It is contemplated that the methods400and600may also be performed using image data presented inFIG.7to determine various binary (successful/unsuccessful) stages, such as a determination of inebriation, fatigue, distraction, or the like.

Referring now toFIG.8, a method800for determining biometric evaluation classification tests will now be described in reference to an identification task, though any of the previously described functions may employ the present method800. At step802, the method800executes the first classification algorithm in a first preferred mode. For example, the first preferred mode may be one of many testing modes for performing the rapid user identification methods previously described, such as performing edge detection on the face of the user14to identify unique facial features to determine the identity of the user14. At step804, the method determines whether or not the user14passed the identification test by matching with a user identity. If so, such a result is recorded and fed to a machine learning model at step806. Although illustrated as incorporating a plurality of nodes in a neural network, it is contemplated that the machine learning model62trained to determine preferred testing modes of the present method may employ alternative means for updating the prioritization of the testing mode for the first and second classification algorithms, as will be further described below.

If the first biometric evaluation does not pass, at step807, the method determines whether the rejection was the result of a conclusive rejection or a result of an inconclusive analysis of the vision signal. For example, if a scarf, glasses, a hood, or other obstructions are donned by the user14to secure one or more features of the user14(e.g., identification features), the first classification algorithm may output an unsuccessful condition as a result of inconclusive analysis. Alternatively, step807may result in determination of a conclusive negative by identifying features of the user14consistent with another user identity not approved by the biometric evaluation system10. The results of either may further be incorporated into the machine learning model62to train the system to select or otherwise prioritize a particular mode for performing the first classification algorithm.

Steps808-812may then be performed with respect to the second classification algorithm via a second preferred mode. For example, the second preferred mode may be performing iris34authentication analysis, performing pose detection to determine a height of the user14, or various other identification methods previously described in relation to employing a computationally heavy algorithm. Determination of passing, rejecting conclusively, or rejecting inconclusively may include communicating an output of each to a specific neural node104of the neural network64employed to train the machine learning model62.

In step814, the machine learning model62is trained to update the software architecture scheme based on the rates of false positives, false negatives, and inconclusive negatives of either or both the first classification algorithms and the second classification algorithms and the preferred modes. The machine learning model62is trained to output updated preferred modes based on this data to optimize runtime and reduce computational load and/or electrical power load. Thus, the method800may recursively track the test results and modify the testing modes based on the results. For example, if the first preferred mode of the first classification algorithm described above results in a significant (e.g., greater than average) false negative rate, the machine learning model62may select a different mode of performing the identification function, such as receiving audible name and identification information verbally from the user14. In some examples, the machine learning model62selects elimination, or bypassing, of the first classification algorithm altogether and skips directly to the second classification algorithm of performing an iris34scan, for example. It is contemplated that this decision is driven by historical data related to false negative/positives and inconclusively of previously performed testing. In further examples, other factors not related to the specific testing may be employed in the training of machine learning models62, including lighting level, date, time of day, particular body shape, a specific user identity, or the like. For example, certain tests may be determined to be unsuccessful for a particular user, whereas for other users, such tests may be accurate and efficient. It is contemplated that these other factors may be determined based on the vision signals being captured and/or audible signals, image data, or the like captured from previously described sensors.

In general, a robust architecture is provided to reduce computation and/or electrical power consumption time required to perform biometric evaluations with sufficient efficiency. For example, by providing two modes of determining a particular evaluation service, the latency and/or electrical power consumption may be reduced significantly. Further optimization may be employed by providing individualized feedback and testing the employment of the machine learning model architecture. Additionally, by offloading various tasks to other processing units and/or separate algorithms, electrical power may be conserved and relegated to one particular module of the system for most cases, and various signaling may be unnecessary to adequately perform the biometric evaluation service.

It will be understood by one having ordinary skill in the art that construction of the described concepts, and other components, is not limited to any specific material. Other exemplary embodiments of the concepts disclosed herein may be formed from a wide variety of materials, unless described otherwise herein.