Roadside impairment sensor

The present disclosure relates generally to a system and method for detecting or indicating a state of impairment of a test subject or user due to drugs or alcohol, and more particularly to a method, system and application or software program configured to creating a virtual-reality (“VR”) environment that implements drug and alcohol impairment tests, and which utilizes eye tracking technology to detect or indicate impairment.

BACKGROUND

The present disclosure relates generally to a system and method for detecting or indicating a state of impairment of a test subject or user due to use of drugs or alcohol, and more particularly to a method, system and application or software program configured to creating a virtual-reality (“VR”) environment that implements drug and alcohol impairment tests, and which utilizes eye tracking technology to detect or indicate impairment.

Impairment can be brought about by or as the result of ingesting or otherwise introducing an intoxicating substance, such as alcohol or a drug. Law enforcement officers commonly engage in the detection of a person's impairment, such as during traffic stops or other situations that may arise during the officers' line of duty.

Law enforcement officers currently have access to devices, such as a breathalyzer, which can detect or indicate impairment due to alcohol. However, there is no accepted or ubiquitous device such as the breathalyzer for marijuana and other non-alcoholic drugs. Accordingly, since law enforcement officers do not currently have access to roadside or otherwise portable impairment detectors, decisions regarding impairment typically rely on the subjective judgement of individual officers.

In addition, often a certified Drug Recognition Expert (“DRE”) is required to make a decision on a person's impairment. However, the training, certification, and re-certification, required by DREs, can be time consuming and costly.

Thus, there is a need for an easy to use, objective, and highly repeatable test, method, and system to assist law enforcement officers in gathering drug impairment indicators. As a result, officers and other officials or test administrators will be empowered to make on-site decisions without needing a certified DRE. Moreover, training and recertification costs will be reduced, allowing time and resources to be redirected to other areas of need.

BRIEF DESCRIPTION

Disclosed herein are systems and methods for creating a virtual-reality (“VR”) environment that implements tests from Standard Field Sobriety Tests (“SFSTs”) and other drug and alcohol impairment tests used by police officers in the field. The exemplary systems and method are configured, within the boundaries or constraints of VR hardware, to implement such impairment tests as closely as possible to guidelines established by police officers and other agents such as drug recognition experts (“DREs”).

More specifically, the impairment tests implemented by the exemplary systems and methods disclosed herein include, but are not limited to: (a) Horizontal Gaze Nystagmus Test—assesses the ability of a test subject to smoothly track a horizontally moving object and checks for eye stability during the test; (b) Vertical Gaze Nystagmus Test—checks for eye stability as the test subject tracks a vertically moving object; (c) Lack of Convergence Test—checks the ability of the test subject to cross his or her eyes when an object is brought towards the bridge of the subject's nose; (d) Pupil size and response test—measures the subject's pupil size in normal lightning conditions, as well as abnormally dark and bright conditions; and, (e) Modified Romberg Balance Test—tests the subject's ability to follow directions, measure time, and balance.

The exemplary systems and methods implement the presently disclosed impairment tests in a virtual world through use of a VR headset configured to include eye tracking hardware and software. As each test is conducted, the exemplary eye tracking hardware and software is capable of accurately measuring pupil size, pupil position, and eye gaze direction independently for each eye at a high sample rate. In order to make determinations of the test subject's level of impairment, the presently disclosed systems and methods calculate various useful values and offsets from the eye tracking data collected during each time step of the VR simulation. The calculated values and offsets based on the eye tracking data is then output as useful information from which determinations of impairment can made objectively, repeatedly, reliably, and accurately, while eliminating or substantially reducing the subjective nature inherent in previous manual impairment tests performed in the field.

In accordance with one particular embodiment, a system configured to implement one or more tests indicative of impairment due to drugs or alcohol is disclosed. The system includes a VR headset configured to display at least one virtual environment and at least one virtual object in the environment. An eye tracking component is also included, and a processor is in communication with a memory which has instructions to measure a change in one or more features of a test subject's eyes with the eye tracking component. The change in one or more features of the test subject's eyes is induced by a manipulation of the at least one virtual environment or the at least one virtual object displayed by the VR headset.

In additional embodiments, one or more sensors are in communication with the eye tracking component. The one or more sensors include cameras, body tracking sensors, accelerometers G-sensors, gyroscopes, proximity sensors, electrodes for obtaining EEG data, and thermometers and other temperature sensors or non-invasive sensing devices.

In other embodiments, a light blocking device is mounted to the VR headset. In further embodiments, one or more light sources are mounted to the VR headset.

In some embodiments, the memory includes a testing component having instructions to perform the one or more tests indicative of impairment. The testing component outputs parameter values for each of the one or more tests.

In further embodiments, the memory also includes a comparison component having instructions to correlate the parameter values output by the testing component with parameter values associated with predetermined baseline standards of impairment. The predetermined baseline standards of impairment can include parameter values related to one or more of a timestamp, a test state, scene settings, left pupil size, right pupil size, eye gaze to target cast distance, eye gaze to target cast vertical angle, eye gaze to target cast horizontal angle, eye horizontal angle to normal, eye vertical angle to normal, distance between eye focus points, eye position, and eye jitter.

In some other embodiments, the memory further includes a decision component having instructions to predict a level of impairment based on the correlation of the comparison component. Moreover, the memory can also include an output component to output the impairment prediction of the decision component or the correlation of parameter values of the comparison component.

In additional embodiments, a host computer is included which is associated or in communication with the VR headset. The host computer has a display device configured to show a real-time representation of the test subject's eyes. In some embodiments, the real-time representation includes a pair of animated eyes and a visual depiction of tracking by the eye tracking component.

In some further embodiments, one or more peripheral devices are included which are connected in communication with the VR headset. The one or more peripheral devices are configured to supplement data generated by the eye tracking component.

In some embodiments, the manipulation of the at least one virtual environment includes a change in brightness, contrast, or color of a scene in the virtual environment. In other embodiments, the manipulation of the at least one virtual object includes a change in brightness, contrast, color, direction, or location of the virtual object. In further embodiments, the manipulation of the at least one virtual object includes random movement, discrete increasing angular movement, smooth movement, or jittery movement of the virtual object.

In accordance with another particular embodiment, a method of indicating impairment due to drugs or alcohol is disclosed. The method includes displaying at least one virtual environment and at least one virtual object in the environment with a VR headset. The VR headset includes an eye tracking component and a processor in communication with a memory, and the memory includes instructions for measuring a change in one or more features of a test subject's eyes with the eye tracking component. The method further includes manipulating the at least one virtual environment or the at least one virtual object displayed by the VR headset; measuring parameter values representative of the change in one or more features of the test subject's eyes after manipulating the virtual environment or the virtual object; and correlating the measured parameter values with predetermined baseline standards of impairment. In addition, a level of impairment based on the correlation with the predetermined baseline standards of impairment can be predicted and output. Further, the correlation with the predetermined baseline standards of impairment can also be output.

In some embodiments, the measuring of parameter values further includes recording at least one of a timestamp, a test state, scene settings, left pupil size, right pupil size, eye gaze to target cast distance, eye gaze to target cast vertical angle, eye gaze to target cast horizontal angle, eye horizontal angle to normal, eye vertical angle to normal, distance between eye focus points, eye position, and eye jitter.

In other embodiments, the manipulating of the at least one virtual object includes changing a brightness, contrast, color, direction, location, or movement of the virtual object.

In further embodiments, the manipulating of the at least one virtual environment comprises changing a brightness, contrast, or color of a scene in the virtual environment.

In accordance with an additional particular embodiment, a method of operating a VR headset to indicate impairment due to drugs or alcohol is disclosed. The method includes: initiating one or more software components configured to perform one or more tests with the VR headset which indicate impairment due to drugs or alcohol; running the selected test according to instructions provided by the one or more software components; activating an eye tracking hardware component to begin recording raw eye tracking data and conditions of at least one virtual tracking object and at least one virtual environment generated by the VR headset; and saving and outputting data generated after running the selected test.

These and other non-limiting characteristics of the disclosure are more particularly disclosed below.

DETAILED DESCRIPTION

As used in the specification and in the claims, the terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that require the presence of the named components/ingredients/steps and permit the presence of other components/ingredients/steps. However, such description should be construed as also describing systems or devices or compositions or processes as “consisting of” and “consisting essentially of” the enumerated components/ingredients/steps, which allows the presence of only the named components/ingredients/steps, along with any unavoidable impurities that might result therefrom, and excludes other components/ingredients/steps.

A value modified by a term or terms, such as “about” and “substantially,” may not be limited to the precise value specified. The modifier “about” should also be considered as disclosing the range defined by the absolute values of the two endpoints. For example, the expression “from about 2 to about 4” also discloses the range “from 2 to 4.” The term “about” may refer to plus or minus 10% of the indicated number.

The following examples are provided to illustrate the methods, processes, systems, and properties of the present disclosure. The examples are merely illustrative and are not intended to limit the disclosure to the materials, conditions, or process parameters set forth therein.

With reference toFIGS. 1-2, block diagrams are illustrated showing a system100for performing an impairment test according to an embodiment of the present disclosure. The system100generally includes a virtual-reality (“VR”) headset unit102and an associated host computer104.

As shown inFIG. 1, the VR headset102includes various hardware components, including but not limited to eye tracking hardware106, a display device such as screen108, one or more sensors110, optionally one or more add-on hardware components112. The eye tracking hardware106can be provided as a single chip, such as an application-specific integrated circuit (“ASIC”), which includes a processor114, local memory116, and instructions118for processing the data generated from the tracking hardware. The instructions118may include one or more software components, here illustrated as eye tracking component software120.

More particularly, eye tracking component software120includes computer program code configured to locate, measure, analyze, and extract data from a change in one or more features of a test subject's eyes. The change in one or more features of the test subject's eyes is generally induced by a moving object which the test subject tracks in a virtual scene displayed on the screen108of the VR headset102.

Other changes in the one or more features of the test subject's eyes can be induced, for example, by changing one or more virtual environmental conditions of the virtual scene displayed on the screen108of the VR headset102(e.g., the brightness of the virtual scene). The local memory116stores the instructions118to implement the eye tracking software120, and the instructions are configured to perform at least part of the method illustrated inFIG. 12(discussed in further detail below). The processor114, being in communication with the memory116, executes the instructions118.

The data generated during processing by the eye tracking hardware106and software120can be stored in data memory132, which is separate or integral with local memory116. In addition, or alternatively, data generated by the eye tracking hardware106and software120can be output to the host computer104for further processing, via input/output (I/O) device122.

As illustrated inFIG. 1, the raw data stored in memory132includes, for example, timestamp162, eye gaze origin164, eye gaze direction166, pupil position168, and absolute pupil size170datasets. Data related to the screen108, the one or more sensors110, and the optional one or more add-on hardware components112can be similarly stored in data memory132and/or output to host computer104. Hardware components106,108,110,112,114,116,122,132of the VR headset102can be communicatively connected by a data/control bus124.

In some embodiments, the one or more additional sensor components110of the VR headset102include but are not limited to cameras110a, body tracking sensors110b, infrared (“IR”) sensors110c, G-sensors110d, gyroscopes110e, proximity sensors110f, and electrodes110gfor obtaining electroencephalogram (EEG) data. The cameras110afurther optionally include a video recording device which records eye movement during testing. Furthermore, the one or more additional sensor components can include a thermometer or other temperature sensor, along with other non-invasive sensing devices (not shown). The preceding is a non-exhaustive list of exemplary additional sensors and components, and any other desired sensors, devices, or components may be used without departing from the scope of the present disclosure. That is, the skilled artisan will appreciate that other suitable sensors and devices for gathering relevant test subject data can be used in connection with the VR headset102.

In other embodiments, the add-on hardware112can include one or more additional sensors, such as accelerometers (not shown). The add-on hardware112can also include one or more additional condition modifiers mounted to the VR headset102. For example, a light blocking device can be mounted to the VR headset102for individual pupil response testing on each eye separately. Similarly, additional illumination devices such as small LED light sources can be mounted to the VR headset102for additional testing control on each eye separately.

In other additional embodiments, the eye tracking hardware component106can include one or more sensors (not shown), such as eye tracking sensors and IR illuminators. Moreover, the eye tracking hardware component106can implement different types of eye tracking techniques, including but not limited to binocular dark pupil tracking.

As shown inFIG. 2, the host computer104includes a display device105, a processor126, main memory128, instructions130, and data memory133. All the data generated by the eye tracking hardware106and the data provided by the screen108, one or more sensors110, and one or more optional add-on hardware components112of the VR headset102and stored in memory116/132can be input via input/output (I/O) device134. The same I/O device134may be used to receive information from one or more devices associated with the VR headset102and/or host computer104, such as the illustrated one or more peripheral devices146.

The main memory128stores the instructions130, which are configured to perform at least part of the method illustrated inFIG. 12. The processor126, being in communication with the memory128, executes the instructions130. Data memory133, which is separate or integral with main memory128, stores data produced during execution of the instructions130by the processor126. The data stored in memory128/133can be output as impairment indicator information140, via the same I/O device134mentioned above or a separate I/O device136. An impairment prediction142(i.e., degree and probability of impairment), based on the impairment indicator information140, may also be output by the system.

Hardware components126,128,133,134, and136of the host computer104can be communicatively connected by a data/control bus138. The VR headset102and the one or more peripheral devices146may be communicatively connected with the host computer102by a wired or wireless link144, including but not limited to the Internet, Bluetooth, USB, HDMI, and/or DisplayPort, for example.

The instructions130may include several software components, here illustrated as a user interface or VR software148, an impairment testing component150, a test processing/comparison component152, a decision component154, an output component156, and any other add-on software158associated with the optional add-on hardware112included with the VR headset102. The processor126and software components148-152,156, and optionally158are configured to analyze, extract, calculate, and/or correlate information from the raw data generated by the eye tracking hardware106and stored in data memory132.

More particularly, the user interface148includes computer program code to communicate with the user or test administrator via the screen108or display device105of the host computer104. For example, once instructed by a user or test administrator, the user interface148causes the screen108of the VR headset102(or host display device105) to display any number of virtual scenes (such as scene202inFIGS. 4-8 and 10or scene204inFIG. 9discussed in further detail below). Each virtual scene generally includes one or more dynamic component(s) (such as moving target208illustrated inFIGS. 9-10) configured to generate a change in one or more features of a subject's eye(s). In addition, when the host computer104includes a separate display device105, real-time test data can be shown on the display device and include, for example, graphical representations of eye position, graphs, charts, etc.

In some particular examples, as best shown inFIG. 4, the real-time test data displayed on the screen105of the host computer104is represented by a pair of animated eyes149which correspond to the eyes of the test subject. In this regard, the animated eyes149presented on the screen105show what the test administrator would see if the administrator were able to look directly at the test subject's face. The animated eyes149communicate various points of information to the test administrator, such as eyeball position, pupil size, and the open/closed state of the eye. A graphical top down display151may also be included on the screen105to provide options to display, for example, eyeball location, target ball location, and gaze direction of each eye. The cornea and/or pupil of the animated eyes are each positioned in a square153that represents the eye tracking sensor arrays on a normalized coordinate system. However, other visual depictions of the tracking by the eye tracking component could also be used. The display of animated eyes149and the information described above on the screen105of the host computer104displays are useful for the test administrator to verify that proper eye tracking calibration has been performed and that the test subject is following directions.

As discussed above, the eye tracking component106of the VR headset102is configured to locate, measure, analyze, and extract data from the change in one or more eye features which has/have been induced by the virtual scene displayed on the screen108by the user interface148.

The testing component150includes computer program code to store and retrieve from memory components116,128,132, and/or133, information necessary to perform various impairment tests, including but not limited to lack of convergence (“LOC”) test, horizontal and vertical gaze nystagmus tests (“HGN” and “VGN”, respectively), pupil dilation test, color sensitivity test, and targeting test. The testing component150can further include computer program code to store and retrieve information necessary to perform tests that screen for medical conditions that may affect the results of the various impairment tests (e.g., eye injury, astigmatism, etc.). Similarly, program code for performing tests which rule out conditions could also be included as part of testing component150. For example, conditions such as optokinetic nystagmus, vestibular nystagmus (resting), etc., may result in a positive impairment indication unless the testing component150is configured to test for and rule out these conditions. The information retrieved with the testing component150includes but is not limited to: predetermined testing parameters/equations for each impairment test; and, the raw data generated by the eye tracking component106and stored in data memory132of the VR headset102.

Moreover, the impairment testing component can optionally be configured to retrieve user data on the subject undergoing the test. User data can be input through the one or more peripheral devices146communicatively connected to the VR headset102and/or host computer104, or through a user input device (not shown) associated with the host computer. Once the necessary information is retrieved, the testing component150inputs the information into the testing parameters/questions to determine output parameter values for each impairment test performed. The testing component150can further include computer program code to update or manage memory components116and128. For example, memory can be updated with customized parameters used by the processing/comparison component152and the decision component154to calculate the resultant impairment indicating information140and impairment prediction142of the subject. The parameter values output from the testing component150will subsequently be used to determine a test subject's level of impairment and can optionally be stored in data memory133.

The processing/comparison component152includes computer program code to correlate the retrieved testing parameters and associated output values from the testing component150with a corresponding baseline standard of impairment/non-impairment and its associated parameter values. More particularly, each of the testing parameters utilized by the testing component150are compared with local data-store160, which contains predetermined or premeasured baseline standards of impairment/non-impairment and their baseline parameter values. If a match is found between the testing parameters and the baseline standards, the associated baseline parameter values, or a representation thereof, is/are extracted. The correlations made by the processing/comparison component152can optionally be stored in data memory133. The data-store of baseline standards160can also be stored in memory128.

The processing/comparison component152further includes computer program code configured to compare the output values and/or baseline standards with one or more confidence metrics stored in local data-store160. For example, one confidence metric includes historical data of each individual impairment test result which is accessed by the processing/comparison component152to assess the confidence of an indication of impairment. Such historical data could further include drug class identification results with probability or percent matches associated with one or more drug classes. Each of the testing parameters utilized by the testing component150are compared with these confidence metrics in the local data-store160, and if a match is found between the testing parameters/baseline standards and the confidence metrics, the associated confidence metric, or a representation thereof, is/are extracted and are optionally stored in data memory133.

In addition, or alternatively, the baseline standards, associated baseline parameter values, and associated confidence metrics from data store160that have been matched with the testing parameters and associated values output from the testing component150are output as impairment indication information140. That is, the values of each the baseline parameters, confidence metrics, and testing parameters, or representations thereof, are output via output component156. As illustrated inFIG. 2, the baseline parameters and testing parameters, as well as the values associated therewith, can be related to one or more of a timestamp172, test state174, scene settings176, left pupil size178, right pupil size180, eye gaze to target cast distance182, eye gaze to target cast vertical angle184, eye gaze to target cast horizontal angle186, eye horizontal angle to normal188, eye vertical angle to normal190, distance between eye focus points192, eye position194, and eye jitter196. These testing parameters are discussed in greater detail below.

Some of the aforementioned testing parameters are directed the state or status of the system100itself. For example, the timestamp172testing parameter refers to the time that each set of data originates from, measured in seconds, minutes, hours, etc. The test state174refers to an integer representing what part of the test is running at the time the sample is taken. For example, the integer “1” may be a test state integer indicating that a first part of the lack of convergence test (“LOC”) was running at a timestamp of 30 seconds into the test.

Other testing parameters are directed toward information and data that may be useful for the aforementioned pupil size and response test, along with the color sensitivity test. For example, the scene settings176refers to various characteristics of the scene displayed on the screen108of the VR headset, including but not limited to scene brightness and scene colors. The brightness in the scene settings176is changed for the pupil response test, and specific colors in the scene settings are changed for the color sensitivity test. For example, in the color sensitivity test, VR headset102is configured to observe whether the test subject responds to yellow and/or blue colors. In this regard, yellow/blue color vision loss is rare and thus serves as an indicator of impairment. Left pupil size178refers to the size of the test subject's left pupil, measured in millimeters by the eye tracking hardware106and software120. Right pupil size180refers to the size of the test subject's right pupil, measured in millimeters the eye tracking hardware106and software120.

Some of the other testing parameters are directed toward information and data that may be useful for the aforementioned horizontal and vertical gaze nystagmus tests, as well as the lack of convergence test. For example, the eye gaze to target cast distance182refers to the distance between the point where the test subject is looking and the object the test subject is supposed to be looking at, measured in meters by the eye tracking hardware106and software120. The eye gaze to target cast distance182is calculated separately for each eye, and the estimated overall point of focus with both eyes is calculated with the eye tracking software120. The eye gaze to target cast vertical angle184refers to the angle between the test subject's gaze and a direct line from their eyes to the tracking object, measured in degrees on the vertical plane by the eye tracking hardware106and software120. The eye gaze to target cast vertical angle184is also calculated for each eye and the total gaze. The eye gaze to target cast horizontal angle186refers to the angle between the test subject's gaze and a direct line from their eyes to the tracking object, measured in degrees on the horizontal plane by the by the eye tracking hardware106and software120. The eye gaze to target cast horizontal angle186is also calculated for each eye and the total gaze. The eye horizontal angle to normal188refers to the angle of each eye's gaze relative to the forward direction of the test subject's head, measured in degrees on the horizontal plane by the eye tracking hardware106and software120. The eye vertical angle to normal190refers to the angle of each eye's gaze relative to the forward direction of test subject's head, measured in degrees on the vertical plane by the eye tracking hardware106and software120.

As illustrated inFIG. 3, the distance between eye focus points192testing parameter refers to a numerical indicator, measured by the eye tracking hardware106and software120, of how well the test subject can cross his or her eyes in the lack of convergence test. The distance between eye focus points192is measured on reference plane B, relative to the test subject's eyes e1, e2and focus point A on tracked object208, where focus point A is located a distance X1away from the test subject's eyes, and reference plane B is located a distance X2away from the test subject's eyes that is twice as far as distance X1.

The remaining testing parameters mentioned above are related to eye movement in general, which may be useful for all the aforementioned impairment tests. The eye position194refers to the X and Y coordinate position of each of the test subject's pupils within the eye socket, measured by the tracking hardware106and software120. The eye jitter196refers to the angle between each test subject's eye's direction and the direction of each eye at the last sample, measured in degrees by the eye tracking hardware106and software120. Eye position194and eye jitter196information may be particularly useful for the previously mentioned targeting test. The targeting test measures ability to detect the presence of an object that appears in a test subject's field of view and the test subject's ability to focus their gaze on that object. The targeting test is administered by making an object appear at several locations for a set amount of time in the test subject's field of view. In some particular examples, an object appears in eight different locations in the test subject's field of view for about 3 to 5 seconds, where each object location includes a different direction and distance metric. The test subject is instructed to focus their gaze on the target object when detected, and the appropriate eye data is measured and recorded upon detection.

In some embodiments, after the processing/comparison component152has made correlations, the optional decision software component154is then utilized. The decision software component154includes computer program code to predict a level of impairment (that is, predict a probability and degree of impairment of a test subject), based on the correlated parameter values determined by the processing/comparison component152. That is, for any testing parameter and baseline standard being correlated by the processing/comparison component152, if the testing parameter output value(s) exceeds one or more thresholds (e.g., value(s) over a period of time, too many high and/or low values, total value too high/too low, etc.) set for the corresponding baseline output value(s), the decision component154may output a prediction142that the test subject is impaired at an estimated degree. The impairment prediction142of the decision component154can optionally be stored in data memory133. In addition, or alternatively, the impairment prediction142, or a representation thereof, can be output to the test subject or test administrator via the output component156. The output component156can output the impairment prediction142alone or together with the correlated baseline standards, associated baseline parameter values, testing parameters, and associated testing parameter values.

In other embodiments, the decision software component154is not utilized. That is, the host computer104does not make an impairment prediction. In such embodiments, a user or administrator of the VR headset102and host computer104may prefer to make his/her own impairment prediction based on a review of the impairment indication information140. In such cases, only the impairment indication information140(i.e., the correlated baseline standards, associated baseline parameter values, testing parameters, and associated testing parameter values, or representations thereof) is output via output component156.

In any event, the output component156includes computer program code to output one or both impairment indication information140and impairment prediction142, or a representation thereof. More particularly, information140and prediction142are output by the output component156and to the user interface148, such that the screen108of the VR headset102and/or display device105of the host computer104can display the information to the test subject or test administrator. Moreover, the eye data saved for each test subject in information140is saved in at least one memory component116,128,132, or133of the VR headset102or host computer104. Generally, the information140is saved in an appropriate format which enables the loading and replaying of test data files for any test subject. If desired, the entire test for a test subject can be replayed using the animated eyes149shown on the display device105of the host computer104as described above. In some particular examples, the information140can be saved to memory in the XML format.

The host computer104can include any number of known user input devices (not shown), such as a keyboard, touch or writable screen, voice or AI assistant, and/or a cursor control device, such as mouse, trackball, or the like, used by a user for inputting additional desired information. Alternatively, such user input devices can be directly connected to the VR headset102for inputting the additional information. The computer104may include a PC, such as a desktop, a laptop, server computer, cellular telephone, tablet computer, etc., or a combination thereof.

The memory components116,128,132, and133of both VR headset102and host computer104may represent any type of non-transitory computer readable medium such as random-access memory (RAM), read only memory (ROM), magnetic disk or tape, optical disk, flash memory, or holographic memory. In one embodiment, the memory components116,128,132, and133comprises a combination of random access memory and read only memory.

In some embodiments, the processors114and126of the VR headset102and host computer104, respectively, along with associated memory components116,132and/or128,133may be combined in a single chip. In addition, processors114and126can be variously embodied, such as by a single-core processor, a dual-core processor (or more generally by a multiple-core processor), a digital processor and cooperating math coprocessor, a digital controller, or the like.

Turning now toFIG. 4, an illustration is provided which demonstrates use of the exemplary systems and methods for creating a VR environment to implement impairment tests. Test subject T is illustrated wearing the exemplary VR headset102having all the features described above. The VR headset102is communicatively connected to the host computer104, such as through peripheral connection or link device146. The “virtual world” in which the VR impairment test of the present disclosure is administered to test subject T can be given dimensions through the use of one or more peripheral devices146aand146b, which are communicatively connected to VR headset102and/or host computer104. Peripheral devices146aand146billuminate the outlined space inFIG. 4with IR light. As described in greater detail above, the VR headset102can include one or more IR sensors configured to receive the IR light generated by the peripheral devices146aand146b, and thereby define the dimensions of the virtual world.

Moreover, one or more handheld peripheral devices146cand146dcan also be communicatively connected with one or both the VR headset102and host computer104. The one or more handheld peripheral devices146cand146dcan also include one or more IR sensors configured to receive the IR light generated by the peripheral devices146aand146b. In this regard, the eye tracking data generated by the eye tracking hardware of the VR headset102can be supplemented with additional data generated, recorded, extracted, analyzed, etc., by the responses of test subject T's hands and arms via handheld peripheral devices146cand146d. However, such a configuration is non-limiting.

Referring now toFIGS. 4-10, one or more virtual scenes202,204are illustrated as would be displayed: (a) to a test subject undergoing the one or more impairment tests of the present disclosure (such as test subject T inFIG. 4); and (b) on the screen108of the VR headset102described above. InFIG. 5, the screen108shows a virtual scene202representing an empty room where the subject VR impairment tests will take place. It should be understood that the empty room illustrated in virtual scene202is only exemplary, and any desired scene may be replicated in the virtual world on screen108without departing from the scope of the present disclosure. The screen108and virtual scene202illustrated inFIG. 5are representative of a welcome screen that would be seen by a test subject, as indicated by the test labeled206a.

InFIG. 6, the screen108shows the same virtual scene202as shown inFIG. 5, with the exception that two tracking objects208aand208bare now presented. As indicated by the text206b, the virtual scene202illustrated inFIG. 6is representative of a calibration process for a test subject to ensure that the exemplary VR headset of the present disclosure is properly positioned, with respect to the test subject's field of vision, to obtain data from the eye tracking hardware and software components, as well as any from other connected devices, sensors, etc., that may be communicatively connected to the VR headset or host computer. The two tracking objects208aand208billustrated inFIG. 6are simple spheres which are configured to move around the screen108based on the instructions provided by the eye tracking software.

InFIG. 7, the screen106shows a partially zoomed in view of virtual scene202fromFIGS. 5 and 6. However, the illustration inFIG. 7is representative of a calibration process for adjusting the position of the entire VR headset, as indicated by text206c.

InFIG. 8, the screen108shows a similar virtual scene202as that ofFIGS. 5-7. However, as indicated by the text206d, the virtual scene202illustrated inFIG. 8is representative of an instruction screen presented to a test subject undergoing the exemplary impairment tests of the present disclosure.

InFIG. 9, the screen106shows another zoomed in view of virtual scene202fromFIGS. 5, 6 and 8. In accordance with one example, the illustration inFIG. 9is representative of a calibration process for the tracking of the eye tracking hardware and software components. As illustrated inFIG. 9, one tracking object208is presented. The tracking object208illustrated inFIG. 9is a simple sphere which is configured to move around the screen108based on the instructions provided by the eye tracking software120. As another example, the illustration inFIG. 9is representative of a screen presented to a test subject during eye tracking data generation and collection for the aforementioned targeting test. In this regard, during administration of the targeting test, the tracking object208would appear at one of several different locations on screen108, for about 3 to 5 seconds at each location, so that the test subject's ability to detect and focus on the object can be measured.

InFIG. 10, the screen108shows a new virtual scene204, where the tracking object208fromFIG. 9is still present, but one or more virtual environmental conditions have changed. More particularly,FIG. 10is representative of a screen presented during eye tracking data generation and collection when a brightness of the scene settings176is increased for the pupil response test. Such a change in environmental conditions can be presented to a test subject when it is desired to perform a pupil size and response test for impairment, where left pupil size178and right pupil size180are measured. Another example of changing a virtual environmental condition similar to the scene brightness inFIG. 10includes changing the color of the tracking object208to yellow/blue and measuring the test subject's tracking response. An additional example, referred to as a “random tracking test”, involves recording the test subject's eye reactions when the tracking object208appears randomly at points in the space of virtual scene204. In yet another example, the tracking object208or other visual stimulus appears at discrete increasing angles from the test subjects instead of smooth movement across the virtual scene204. Such a test is useful for measuring the angle of onset of gaze nystagmus. In a further example, the contrast of the tracking object208or other visual stimulus can be changed. Contrast control is useful to ensure that the VR headset102operates correctly for color blind test subjects.

Finally, inFIG. 11, the screen108shows the previous virtual world202with the brightness level returned to normal compared withFIG. 10.FIG. 11also includes the one tracking object108configured to move around the screen108based on the instructions provided by the eye tracking software120.

As indicated by text206e, the virtual scene202as illustrated inFIG. 11is representative of a result or output screen that would be presented to a test subject or test administrator after completion of one of the one or more exemplary VR impairment tests of the present disclosure. More particularly, as indicated by text206e, theFIG. 10is representative of a result or output screen presented after eye tracking data generation and collection for one or more of the impairment indication information described above, such as eye gaze to target cast vertical angle184, eye gaze to target cast horizontal angle186, or eye horizontal angle to normal188. Such data may be generated and collected during administration of, for example, the horizontal or vertical gaze nystagmus impairment tests discussed above.

Turning now toFIG. 12, a flowchart is illustrated that represents an exemplary method for performing an impairment test according to an embodiment of the present disclosure. The method begins at S300, where the software components described above with respect to the VR headset102and host computer104are initiated for performing the one or more VR impairments tests. Next, at S302, at least one of the one or more VR impairments tests are selected for administration by a test subject or test administrator using VR headset102and host computer104. At S304, the one or more selected VR impairment tests are run according to the instructions and software components included in the VR headset102and host computer104. In addition, the associated hardware components of VR headset102and host computer104, such as eye tracking hardware106, are activated to begin recording tracking object position, room conditions, raw eye tracking data, and calculated performance metrics, such as those described in data memory132. At S306, the method continues with saving test data generated after running the one or more selected VR impairment tests and results thereof are output and displayed to one or both of the test subject and test administrator. At S308, the method determines whether it is necessary to run one or more additional VR impairment tests, and/or repeat the previously selected one or more VR impairment tests. If yes, the method returns to step S302discussed above. If no, the test(s) end at S310.

Turning now toFIGS. 13-17, various charts are presented which illustrate some results of representative of the exemplary one or more VR impairment indicator tests discussed above. InFIG. 13, a chart titled “Smooth Pursuit Test” is provided which shows an exemplary eye tracking test having short durations. The Smooth Pursuit Test ofFIG. 13is representative of the type of information output as part of the impairment indication information140described above. In particular,FIG. 13includes values for at least one of the eye gaze to target cast vertical angle184or eye gaze to target cast horizontal angle186testing parameters, as shown on the Y-axis labeled “Gaze Offset from Target (degrees)”. Such data would be generated and collected over the timestamp testing parameter values shown on the X-axis and during administration of, for example, the horizontal or vertical gaze nystagmus impairment tests discussed above. Moreover, the testing parameter labeled as “Deviations from tracking target” are representative of values which may indicate impairment, as these deviations fall well outside the indicated baseline or “Normal Tracking” parameter values.

InFIG. 14, another chart titled “Smooth Pursuit Test” is provided which shows an exemplary eye tracking test for a horizontally moving object having jittery motions. The Smooth Pursuit Test ofFIG. 14is again representative of the type of information output as part of the impairment indication information140described above. In particular,FIG. 14includes values for the eye gaze to target cast horizontal angle186and eye jitter196testing parameters. Such data would be generated and collected over the timestamp testing parameter values shown on the X-axis and during administration of, for example, the horizontal or vertical gaze nystagmus impairment tests discussed above. Moreover, the results illustrated in the chart ofFIG. 14are representative of testing parameter values which may indicate impairment, as the plot line inFIG. 14is jagged and jittery and an indication of non-impairment would likely result in a smooth line over time (i.e., less deviations on the Y-axis labeled “Gaze Angle to Target (degrees)”.

InFIG. 15, a chart titled “Lack of Convergence Test” is provided which shows an exemplary eye tracking test when the test subject is crossing his or her eyes. The Lack of Convergence Test result in the chart ofFIG. 15is further representative of the type of information output as part of the impairment indication information140described above. More particularly,FIG. 15includes values that would be expected for the eye horizontal angle to normal188and eye vertical angle to normal190testing parameters, as indicated by the chart key which labels the two data plot lines as “Left eye h angle to normal” and “Right eye h angle to normal”, as well as the Y-axis labeled “Eye Rotational Angle (Degrees)”. Such data would be generated and collected over the timestamp testing parameter values shown on the X-axis and during administration of the Lack of Convergence Test as described above. Moreover, the testing parameter values, as represented by the data plot lines for each eye inFIG. 15, are representative of values which may indicate non-impairment. This is because the left and right eye angles h to normal run substantially proportional to one another at the data plot lines indicated as “Tracking at a distance”, as would be expected in a non-impaired state. This is also because the left and right eye angles h to normal run substantially inversely proportional to one another at the data plot lines indicated as “Eyes crossing to track close to face”, as would be expected in a non-impaired state.

In contrast to the chart illustrated inFIG. 15, the chart inFIG. 16illustrates a failure of the Lack of Convergence Test where the test subject is unable to cross his or her eyes. That is,FIG. 16illustrates results which would be expected from a test subject in an impaired state.

Finally, inFIG. 17, a chart titled “Pupil Size Test” is provided which shows exemplary eye tracking results when the test subject's pupil's decrease in size in response to a change in scene brightness. The Pupil Size result in the chart ofFIG. 17is further representative of the type of information output as part of the impairment indication information140described above. More particularly,FIG. 17includes values that would be expected for the scene settings176, left pupil size178, and right pupil size180testing parameters, as indicated by the Y-axis labeled “Pupil Size Measurement”. Such data would be generated and collected over the timestamp testing parameter values shown on the X-axis and during administration of the Pupil Size Test as described above. Moreover, the testing parameter values, as represented by the data plot lines for the pupil of each eye inFIG. 17, are representative of values which may indicate non-impairment. This is because the left and right pupil sizes decreased nearly immediately concurrent with the light intensity change indicated in the chart shortly after the 30 second timestamp on the X-axis. Moreover, a slight rebound in pupil size is indicated between the 32 and 35 second timestamps, which is representative of pupil dilation for test subjects in a non-impaired state.

EXAMPLES

Various impairment tests were performed using a VR headset102according to the embodiments described above. That is, a VR headset102configured for detecting impairment of a test subject, as discussed above, was used at a Drug Recognition Expert (“DRE”) training event at the Ohio State Highway Patrol Training Academy. The DRE training event was an alcohol “wet lab”, where controlled doses of alcohol were administered to volunteer test subjects. These wet labs are useful to train DRE students and officers in conducting field sobriety testing on volunteer test subjects. Wet labs like these are also useful for testing new impairment detecting devices, such as the VR headset102described herein.

During the lab, volunteer test subjects were asked to wear a VR headset102configured to act as an impairment sensor. Data was then gathered from sober and impaired subjects. Each subject that used the VR headset102was tracked with a unique ID number along with their most recent blood alcohol content (BAC) reading as measured by the officers conducting the lab. In total, eight (8) test subjects volunteered to use the VR headset102along with seven (7) other sober individuals. Most test subjects that used the VR headset102provided a sober baseline measurement before consuming alcohol. In addition, one or more additional tests were then performed at varying BAC levels to obtain measurements of impairment. All the relevant eye and testing data was recorded by sensor software of the VR headset102during each test.

The VR headset102conducted five (5) tests on each test subject. During each test, the VR headset102tracked both the test subject's eyes and gaze relative to an object. These five tests included: (1) a smooth tracking test; (2) a horizontal gaze nystagmus (HGN) test; (3) a vertical gaze nystagmus (VGN) test; (4) a lack of convergence (LOC) test; and, (5) a pupil response test. According to the DRE training material, alcohol affects the test subject's performance on each of these five tests. However, altered performance was not exhibited by all test subjects on these tests, depending on each test subject's level of alcohol intoxication, and some users exhibited characteristics of impairment despite being sober.

A general look at the data obtained from these tests reveals that LOC test failures are easily discernable (FIGS. 23-26). Moreover, numerical metrics that accurately assess whether the test subject was able to cross their eyes can be readily developed from the LOC test. The smooth tracking tests (FIGS. 18-19) also show a clear line between impaired and sober subjects, but a more advanced algorithm is utilized for the software to assess the “smoothness” of the tracking compared with the LOC test. The HGN and VGN tests (FIGS. 20-22) revealed gaze nystagmus in some test cases with impaired subjects, but not all. This could be due to many factors, but may be at least partially attributed to the limited field of view of the headset102preventing the software from pushing the eyes to an extreme enough angle for nystagmus to be present.

The pupil response tests (FIGS. 27-28) yielded very precise results for almost all test subjects. The pupil size for each test subject in high and low light conditions was easily extracted. A precise rate of change for each pupil when the lighting was changed from darkness to light was also able to be extracted. However, there was not a clear correlation between pupil response rate and intoxication level, as some subjects experienced a slower response rate after consuming alcohol, but others experienced an increased response rate. Overall, based on a review of the data gathered at the aforementioned wet lab, the VR headset102successfully and accurately implemented visual tracking tests that were able to adequately detect physical evidence of impairment. The physical evidence of impairment detected by the VR headset102is the same as the impairment indicators exposed in known sobriety tests which generally must be performed manually by DRE officers.

Though these tests were conducted on subjects dosed with alcohol, the aforementioned wet lab still provides useful information to the VR headset's102applicability in the detection of other forms of impairment, such as cannabis impairment. The HGN and VGN tests (FIGS. 20-22) that are more difficult to detect and that weren't as present in the subjects at this event are not known to be good indicators of cannabis impairment. Lack of convergence, dilated pupils, and a transient effect called rebound dilation that occurs in the pupil response test are the reliable signs of cannabis impairment. The data gathered by the VR headset impairment sensor for both the LOC test and the pupil response tests revealed that impairment symptoms were very apparent in the data.

Example 1—Smooth Tracking Test

The smooth tracking test measured the test subject's ability to track an object that moves horizontally back and forth in front of them. A sober subject can move their eyes smoothly along with the object as it moves, but an impaired subject will exhibit jumpy, jittery, or delayed tracking.FIG. 18shows an example of a sober subject in the tracking test. When the subject becomes impaired, their inability to smoothly track the object is easily visible in the test data.FIG. 19shows an example of an intoxicated subject in this test. The jerky tracking shown inFIG. 19is easy to distinguish visually and is easily detected by the software of the VR headset102.

Example 2—HGN and VGN Tests

The horizontal and vertical gaze nystagmus tests moved the tracked object to the edge of the test subject's vision to induct nystagmus or jitter in the subject's eyes. According to the DRE instructors, horizontal nystagmus is more common and tends to be present at a lower level of intoxication than vertical nystagmus. Both forms of nystagmus also tend to present themselves at smaller eye offset angles as the test subject becomes more intoxicated. Not all users exhibited nystagmus at the angles of the tracking object created by the VR headset102, but there were clear examples of nystagmus in more intoxicated subjects.FIGS. 20 and 21show a test subject's baseline and intoxicated HGN tests, respectively.

Conducting HGN and VGN tests in software also permitted the analysis of nystagmus in ways that an officer would not be able to in the field. For example,FIG. 22shows a spectral analysis of the test result fromFIG. 21that shows very clear jitter at about 3.5 Hz. Spectral analysis like that inFIG. 22is useful both for identifying the presence of nystagmus but also for categorizing different types of nystagmus based on the frequency of the jitter. This categorizing of different types of nystagmus cannot by performed by an officer in real time during a field test.

Example 3—LOC Test

The LOC test involved moving the tracked object towards the bridge of the test subject's nose and assessing the test subject's ability to cross their eyes and maintain focus on the tracked object. The angles of the subject's eyes were measured by the sensor directly, so the data from this test clearly shows the ability or inability of a test subject to cross their eyes. The DRE training material indicates that alcohol contributes to a test subject exhibiting lack of convergence.FIG. 23shows an average quality sober result of the LOC test. The test subject from theFIG. 23was able to cross their left eye more accurately than their right eye but could still reach a reasonable level of convergence. As the test subject becomes impaired, lack of convergence can be demonstrated in various ways. With the test subject fromFIG. 23, intoxication reduced the ability of the right eye to converge on the tracking object. This result is shown inFIG. 24.

The LOC test above shows a subtler example of lack of convergence, where both eyes moved towards the stimulus but were not able to accurately converge. Some test subjects exhibited a more pronounced lack of convergence, such as the test subjects shown inFIGS. 25 and 26. InFIG. 25, one eye of the test subject tracked correctly, and the other eye went the wrong direction. InFIG. 26, the test subject was completely unable to track the object as it moved toward the test subject.

Example 4—Pupil Response Test

The pupil response test measured the test subject's reaction to light by examining how the pupils responded to changing light intensities. This was conducted by putting the test subject in a low light condition for 90 seconds then quickly shining a bright light into the eyes, thereby causing the test subject's pupils to restrict. The test examines the resting pupil size in both the low light and bright light conditions, as well as the rate of change of the pupil size when exposed to light. The DRE training material suggests that test subjects under the influence of alcohol should have normal pupil sizes in any given lighting condition but have a slowed reaction speed to light.

The pupil size measurements from the VR headset102during testing was accurate and repeatable. Both the change in steady state pupil size and the transient size change were easily visible, as seen in the baseline test shown inFIG. 27and the impaired test shown inFIG. 28. Some noise resulted in the measurements from test subjects with particularly droopy eyelids and subjects that blinked frequently. However, as seen at about the 5 second mark inFIG. 27, the VR headset102registers this as extremely high amplitude noise that is easily filtered out of the data. Test subjects tended to blink more and create more noise in the data as intoxication levels increased. However, software data analysis was still able to extract the correct response rate slope, as shown inFIG. 28.

While the subject ofFIG. 28exhibited about a 25% reduction in pupil reaction time, the pupil reaction time in other subjects sped up after being dosed with alcohol. Additionally, some subjects that had pupil reaction times slow after consuming alcohol still had reaction times faster than sober baselines of other subjects. A larger sample size and more refined data analysis techniques may be able to more successfully use pupil response rate to assess intoxication levels.

Regardless of the aforementioned inconsistencies, the VR headset102was able to accurately and consistently measure both pupil size and transient response rates for almost all tests. Even a crude slope detection algorithm was able to successfully detect the pupil size rate of change in most cases without prefiltering the blinks from the data, which gives confidence that a more developed analysis approach would be extremely consistent.

CONCLUSIONS

Data from the examples discussed above clearly showed the physical symptoms of impairment that officers look for in each of the 5 currently implemented impairment tests. The consistency, sensitivity, usability, and reliability of the VR headset102will be improved with larger sample sizes, but the preceding examples show that the impairment “clues” which officers look for are detectable by the VR headset102in at least some subjects at relatively low levels of impairment.

The system100including the VR headset unit102disclosed herein may be implemented with the capability to check that the device measurements are within allowed tolerances. These measurements may include but are not limited to eye angle, pupil diameter, head position/angle, and measurement frequency. In this regard, the VR headset unit102may be equipped with an optional calibration check instrument (not shown). Such a calibration check instrument generally includes a fixture with one or more mannequin style heads or partial heads including sets of false eyes or other representations that simulate eyes. The fixture is configured to provide various pupil diameters and eye angles to check the accuracy of the device under test. The fixture of the calibration check instrument may also include the capability to check the accuracy of head position and head angle measurements, in addition to checking measurement frequency to ensure accuracy of tests that involve time components. In some embodiments, the fixture may be mechanical. In other embodiments, the fixture may be driven by electric motors. Moreover, the false eyes may be fixed in position or they may be dynamic. The head position and angle may also be fixed or dynamic. Finally, the fixture may include of one or more sets of heads, each with one or more sets of false eyes.

To aid the Patent Office and any readers of this application and any resulting patent in interpreting the claims appended hereto, applicants do not intend any of the appended claims or claim elements to invoke 35 U.S.C. § 112(f) unless the words “means for” or “step for” are explicitly used in the particular claim.