Abstract:
A method of diagnosing attention deficit hyperactivity disorder (ADHD). The method employs a tangible graphical interactive game wherein the interactive game employs a plurality of tangible graphical cubes. The game induces stimuli, measures responses and accumulates the responses using a predefined set of variables into a predefined set of metrics, wherein the variables are determined using an interactive machine learning feedback algorithm.

Description:
RELATED APPLICATIONS 
       [0001]    The present application is a nonprovisional of U.S. provisional application No. 61/719,280, filed Oct. 26, 2012; and is a continuation-in-part of U.S. patent application Ser. No. 13/657,332, filed 22 Oct. 2012, which claims the benefit of provisional application Nos. 61/549,698, filed 20 Oct. 2011, and 61/551,384, filed 25 Oct. 2011; the contents of which are hereby incorporated by reference in their entirety. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of Invention 
         [0003]    The invention relates to the field of cognitive disorders including assessing attention deficits and/or comorbidities, treating attention deficit hyperactivity disorder, and/or conditions with attentional impairment such as autism spectrum disorder (ASD), anxiety, depression, and Alzheimer&#39;s. 
         [0004]    2. Background of the Art 
         [0005]    Based on the National Survey of Children&#39;s Health, 5.4 million children, 4-17 years of age, met criteria for attention deficit hyperactivity disorder (ADHD) in 2007.1 Despite this high number, only 50 percent of children and adolescents are estimated to be diagnosed and treated. A core deficit in ADHD is impairment in executive functioning. Significant debate about the causes and pathophysiology associated with its development persists. Studies investigating the neurochemistry, neuroimaging, and neuropsychological genetics have found evidence suggesting that frontal/striatal regions play a key role in the pathophysiology of executive functioning disorders, namely, the caudate, putamen, lateral prefrontal cortex, and dorsal anterior cingulate cortex. Abnormal structure, functioning, or both of these regions have been implicated in ADHD,4 further linked to abnormal functioning of norepinephrine and dopamine systems, which affects working memory. Working memory is the capacity to hold in mind information that is used to guide behavior, and deficits in this regard have been proposed to underlie inattention, distractibility, and poor academic performance identified in ADHD. 
         [0006]    Current diagnostic tools used to identify executive functioning deficits and child behavior problems are subjective, such as the Conners&#39; Rating Scales, and can thus produce variable clinical impressions. The Continuous Performance Test (CPT) is objective, but has not proved to be a reliable means to identify ADHD. The gold standard of diagnosis remains a professional psychiatric evaluation, which unfortunately is a limited resource in the United States and worldwide. 
         [0007]    A rapidly growing field called “Games-With-a-Purpose” has been successful in solving several serious scientific problems. Tangible user interfaces are a novel and useful medium for games meant to aid in diagnosis of behavioral disorders. Differing from traditional graphical user interfaces, tangible user interfaces allow participants to have more naturalistic interaction with presented information and experience through physically holding and moving the devices representing game objects and interact with games. This is an important advancement from the traditional use of simple button press responding. Furthermore, this technology introduces vastly more measurement opportunities not available with a graphical user interface (i.e., button presses). Precise movement can be detected, which is especially useful for examining executive functioning-based disorders that bear on motor behaviors such as hyperactivity involving excessive movement, while inattention leads to delays and increased variability in response to stimulation. 
       SUMMARY OF THE INVENTION 
       [0008]    In an effort to aid diagnosis in schools and medical provider offices, the invention provides a new approach for identification of executive functioning deficits using quantitative data of subjects&#39; actual behavior, eliminating human-made subjective observation. The data are collected in a game (“Groundskeeper”; CogCubed Inc., Minneapolis, Minn.) that is designed to stimulate and entertain the subject. 
         [0009]    “Groundskeeper” engages participants through a tangible platform with interesting images, sounds, and challenges to motivate the child. The inventors have developed models to predict executive functioning disorders, with data collected from test subjects&#39; gameplay using a machine learning approach. Machine learning is a branch of artificial intelligence that focuses on developing and utilizing algorithms to process data that automatically identifies patterns in data that model latent real-world relationships using multiple techniques, including supervised learning. 
         [0010]    Such game-based objective measurement approaches can bear several important advantages, such as 
         [0011]    (1) they are highly engaging, allowing investigators to consider potentially confounding effects of boredom, 
         [0012]    (2) they are objective, 
         [0013]    (3) they provide more dimensions by which behavior data can be captured (see below), 
         [0014]    (4) they provide more ecologically valid assessment conditions, and 
         [0015]    (5) they are limited to a finite period of time and are not staff-intensive to administer, making it more accessible. 
         [0016]    Such assessment technologies will play important future role as diagnostic aids by improving psychiatric diagnostic fidelity and making diagnostic aids more accessible for earlier identification and treatment. To this end, the present invention employs the machine learning-based clinical assessment protocol as a diagnostic aid for ADHD. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0017]      FIG. 1  illustrates a plurality of tangible-graphical interactive cubes. 
           [0018]      FIG. 2  is a three-dimensional graph illustrating the feature variables measured by different sessions. 
           [0019]      FIG. 3  is a flow diagram illustrating the machine learning feedback algorithm. 
           [0020]      FIG. 4  is a flow diagram illustrating a method of evaluating quality of the game platform method in variables chosen. 
           [0021]      FIG. 5  is a flow diagram comparing the accuracy of the game platform method versus a pre-existing method. 
       
    
    
     DETAILED DESCRIPTION 
       [0022]    Subjects were children and adolescents between 6 and 17 years of age (n=52) as indicated in Table 1. They were recruited from two outpatient clinics. Twenty-six subjects were included in the study who met diagnostic criteria for ADHD, along with 26 age- and sex-matched comparison subjects without ADHD (Table 1). The study participants were administered the Schedule for Affective Disorders and years of age (n=52) as indicated in Table 1. They were recruited from two outpatient clinics. Twenty-six subjects were included in the study who met diagnostic criteria for ADHD, along with 26 age- and sex-matched comparison subjects without ADHD (Table 1). The study participants were administered the Schedule for Affective Disorders and Schizophrenia for School-Age Children—Present and Lifetime Version, a semistructured diagnostic interview conducted by a psychiatric nurse trained by a child/adolescent psychiatrist at a community-based clinic (j=1.0) and reviewed by two independent child/adolescent psychiatrists. Participants were eligible to participate if they met criteria for ADHD, depressive disorder, anxiety disorders, oppositional defiant disorder, panic disorder, or eating disorders. Across groups, subjects were excluded if they had a history of psychosis, IQ of 55 and below, chemical use disorders, conduct disorder, tic disorders, or any physical ailment that would impede gameplay with the Sifteo Cubes (Sifteo, Inc., San Francisco, Calif.) game system (small tangible computers). Stimulant medications for ADHD were withheld on the day of testing. 
         [0023]    The procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008. 
         [0000]    
       
         
               
             
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 1 
               
             
             
               
                   
               
               
                 DEMOGRAPHIC CHARACTERISTICS 
               
               
                 OF STUDY SUBJECTS 
               
             
          
           
               
                   
                   
                 Number (percentage of total) 
               
             
          
           
               
                   
                   
                 ADHD 
                 No ADHD 
               
               
                   
                 Characteristic 
                 (n = 26) 
                 (n = 26) 
               
               
                   
                   
               
               
                   
                 Age (years) (mean)  
                 12.6 
                 14.7 
               
               
                   
                 Gender 
                   
                   
               
               
                   
                 Male 
                 16 (62) 
                 10 (38) 
               
               
                   
                 Female 
                 10 (38) 
                 16 (62) 
               
               
                   
                 Race 
                   
                   
               
               
                   
                 White 
                 17 (65) 
                 23 (88) 
               
               
                   
                 Black/African American 
                  6 (23) 
                 2 (8) 
               
               
                   
                 Hispanic 
                 1 (4) 
                 0 (0) 
               
               
                   
                 Other 
                 2 (8) 
                 1 (4) 
               
               
                   
                 Diagnoses 
                   
                   
               
               
                   
                 ADHD with inattentive 
                  26 (100) 
                   
               
               
                   
                 symptoms 
                   
                   
               
               
                   
                 ADHD with both inattentive 
                 17 (65) 
                   
               
               
                   
                 and hyperactive symptoms 
                   
                   
               
               
                   
                 Comorbidities 
                   
                   
               
               
                   
                 Depressive disorders 
                 9 (35) 
                 16 (62) 
               
               
                   
                 Anxiety disorders 
                 18 (69)  
                 24 (92) 
               
               
                   
                 Autism spectrum disorders 
                 9 (35) 
                  4 (15) 
               
               
                   
                 Disruptive behavior disorders 
                 10 (38)  
                  3 (12) 
               
               
                   
                   
               
               
                   
                 ADHD, attention deficit hyperactivity disorder. 
               
             
          
         
       
     
       The Game 
       [0024]    The invention employs game called “Groundskeeper,” using the Sifteo Cubes gaming platform, for aiding in the diagnosis of ADHD. The Sifteo Cubes are based on the principle of tangible user interfaces. They have screens on one side and are used through motion (shaking, tilting, pressing, etc.). The Cubes communicate wirelessly and can be synced to a computer that runs the game and collects data. As shown in  FIG. 1 , “Groundskeeper” is played using the Sifteo Cubes  10  and a placement board (not shown). One Cube  12  is always used as a “mallet” to hit game targets. Initially, three Cubes are placed in a straight vertical line. Each of these three Cubes has an image of green grass and the blue sky as a backdrop. Images of a rabbit, a groundskeeper (a man with a lawnmower), a gopher, or a few small birds appear on each screen for 1, 1.5, or 3 seconds at random. The object of the game is to avoid distracters and wait for only an image of a gopher to appear  14 , at which time the player should touch the mallet Cube  12  to the gopher Cube  14 , which marks a successful hit with a “bonk” noise. Each of the 17 game sessions is 90 seconds long, with a 20-second interval in between each session. 
       Procedures 
       [0025]    All participating subjects were requested to come to one of the test sites for an initial evaluation. They were also asked to play the game “Groundskeeper.” Parents of all subjects, including the comparison population, were requested to complete a Conners&#39; Brief Rating Scale, Parent Version. A Teacher Version was supplied to the parent to be completed by a schoolteacher and returned in a pre-addressed envelope. Previous scores from the Conners&#39; CPT were requested from parents if it was previously done for participating subjects. Both the Conners&#39; Brief Rating Scale and the Conners&#39; CPT are highly validated tools commonly used as a research and clinical tool to assess child behavior. The game data were automatically uploaded to a cloud server, and the de-identified data were submitted to the University of Minnesota for analysis. 
         [0026]      FIG. 2  and Table 1.5 shows the design of the overall evaluation protocol, which consists of 17 game sessions, numbered 0 through 16, each with different levels and types of distractions: visual, audio, and spatial, represented by the three coordinate axes. Low visual distraction consists of a bird appearing on the cube screens; high visual distraction adds large rabbits. Low audio distraction consists of occasional tweeting noises; high audio distraction increases tweet frequency. When there is no spatial distraction, the image cubes are in a vertical line. In low spatial distraction, they are set diagonally 2 inches apart as shown in  FIG. 1 ; high spatial distraction consists of each cube put 3 inches apart. Sessions 0, 1, and 16 have no distraction. Each session is 90 seconds long, consisting of randomized number of trials and frequency of target stimulus presentation. Session 0 for practice and is not used for analysis. Session 1 is used to measure the subject&#39;s initial ability and focus level, and Session 16 is a control at the end of gameplay in order to measure learning and endurance 
         [0000]    
       
         
               
             
               
             
           
               
                 TABLE 1.5 
               
               
                   
               
               
                 Game Levels 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 Session 0: Screen shot of gopher, groundskeeper or grass (neutral) 
               
               
                 presented for 1, 1.5 or 3 seconds at a random frequency. 
               
               
                 This is for instructional purposes only for the patient. This 
               
               
                 should not be used for analysis. 
               
               
                 Session 1: Screen shot of gopher, groundskeeper or grass (neutral) 
               
               
                 presented for 1, 1.5 or 3 seconds at a random frequency. 
               
               
                 Go/no go task 
               
               
                 Session 2: Gopher, groundskeeper or grass screenshot with visual 
               
               
                 disturbance at a low degree (one bird showing up on screen) alternating 
               
               
                 with screenshot of grass presented for 1, 1.5 or 3 seconds at random 
               
               
                 frequency. 
               
               
                 Visual disturbance, low frequency 
               
               
                 Session 3: Gopher, groundskeeper or grass screenshot with visual 
               
               
                 disturbance at a high degree (bird and rabbit) alternating with 
               
               
                 screenshot of grass presented for 1, 1.5 or 3 seconds at random 
               
               
                 frequency. 
               
               
                 Visual disturbance, high frequency 
               
               
                 Session 4: Screen shot of gopher, groundskeeper or grass screenshot 
               
               
                 presented for 1, 1.5 or 3 seconds at a random frequency with auditory 
               
               
                 disturbance at a low degree (one bird chirping) occurring at random 
               
               
                 frequencies for 1, 1.5 or 3 seconds, not in concert with screen shot 
               
               
                 frequency. 
               
               
                 Auditory disturbance, low frequency 
               
               
                 Session 5: Screen shot of gopher, groundskeeper or grass screenshot 
               
               
                 presented for 1, 1.5 or 3 seconds at a random frequency with auditory 
               
               
                 disturbance at a high degree (multiple birds chirping) occurring at 
               
               
                 random frequencies for 1, 1.5 or 3 seconds, not in concert with screen 
               
               
                 shot frequency. 
               
               
                 Auditory disturbance, high frequency 
               
               
                 Session 6: Gopher, groundskeeper or grass screenshot with visual and 
               
               
                 auditory disturbances at a low degree (one bird chirping) occurring at 
               
               
                 random frequencies for 1, 1.5 or 3 seconds 
               
               
                 Visual and auditory disturbance, low frequency 
               
               
                 Session 7: Gopher, groundskeeper or grass screenshot with visual and 
               
               
                 auditory disturbances at a high degree (bird and rabbit and chirping) 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Visual and auditory disturbance, high frequency 
               
               
                 Session 8: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a low degree. Cube set diagonally. Spaced at 2 inches 
               
               
                 apart and occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, low frequency 
               
               
                 Session 9: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a low degree combined with low frequency visual 
               
               
                 disturbance. Cube set diagonally. Spaced at 2 inches apart and 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, low frequency, low frequency visual 
               
               
                 disturbance 
               
               
                 Session 10: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a low degree combined with low frequency auditory 
               
               
                 disturbance. Cube set diagonally. Spaced at 2 inches apart and 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, low frequency, low frequency auditory 
               
               
                 disturbance 
               
               
                 Session 11: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a low degree combined with low frequency visual and 
               
               
                 auditory disturbance. Cube set diagonally. Spaced at 2 inches apart and 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, low frequency, low frequency visual and 
               
               
                 auditory disturbance 
               
               
                 Session 12: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a low degree. Cube set diagonally. Spaced at 3 inches 
               
               
                 apart and occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, high frequency 
               
               
                 Session 13: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a high degree combined with low frequency visual 
               
               
                 disturbance. Cube set diagonally. Spaced at 3 inches apart and 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, high frequency, low frequency visual 
               
               
                 disturbance 
               
               
                 Session 14: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a high degree combined with low frequency auditor 
               
               
                 disturbance. Cube set diagonally. Spaced at 3 inches apart and 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, high frequency, low frequency auditory 
               
               
                 disturbance 
               
               
                 Session 15: Gopher, groundskeeper or grass screenshot with spatial 
               
               
                 disturbance at a high degree combined with low frequency visual and 
               
               
                 auditory disturbance. Cube set diagonally. Spaced at 3 inches apart and 
               
               
                 occurring at random frequencies for 1, 1.5 or 3 seconds. 
               
               
                 Spatial disturbance, high frequency, low frequency visual and 
               
               
                 auditory disturbance 
               
               
                 Session 16: Screen shot of gopher, groundskeeper or grass (neutral) 
               
               
                 presented for 1, 1.5 or 3 seconds at a random frequency. 
               
               
                 Go/no go task with learning curve. 
               
               
                   
               
             
          
         
       
     
         [0027]    In order to isolate different stages of learning, several data aggregations were for specific sessions rather than the overall. Sessions 2 through 7 measured initial learning while including less distraction. Sessions 8 through 11 defined the variable to be considering only the low spatial distraction sections, as well as the late-intermediate portion of their learning. The final stage of the learning curve included sessions 12 through 15, when the most distraction (audio, visual, and high spatial) were being employed and subjects had the most experience with the game. By restricting summations to specific portions of the game, a subject&#39;s progression in learning as well as distraction and boredom was taken into account. The difference between various types of distraction was also stressed through these sectioned aggregations in the same way that the accuracy ratio variables compared values from different sections. 
         [0028]    The creation of feature variables involved a great deal of trial and error. Initially, design of variables compared across the entire game. As the goals became more refined, new sets of variables were created that were specific for each section. Ultimately, the variables for session-specific tasks were chosen. While creating the comparative rates, percentage values were found to be the most accurate and efficient for the algorithms that were being utilized. These rates were most useful for accuracy measures, especially when comparing different types of distraction by creating ratios out of various session accuracies. For other measurements, total values were used rather than rates. This system was more accurate for behavior such as tilt or response time as the goal was to determine how the patient responded to the environment in general instead of comparing measurements. Table 2 is a table of the variables chosen. 
         [0000]    
       
         
               
             
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 2 
               
             
             
               
                   
               
               
                 Data Space 
               
             
          
           
               
                 ID 
                 Variable 
                 Description 
                 Formula and/or Origin 
               
               
                   
               
               
                 1 
                 AccountId 
                 Unique owner/account id 
                 Atomic data from devices. 
               
               
                   
                   
                 mapping to the device 
                   
               
               
                   
                   
                 registration. 
                   
               
               
                 2 
                 PlayerId 
                 Unique player id mapping 
                 Atomic data from device. 
               
               
                   
                   
                 to the player of the game. 
                   
               
               
                 3 
                 GameId 
                 Random id to differentiate 
                 Atomic data from device. 
               
               
                   
                   
                 game played. 
                   
               
               
                 4 
                 Event 
                 Type of activity occurring 
                 Game logic. 
               
               
                   
                   
                 in game (hit, tilt, stimulus, 
                   
               
               
                   
                   
                 distracter, etc.). 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 5 
                 DeviceId 
                 Identifier of device 
                 Atomic data from device. 
               
               
                   
                   
                 exhibiting event 
                   
               
               
                 6 
                 ImageId 
                 Identifier for image 
                 Game logic. 
               
               
                   
                   
                 displayed. Configurable. 
                   
               
               
                 7 
                 NewImageId 
                 Boolean 1 or 0 variable 
                 Game logic. 
               
               
                   
                   
                 indicating a new image is 
                   
               
               
                   
                   
                 displayed or not. 
                   
               
               
                   
                   
                 NewImageId = 1 when new 
                   
               
               
                   
                   
                 image is displayed. 
                   
               
               
                   
                   
                 Decision to display new 
                   
               
               
                   
                   
                 image can be random or 
                   
               
               
                   
                   
                 explicit in the game. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 8 
                 SessionLevel 
                 Indicator for level in game. 
                 Game logic. 
               
               
                   
                   
                 Session = Level. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 9 
                 TiltFlip 
                 Boolean 1 or 0. TiltFlip 1   
                 Atomic data from device 
               
               
                   
                   
                 when device is flipped over, 
                 accelerometer. 
               
               
                   
                   
                 else flipped back TiltFlip 0 . 
                   
               
               
                 10 
                 TiltX 
                 Total movement of device 
                 TiltX = TiltLeft + TiltRight 
               
               
                   
                   
                 in X direction. 
                 Player decision involving 
               
               
                   
                   
                   
                 game logic. 
               
               
                 11 
                 TiltY 
                 Total movement of device 
                 TiltY = TiltUp + TiltDown 
               
               
                   
                   
                 in Y direction. Atomic data 
                 Player decision involving 
               
               
                   
                   
                 from device accelerometer. 
                 game logic. 
               
               
                   
               
               
                 12 
                 TiltZ 
                 Total movement of device in Z direction. Atomic data from device accelerometer. 
                           Σ     m   &lt;   TiltZ   &lt;   n                       f        (   TiltZ   )           
   m = TiltFlip 0 , n = TiltFlip 1  Player decision involving game logic. 
               
               
                   
               
               
                 13 
                 Movement 
                 Total movement in all 
                 Movement = TiltX + TiltY + 
               
               
                   
                   
                 directions. Aggregate iof 
                 TiltZ 
               
               
                   
                   
                 atomic data from device 
                 Player decision involving 
               
               
                   
                   
                 accelerometer. 
                 game logic. 
               
               
                 14 
                 NeighborEvent 
                 Boolean 1 or 0 variable 
                 Player decision involving 
               
               
                   
                   
                 indicating cubes moved 
                 game logic. 
               
               
                   
                   
                 together from neighboring 
                   
               
               
                   
                   
                 event. NeighborEvent 1  or 
                   
               
               
                   
                   
                 ON when cubes 
                   
               
               
                   
                   
                 neighbored, else OFF and 
                   
               
               
                   
                   
                 NeighborEvent 0 . 
                   
               
               
                 15 
                 TiltLeft 
                 Tilt or device left. 
                 TiltLeft ε TiltX 
               
               
                   
                   
                   
                 Atomic data from device 
               
               
                   
                   
                   
                 accelerometer. 
               
               
                 16 
                 TiltRight 
                 Tilt of device right. 
                 TiltRight ε TiltX 
               
               
                   
                   
                   
                 Atomic data from device 
               
               
                   
                   
                   
                 accelerometer. 
               
               
                 17 
                 TiltUp 
                 Tilt of device upwards. 
                 TiltUp ε TiltY 
               
               
                   
                   
                   
                 Atomic data from device 
               
               
                   
                   
                   
                 accelerometer. 
               
               
                 18 
                 TiltDown 
                 Tilt of device downwards. 
                 TiltDown ε TiltY 
               
               
                   
                   
                   
                 Atomic data from device 
               
               
                   
                   
                   
                 accelerometer. 
               
               
                 19 
                 TiltMiddle 
                 Tilt device return to 
                 Atomic data from device 
               
               
                   
                   
                 middle. 
                 accelerometer. 
               
               
                 20 
                 VirtualTicks 
                 Measureable time unit 
                 Game logic. 
               
               
                   
                   
                 recorded as ticks which 
                   
               
               
                   
                   
                 operate within the state 
                   
               
               
                   
                   
                 machine. Configurable 
                   
               
               
                   
                   
                 measure. 
                   
               
               
                 21 
                 Correct 
                 Number of correct 
                 Player decision involving 
               
               
                   
                   
                 responses, as deemed by 
                 game logic. 
               
               
                   
                   
                 rules in the game, based on 
                   
               
               
                   
                   
                 responses by a player. For 
                   
               
               
                   
                   
                 example, game dedines 
                   
               
               
                   
                   
                 correct response as 
                   
               
               
                   
                   
                 NeighborEvent 1  while only 
                   
               
               
                   
                   
                 defined correct image is 
                   
               
               
                   
                   
                 displayed. 
                   
               
               
                 22 
                 Incorrect 
                 Number of incorrect 
                 Player decision involving 
               
               
                   
                   
                 responses game, as deemed 
                 game logic. 
               
               
                   
                   
                 by rules in the game, based 
                   
               
               
                   
                   
                 on responses by a player in 
                   
               
               
                   
                   
                 a game. For example, game 
                   
               
               
                   
                   
                 defines incorrect response 
                   
               
               
                   
                   
                 as NeighborEvent 1  while 
                   
               
               
                   
                   
                 defined correct image is 
                   
               
               
                   
                   
                 NOT displayed. Incorrect = 
                   
               
               
                   
                   
                 Commissions. 
                   
               
               
                 23 
                 Response 
                 Indicator of player decision 
                 Response = 
               
               
                   
                   
                   
                 NeighborEvent 1  + 
               
               
                   
                   
                   
                 NeighborEvent 0   
               
               
                 24 
                 ImageDraws 
                 Number of images 
                 Game logic. 
               
               
                   
                   
                 displayed during the game. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 25 
                 CorrectDisplay 
                 Number of correct images 
                 Game logic. 
               
               
                   
                   
                 displayed during the game, 
                   
               
               
                   
                   
                 as deemed correct by rules 
                   
               
               
                   
                   
                 within the game. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 26 
                 ImageDisplayLength 
                 Measurement of time 
                 Game logic. 
               
               
                   
                   
                 image is displayed as 
                   
               
               
                   
                   
                 defined by the number of 
                   
               
               
                   
                   
                 VirtualTicks in image is 
                   
               
               
                   
                   
                 assigned display on device. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 27 
                 BirdRabbitDisLength 
                 Display length of special 
                 Game logic. 
               
               
                   
                   
                 distracter like bird or 
                   
               
               
                   
                   
                 rabbit. Configurable. 
                   
               
               
                 28 
                 DateTime 
                 Actual date and time 
                 Atomic data from device 
               
               
                   
                   
                 stamp of game played. 
                 time. 
               
               
                   
               
             
          
           
               
                 29 
                 DoubleHit 
                 Count for double or multiple neighboring responses of device after primary until next image displayed. 
                           ∑     i   =   m     n                     x   i         
   m = Response 2 , n = NewImageId Player decision involving game logic. 
               
               
                   
               
               
                 30 
                 FrameElapse 
                 Actual state machine time 
                 Atomic data from device 
               
               
                   
                   
                 in ticks elapsed for level 
                 internal clock. 
               
               
                   
                   
                 and/or game. 
                   
               
               
                 31 
                 TimeMovement 
                 Amount of movement over 
                 Movement/VirtualTicks 
               
               
                   
                   
                 virtual time period 
                 Player decision involving 
               
               
                   
                   
                   
                 game logic. 
               
               
                 31 
                 BirdCounter 
                 Number of bird 
                 Game logic. 
               
               
                   
                   
                 distractions displayed. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 32 
                 RabbitCounter 
                 Number of rabbit 
                 Game logic. 
               
               
                   
                   
                 distractions displayed. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 33 
                 BirdSoundCounter 
                 Number of special sound 
                 Game logic. 
               
               
                   
                   
                 distractions played. 
                   
               
               
                   
                   
                 Configurable. 
                   
               
               
                 34 
                 TopBotomHit 
                 Measured by response to 
                 Player decision involving 
               
               
                   
                   
                 top or bottom of device. 
                 game logic. 
               
               
                   
                   
                 Count for device responses 
                   
               
               
                   
                   
                 or hits from top or bottom. 
                   
               
               
                 35 
                 CubedPressed 
                 Number of physical 
                 Atomic data from device 
               
               
                   
                   
                 touches or presses to 
                 accelerometer. 
               
               
                   
                   
                 device surface. 
                   
               
               
                   
               
               
                 37 
                 ResponseCounter 
                 Measurement of time between image displays, acts as control for response correct/incorrect. 
                           Σ     m   &lt;   VirtualTicks   &lt;   n                       f        (   VirtualTicks   )           
   m = Response x  n = NewImageId Player decision involving game logic. 
               
               
                   
               
               
                 38 
                 CubeLocation 
                 Geometric coordinates of 
                 Player decision involving 
               
               
                   
                   
                 the device physical location 
                 game logic. 
               
               
                   
                   
                 based on atomic game 
                   
               
               
                   
                   
                 data. 
                   
               
               
                 39 
                 DeviceDistance 
                 Fixed distance in inches 
                 Game logic. 
               
               
                   
                   
                 devices may be placed 
                   
               
               
                   
                   
                 apart. This varies by type 
                   
               
               
                   
                   
                 of game and will vary by 
                   
               
               
                   
                   
                 session as part of setup. 
               
               
                   
               
             
          
         
       
     
         [0029]    Correct responses and incorrect responses are recorded for accuracy measures. Omissions (lack of response to a gopher image) and commissions (responding to an incorrect image) are also counted. Other behavioral responses are also observed, such as DoubleHit, or hitting the cube twice quickly. This specific type of commission error tends to arise when a person thinks they have not adequately hit the gopher and is an indication of hyperactivity. ResponseCounter measures how long two cubes are pressed together, from the time of contact to the time of release. This aids in detecting inattentiveness and hyperactivity even in a combined patient whose overall timing may be similar to that of a normal subject. Patients with a diagnosis of ADHD-combined type tend to start slow, but hit the cubes together very quickly. Finally, ResponseCorrect and ResponseIncorrect measure the time from which an image is displayed until a response is hit, with correct and incorrect responses noted respectively. 
         [0030]    Measures of distractibility included mCRR (motor correct response rate) and mIRR (motor incorrect response rate). These measure the number of correct or incorrect responses by the total responses respectively. Further accuracy measures included mEOM (omissions divided by correct responses—an indicator of inattention). These divide the mIRR to clarify if patients have excessive motion and stimulation as opposed to lethargy and lack of stimulation. Omissions and commissions were also placed as raw numbers to provide a more direct comparison between trial identifiers. DoubleHit was calculated as a raw number of the times a person hurriedly struck the gopher twice, an indicator of impulsivity and scattered, fast behavior. 
         [0031]    Further assessment was then conducted on performance between sessions. Comparisons between visual and audio distractors and subsequent accuracy allowed evaluation of mCRRs and mIRRs at different intensities of distraction. A higher level of failure appeared to indicate that external distractors had a more profound impact on the subject. Comparison of the success and failure rates for different combinations of distractors and intensities allowed investigators to determine if audio or visual distractors were more problematic for the subject. This was determined by comparing the mCRR of a session containing only one type of distraction to a session with both. The intensity of distraction remained constant in each comparison, resulting in the variables instead of two. Spatial accuracy was then compared between sessions containing short distances between the cubes against those with increased distance. While other factors remained constant, the mCRRs of the spatial distractor sessions were assessed. For example, Session 6 (low audio, low visual) was compared to Session 11 (low audio, low visual, low spatial) by making a ratio of their comparative mCRRs. Spatial success/failure ratio was created to compare varying degrees of spatial distraction, comparing the mCRR of low distraction (Session 8) to that of high distraction (Session 12). The complete spatial success/failure ratio utilized the same approach, but involved audio and visual distractors as constant factors. 
         [0032]    Finally, LearningCurve was used to measure learning. The percentage of improvement was calculated between Session 16 and Session 1 to determine if the subject had improved, thus learning, or had gotten worse, implying boredom or distraction. 
         [0033]    In-game behavior features were more extensively investigated. TiltX, TiltY, and TiltZ were aggregations of degrees of tilt throughout each session. The greater the degree of tilt, the more indicative of hyperactive behavior, especially in the case of TiltZ which is the most extreme motion. OffTiltReaction and OnTiltReaction were summed to specify how much the person moved the cube around before and after making a response. More movement implied more distraction surrounding a response, whereas less movement could indicate either more focus or simply less activity. CorrectReaction and IncorrectReaction were summed to determine slower versus quicker speeds—less speed implying inattention. As these two measures would be abnormally slow for inattentive patients while remaining comparatively equal for ADHD-combined type and non-ADHD patients, NeighborReaction was summed to isolate the amount of time that two cubes were pressed together. As stated previously, an ADHD-combined type patient would start slowly and, once they got closer, press the cubes together and let go quickly, becoming more engaged and impulsive as the stimulation set in. However, a non-ADHD subject would be more balanced in their time distribution. Thus, NeighborReaction differentiates reaction time between ADHD-combined type and non-ADHD subjects. 
         [0034]    Some variables were used as extraneous distraction collectors—movements that should not take place during gameplay that could signify hyperactive behavior. These included TopBottomHit, a sum of all hits to the tops or bottoms of cubes, and CubePressed, a count of how many times the screen was pressed in. 
         [0035]    In order to isolate different stages of learning, several data aggregations were for specific sessions rather than the overall. Sessions 2 through 7 measured initial learning while including less distraction. Sessions 8 through 11 defined the variable to be considering only the low spatial distraction sections, as well as the late-intermediate portion of their learning. The final stage of the learning curve included sessions 12 through 15, when the most distraction (audio, visual, and high spatial) were being employed and subjects had the most experience with the game. By restricting summations to specific portions of the game, a subject&#39;s progression in learning as well as distraction and boredom was taken into account. The difference between various types of distraction was also stressed through these sectioned aggregations in the same way that the accuracy ratio variables compared values from different sections. 
       Data Analysis Procedures 
       [0036]    The game collects data samples of 33 variables (Table 2, IDs 4-37) at 10 Hz. Over 17 sessions of 90 seconds each, this translates to 15,300 samples per participant. This is a rich and granular dataset of gameplay behavior, from which detailed diagnostic models can be built. In the experiment, data from 52 participants were used to build predictive models for hyperactivity, inattentiveness, and control. 
       Machine Learning-Based Approach to ADHD Diagnosis 
       [0037]    The data from the game were mathematically transformed into feature variables corresponding to specific neurological correlates designated by the child/adolescent psychiatrists. After feature creation, a principal component analysis was applied to generate the most useful features. The transformed data were analyzed using Waikato Environment for Knowledge Analysis, a suite of open sthece machine learning software. 
         [0038]    Once a highly accurate algorithm with successful feature variables was created, the parameters became the diagnostic tool ( FIG. 3 ). 
         [0039]    Feature Space for ADHD Diagnosis 
         [0040]    Success of data-driven model building is highly dependent on the feature space used for the analysis, which can differ from the variables collected. Using domain knowledge to transform the data variables collected into a feature space suitable for effective model building is one of the key innovations of this study. Along with traditional game elements and events acting as identifiers within the game, each data instance contained several data variables. There are various tilt variables that measure different aspects of movement, game-environment variables that count images, time measurements of responses and actions, extraneous action variables that measure cube pressing, hits that shouldn&#39;t occur, and accuracy variables. 
         [0041]    It was found that totals and rates were useful for feature variable creation. Because each session is different in order to avoid predictability, rates were used to normalize subject results. Aggregations of in-game data were also used to highlight differences that would be missed in sequential data. Accuracy measures were made to identify patterns that would be distinctive of the three possible cases. It was predicted that normal subjects have the highest accuracy, inattentive patients have the highest omission rate (lack of a response to stimuli), and combined patients have the highest commission rate (incorrect/extraneous hits). Comparisons between different sessions were included in order to measure effects of different types/intensities of distraction on accuracy. Learning was measured for both diagnosis and potential treatment purposes through a variable called LearningCurve, calculated by the percentage improvement between Session 16 and Session 1, illustrating improvement and thus learning, or regression, implying boredom or distraction. 
         [0042]    In-game behavior features included aggregations of degrees of tilt throughout each session; high tilt is representative of hyperactive behavior. Reaction timings were summed to differentiate between speeds, with slower implying inattention. Additional measures were included to differentiate among ADHD, combined type, and normal subjects who demonstrated similar reaction times but varying patterns of behavior when reacting to visual stimuli. Some variables were extraneous distraction collectors—movements that were uncommon could imply hyperactive behavior. Several of the aggregations were through specific sets of sessions rather than the overall game in order to isolate different stages of a person&#39;s learning as he or she progressed in the game, from which intermediate models were created. Principal component analysis was then applied to reduce dimensionality and to determine the most distinctive feature variables for the model. 
         [0043]    The classification algorithms were tested during model creation: AdaBoost, JRip, J48, and RandomForest. AdaBoost is a meta-algorithm, meaning that it creates solutions and progressively improves them, JRip is a rule-making algorithm, and J48 and RandomForest are decision trees that create parameters and split the data based on them. 
         [0044]    The below Table 2.5 summarizes the feature space mathematical transformations: 
         [0000]    
       
         
               
             
               
               
               
               
             
           
               
                 TABLE 2.5 
               
             
             
               
                   
               
               
                 Feature Space, mathematical transformations of data space 
               
             
          
           
               
                 ID 
                 Variable 
                 Description 
                 Formula 
               
               
                   
               
               
                 1 
                 OffTiltReaction 
                 Off response tilt counter. Counter increments on each tilt from point of device neighbor removal (after neighboring) until next image displayed 
                           Σ     m   &lt;   TimeMovement   &lt;   n                       f        (   TimeMovement   )           
   m = Response x  n = NewImageId 
               
               
                   
               
               
                 2 
                 OnTiltReaction 
                 On response tilt counter. Counter increments on each tilt from point view image is displayed until devices are neighbored 
                           Σ     m   &lt;   TimeMovement   &lt;   n                       f        (   TimeMovement   )           
   n = Response x  m = NewImageId 
               
               
                   
               
               
                 3 
                 ResponseCorrect 
                 Measurement of time when an image is displayed until correct response is registered. Delta between response counter measures response time. 
                           Σ     m   &lt;   VirtualTicks   &lt;   n                       f        (   VirtualTicks   )           
   m = Response correct  n = NewImageId 
               
               
                   
               
               
                 4 
                 ResponseIncorrect 
                 Measurement of time when image is displayed until incorrect response is registered. Delta between response counter measures response time. 
                           Σ     m   &lt;   VirtualTicks   &lt;   n                       f        (   VirtualTicks   )           
   m = Response incorrect  n = NewImageId 
               
               
                   
               
               
                 5 
                 NeighborReaction 
                 Measurement of time from when two devices are put together (neighbored) until the movement they are removed. 
                           Σ     m   &lt;   VirtualTicks   &lt;   n                       f        (   VirtualTicks   )           
   m = NeighborEvent1 n = NeighborEvent0 
               
               
                   
               
               
                 6 
                 CorrectReaction 
                 Measurement of time from when an image is displayed until a correct response. 
                           Σ     m   &lt;   VirtualTicks   &lt;   n                       f        (   VirtualTicks   )           
   n = Response correct  m = NewImageId 
               
               
                   
               
               
                 7 
                 IncorrectReaction 
                 Measurement of time from when an image is displayed until an incorrect response. 
                           Σ     m   &lt;   VirtualTicks   &lt;   n                       f        (   VirtualTicks   )           
   n = Response incorrect  m = NewImageId 
               
               
                   
               
               
                 8 
                 ErrorOmission 
                 Number of times the 
                 ErrorOmission = CorrectDisplay − Correct 
               
               
                   
                   
                 correct stimulus is 
                   
               
               
                   
                   
                 presented and there is no 
                   
               
               
                   
                   
                 response. 
                   
               
               
                 9 
                 mCRR 
                 Ratio of correct to incorrect 
                 Correct/(Incorrect + Correct) 
               
               
                   
                   
                 and correct 
                   
               
               
                 10 
                 mIRR 
                 Ratio of incorrect to 
                 Incorrect/(Incorrect + Correct) 
               
               
                   
                   
                 incorrect and correct 
                   
               
               
                 11 
                 mEOM 
                 Ratio of omission to 
                 (ErrorOmission)/(Incorrect + Correct) 
               
               
                   
                   
                 incorrect and correct 
                   
               
               
                   
               
               
                 12 
                 mMMO 
                 Mallet movement only (multiple for various levels of game) 
                           ∑     i   =   m     n                     x   i         
   m = mMMO ε Movement 
               
               
                   
               
               
                 13 
                 mAOM 
                 Sum of all movement excluding mallet cube (multiple for various levels of game) 
                           ∑     i   =   m     n                     x   i         
   m = mAOM ε Movement 
               
               
                   
               
               
                 14 
                 mCT 
                 Total maximum tilt 
                 TiltX + TiltY + TiltZ 
               
               
                   
                   
                 measurements (multiple 
                   
               
               
                   
                   
                 for various levels of game) 
                   
               
               
                 15 
                 LearningCurve 
                 Learning Curve 
                 ((mCRR16/mCRR1) − 1) * 100 
               
               
                 16 
                 LearningCurve_O 
                 Learning Curve of 
                 (((mCRR16/mCRR1)/(mEOM16/ 
               
               
                   
                   
                 Omissions 
                 mEOM1)) − 1) * 100) 
               
               
                 17 
                 LearningCurve_NR 
                 Learning Curve Neighbor 
                 (((mNR16/mNR1) − 1) * 100) 
               
               
                   
                   
                 Reaction 
                   
               
               
                 18 
                 LearningCurve_CR 
                 Learning Curve of Correct 
                 (((mCR16/mCR1) − 1) * 100) 
               
               
                   
                   
                 Reaction 
                   
               
               
                 19 
                 LearningCurve_CT 
                 Learning Curve of Cube 
                 (((mCT16/mCT1) − 1) * 100) 
               
               
                   
                   
                 Tilts 
                   
               
               
                 20 
                 LearningCurve_ICR 
                 Learning Curve of 
                 (((mICR16/mICR1) − 1) * 100) 
               
               
                   
                   
                 Incorrect Responses 
                   
               
               
                 21 
                 VSC-VFC 
                 Visual Success Comparison 
                 (mCRR2/mCRR3)/(mIRR2/mIRR3) 
               
               
                 22 
                 VSC_VFC_O 
                 Visual Omission 
                 (mCRR2/mCRR3)/(mEOM2/mEOM3) 
               
               
                   
                   
                 Comparison 
                   
               
               
                 23 
                 ASC_AFC 
                 Auditory Success 
                 ((mCRR4/mCRR5)/(mIRR4/mIRR5)) 
               
               
                   
                   
                 Comparison 
                   
               
               
                 24 
                 ASC_AFC_O 
                 Auditory Omission 
                 ((mCRR4/mCRR5)/(mEOM4/ 
               
               
                   
                   
                 Comparison 
                 mEOM5)) 
               
               
                 25 
                 CSSC 
                 Spatial Success 
                 mCRR1/mCRR15 
               
               
                   
                   
                 Comparison 
                   
               
               
                 26 
                 CSSC_O 
                 Spatial Omission 
                 mEOM11/mEOM15 
               
               
                   
                   
                 Comparison 
                   
               
               
                 27 
                 CSSC_IRR 
                 Complete Spatial Failure 
                 mIRR11/mIRR15 
               
               
                   
                   
                 Comparison 
                   
               
               
                 28 
                 SSC_SFC 
                 Spatial Success against 
                 ((mCRR6/mCRR11)/(mIRR6/ 
               
               
                   
                   
                 Incorrect to Low Spatial 
                 mIRR11)) 
               
               
                   
                   
                 and Visual and Auditory 
                   
               
               
                   
                   
                 Comparison 
                   
               
               
                 29 
                 SSC_SFC_O 
                 Spatial Success against 
                 (mCRR6/mCRR11)/(mEOM6/mEOM11) 
               
               
                   
                   
                 Omissions to Low Spatial 
                   
               
               
                   
                   
                 and Visual and Auditory 
                   
               
               
                   
                   
                 Comparison 
                   
               
               
                 30 
                 SSC_SFC_NR 
                 Spatial Neighbor Reaction 
                 (mNR6/mNR11) 
               
               
                   
                   
                 Comparison Low 
                   
               
               
                   
                   
                 Frequency 
                   
               
               
                 31 
                 SSC_SFC_CR 
                 Spatial Correction Reaction 
                 (mCR6/mCR11) 
               
               
                   
                   
                 Comparison Low 
                   
               
               
                   
                   
                 Frequency 
                   
               
               
                 32 
                 SSC_SFC_ICR 
                 Spatial Incorrect 
                 (mICR6/mICR11) 
               
               
                   
                   
                 Comparison Low 
                   
               
               
                   
                   
                 Frequency 
                   
               
               
                 33 
                 CVSC_CVFC 
                 Visual Success against 
                 (((mCRR2/mCRR6)/(mCRR3/ 
               
               
                   
                   
                 Incorrect of Visual Low to 
                 mCRR7))/((mIRR2/mIRR6)/(mIRR3/ 
               
               
                   
                   
                 Visual and Auditory Low 
                 mIRR7))) 
               
               
                   
                   
                 Compared to Visual High 
                   
               
               
                   
                   
                 and Visual and Auditory 
                   
               
               
                   
                   
                 High Compared with 
                   
               
               
                   
                   
                 Spatial Effects of the Same 
                   
               
               
                   
                   
                 Distractors 
                   
               
               
                 34 
                 CVSC_CVFC_O 
                 Visual Success against 
                 ((mCRR2/mCRR6)/(mCRR3/ 
               
               
                   
                   
                 Omissions of Visual Low to 
                 mCRR7))/((mEOM2/mEOM6)/ 
               
               
                   
                   
                 Visual and Auditory Low 
                 (mEOM3/mEOM7)) 
               
               
                   
                   
                 Compared to Visual High 
                   
               
               
                   
                   
                 and Visual and Auditory 
                   
               
               
                   
                   
                 High Compared with 
                   
               
               
                   
                   
                 Spatial Effects of the Same 
                   
               
               
                   
                   
                 Distractors 
                   
               
               
                 35 
                 CASC_CAFC 
                 Auditory Success against 
                 ((mCRR4/mCRR6)/(mCRR5/ 
               
               
                   
                   
                 Incorrect of Auditory Low 
                 mCRR7))/((mIRR4/mIRR6)/(mIRR5/ 
               
               
                   
                   
                 to Visual and Auditory Low 
                 mIRR7)) 
               
               
                   
                   
                 Compared to Auditory 
                   
               
               
                   
                   
                 High and Visual and 
                   
               
               
                   
                   
                 Auditory High Compared 
                   
               
               
                   
                   
                 with Spatial Effects of the 
                   
               
               
                   
                   
                 Same Distractors 
                   
               
               
                 36 
                 CASC_CAFC_O 
                 Auditory Success against 
                 ((mCRR4/mCRR6)/(mCRR5/ 
               
               
                   
                   
                 Omissions of Auditory Low 
                 mCRR7))/((mEOM4/mEOM6)/ 
               
               
                   
                   
                 to Visual and Auditory Low 
                 (mEOM5/mEOM7)) 
               
               
                   
                   
                 Compared to Auditory 
                   
               
               
                   
                   
                 High and Visual and 
                   
               
               
                   
                   
                 Auditory High Compared 
                   
               
               
                   
                   
                 with Spatial Effects of the 
                   
               
               
                   
                   
                 Same Distractors 
               
               
                   
               
             
          
         
       
     
         [0045]    Results 
         [0046]    Model Evaluation 
         [0047]    Two types of evaluations of the model have been performed, details of which are described in this section. The first, proposed by Fisher,23 uses a significance level to assess if the data collected from “Groundskeeper” play have sufficient information to be used for building a diagnosis model, a hypothesis. The second approach, the Neyman-Pearson approach, 23 was used to compare the performance of the proposed diagnosis model against that of existing models by Type I and Type II error calculations. In the following section the models are described evaluations in detail. 
         [0048]    P Value-Based Analysis 
         [0049]    Table 3 shows the results of the P value analysis for inattention and combined deficits of the 52 subjects. The expected values were the diagnoses provided by expert psychiatric evaluation using a semistructured interview. The observed values were the calculated diagnoses produced by the model. For comparison, P values for the CPT ranged from &lt;0.0001 to 0.88, depending on the variable analyzed. 
         [0000]                                                                TABLE 3                   P VALUE SCORES FOR INATTENTION       AND COMBINED DEFICITS                    Yes    No    Total                            Inattention a                         Observed   20   32   52           Expected   26   26   52               5.95   5.95                   35.4025   35.4025               χ 2  statistic   1.361635   1.361635   2.723269           Combined b                         Observed   10   42   52           Expected   17   35   52               6.95   6.95                   48.3025   48.3025               χ 2  statistic   2.841324   1.380071   4.221395                         a One degree of freedom, two-tailed P = 0.0989.             b One degree of freedom, two-tailed P = 0.0399.            
Effect size and accuracy of measurement vary by variable measured based on CPT testing (for example, commissions, omissions, reaction time, standard deviation of reaction time, etc.). For the Conners&#39; test, internal consistency coefficients for the total sample ranged from 0.77 to 0.97 (all correlations significant, P&lt;0.001). Inter-rater reliability coefficients ranged from 0.52 to 0.94. The P value scores obtained for the approach, in its early stage, is extremely promising. The results will improve as the approach is tested and fine-tuned over larger-scale studies. Type I and Type II error-based analysis In order to maximize accuracy, the model was split into two binary classifications: Presence/absence of hyperactivity and presence/absence of inattention. The hyperactive and inattentive components of ADHD, combined type, had a Yes indication for each in the dataset. An ADHD, inattentive type patient would have a Yes for inattention and a No for hyperactivity, and a normal/control patient would have No for both. Thus, the algorithms were able to take advantage of variables specific to one type of ADHD instead of dealing with results from the different classes of the disorder. The combined model, utilizing the F-measure (harmonic mean between precision and recall), ultimately classified the subjects with 75 percent accuracy. The inattention model had 78 percent accuracy. Frequently comorbid with ADHD, anxiety, depression, and autism spectrum disorders (ASD) also have deficits of executive functioning. With certain specifications of variables, the models had 71 percent accuracy for anxiety and 76 percent for depression. There was not a large enough sample to determine the model accuracy for diagnosing ASD ( FIG. 4  and Table 4).
 
         [0000]    
       
         
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
               
             
               
               
               
             
           
               
                 TABLE 4 
               
             
             
               
                   
               
               
                 DIAGNOSIS MODEL RESULTS 
               
             
          
           
               
                   
                 Predicted no 
                 Predicted yes 
               
               
                   
               
             
          
           
               
                 Model Inattention (n = 52, treatment and control population) a   
               
             
          
           
               
                 Actual no  
                 21 
                 5 
               
               
                 Actual yes 
                 6  
                 20 
               
             
          
           
               
                 Model Combined (n = 52, treatment and control population) b   
               
             
          
           
               
                 Actual no 
                 29  
                 6 
               
               
                 Actual yes 
                 7 
                 10 
               
             
          
           
               
                 CPT Inattention (n = 21, treatment and control population) c   
               
             
          
           
               
                 Actual no  
                 16 
                 1 
               
               
                 Actual yes 
                 1 
                 3 
               
             
          
           
               
                 CPT Combined (n = 33, treatment and control population) d   
               
             
          
           
               
                 Actual no 
                 19 
                 2 
               
               
                 Actual yes 
                 1 
                 11 
               
             
          
           
               
                 Conners&#39; Inattention 
               
               
                 Parent (n = 9 for treatment, n = 24 for control) e   
               
             
          
           
               
                 Actual no  
                 14 
                 10 
               
               
                 Actual yes  
                 3  
                 6 
               
             
          
           
               
                 Teacher (n = 23 for treatment, n = 24 for control) f   
               
             
          
           
               
                 Actual no 
                 20  
                 4 
               
               
                 Actual yes  
                 11 
                 12 
               
             
          
           
               
                 Conners&#39; Combined 
               
               
                 Parent (n = 17 for treatment, n = 24 for control) g   
               
             
          
           
               
                 Actual no 
                 14 
                 10 
               
               
                 Actual yes 
                 3 
                 14 
               
             
          
           
               
                 Teacher (n = 15 for treatment, n = 24 for control) h   
               
             
          
           
               
                 Actual no  
                 20 
                 4 
               
               
                 Actual yes 
                 7 
                 8 
               
               
                   
               
               
                   a False-postive = 5, true-positive = 20, false-negative = 6, true-negative = 21; sensitivity = 0.769, specificity = 0.807. 
               
               
                   b False-postive = 6, true-positive = 10, false-negative = 7, true-negative = 29; sensitivity = 0.588, specificity = 0.828. 
               
               
                   c False-postive = 1, true-positive = 3, false-negative = 1, true-negative = 16; sensitivity = 0.75, specificity = 0.941. 
               
               
                   d False-postive = 2, true-positive = 11, false-negative = 1, true-negative = 19; sensitivity = 0.916, specificity = 0.904. 
               
               
                   e False-postive = 10, true-positive = 6, false-negative = 3, true-negative = 14; sensitivity = 0.666, specificity = 0.833. 
               
               
                   f False-postive = 4, true-positive = 12, false-negative = 11, true-negative = 20; sensitivity = 0.916, specificity = 0.904. 
               
               
                   g False-postive = 10, true-positive = 14, false-negative = 3, true-negative = 14; sensitivity = 0.823, specificity = 0.583. 
               
               
                   h False-postive = 4, true-positive = 8, false-negative = 7, true-negative = 20; sensitivity = 0.533, specificity = 0.833. 
               
               
                 CPT, Continuous Performance Test. 
               
             
          
         
       
     
         [0050]    Comparison of the Proposed Approach with Existing Methods 
         [0051]    After the models were generated, the accuracy/F-measure for each model was evaluated against the psychiatrist&#39;s diagnosis. The results from subjects&#39; parents/teachers Conners&#39; Brief Rating Scales and the subjects&#39; CPT score were used to provide clinical support for the diagnosis. The results for ADHD determination were divided into inattention and hyperactivity models. Additionally, a confusion matrix was constructed to convey these symptoms for all three methods, which is shown in Table 3. The complexity added by the Conners&#39; Brief Rating Scale resulted in dividing this specific matrix by the Parent and Teacher Versions ( FIG. 5 ). Inattentive type. In the study, subjects with ADHD, inattentive type, were accurately diagnosed by the CPT 75 percent of the time. Conners&#39; Brief Rating Scale, Parent Version represented an inattention subscale average tilt (T) score of 81.66 (a T score of &gt;70.0 is clinically significant) by which six out of nine subjects were correctly diagnosed, or 67 percent accuracy. The Conners&#39; Brief Rating Scale, Teacher Version represented an inattention subscale average T score of 62.37, by which two out of eight subjects were correctly diagnosed, or 25 percent accuracy. The predictive model provided accuracy of 78 percent. Combined type. For subjects with ADHD, combined type, the CPT had accuracy of 91.66 percent. Conners&#39; Brief Rating Scale, Parent Version represented an inattention subscale average T score of 80.71 and a hyperactivity subscale average T score of 82.12. Ftheteen out of 17 correctly identified subjects with ADHD, combined type, or 82 percent. The Conners&#39; Brief Rating Scale, Teacher Version represented an inattention subscale average T score of 74.87 and a hyperactivity subscale average T score of 76.07. Eight out of 15 subjects were correctly identified as ADHD, combined type, or 53 percent accuracy. The predictive model provided accuracy of 75 percent. Controls. For control subjects, the CPT was 94.11 percent accurate. Conners&#39; Brief Rating Scale, Parent Version represented an inattention subscale average T score of 64.79 and a hyperactivity subscale average T score of 60.21. Fifteen out of 24 subjects were correctly identified as not having ADHD, or 58 percent accuracy. For the Teachers Version, the inattention subscale average T score was 59.83, and the hyperactivity subscale average was 57.91. Twenty out of 24 subjects were correctly identified as not having ADHD, or 83 percent accuracy. The predictive model provided accuracy of 80.77 percent for inattention controls and 82.86 percent for combined type controls. 
       Discussion 
       [0052]    This study shows “Groundskeeper” has the potential to improve accuracy and access to diagnostics of disorders with executive functioning deficits through objective data collection methods using multisensor technology. The main study findings were that the game played on the Sifteo Cubes was able to accurately detect ADHD, combined type, 75 percent of the time and ADHD, inattentive type, 78 percent through machine learning techniques. The game was also able to accurately detect anxiety 71 percent of the time and depression 76 percent of the time. No conclusions were made regarding the assessment of ASD through gameplay given the limited number of participants. These results suggest this game is successful in identifying executive functioning deficits and differentiating based on patterns of behavior among ADHD, combined type, ADHD, inattentive type, anxiety, and depression with high accuracy. In the study, the results were with the participants&#39; performance on the CPT and by the Conners&#39; Brief Rating Scale, Parent and Teacher Versions. These tests in the study population, who were recruited from a higher level of care, were more predictive than “Groundskeeper.” However, the clinical severity of the participant&#39;s mental health diagnoses was above average, indicated by not only the severity but also the frequency of comorbidities, which may have contributed to the increased accuracy of these measures in comparison with previously quoted reliability scores by Conners and other studies using these tools. Games provide a fun and engaging way of gathering data through play that can identify aberrant patterns of behavior indicative of ADHD and other mental health diagnoses. Additionally, through the use of the Sifteo Cubes, not not only can response data be collected, but also behavior exhibited by the participant in between responses. By capturing a vast array of behavioral data, small components of volitional and involuntary movement may be analyzed that may provide additional clues for diagnostics of cognitive disorders. Although the CPT and the Conners&#39; Brief Rating Scale were found to be more accurate compared with the method, several considerations suggest that further pursuit of the approach is warranted. First, the approach was successful. It has several advantages over other clinical diagnostic aids. There are practical logistical advantages using the approach. Based on the results presented in Table 4, it is evident that obtaining CPT scores from every subject was a challenge. Only 33 subjects exhibited symptoms that warranted this expensive computer-based test. Also, for the Conners&#39; Brief Rating Scales, six subjects did not return one or all of the forms. This suggests a significant effort in coordination and time to complete testing. “Groundskeeper” is as accessible as the Conners&#39; Rating Scales but does not require this coordination. Furthermore, given its vast quantity of multidimensional data, the approach may prove to have a better ability to assess simultaneously symptoms across interrelated psychiatric disorders that show high comorbidity. This may help to provide important insight into convergent and divergent aspects of related forms of psychopathology. Given the accessibility of the game medium and ease of use, the machine learning-based diagnostic aid method can become widespread and improve diagnosis accuracy for executive functioning disorders. The introduction of more feature variables and/or a more precise algorithm could improve the model accuracy. The algorithm shapes the model to the training set, which may cause extra error during cross-validation or when applying to a new test set. Another potential source of error in the feature variable creation is a small scale that did not magnify differences sufficiently. Feature variable creation, selection, and additional algorithm testing on a larger sample are primary extensions of this project. Additionally, with a large enough population, patterns of responses can be analyzed for those with ASD. 
       CONCLUSIONS 
       [0053]    In brief, the methods provide extremely rich datasets that aid clinical analysis and decision making. Collecting data along multiple dimensions of expressed behavioral deficits, specifically the response to audio, visual, and spatial stimuli, represents a potent novel approach. In particular, the ability to quantitatively identify the distortion of motor and cognitive skills linked to cognitive deficits may help lead to a better quality of care.