Patent Description:
In the above technical field, patent literature <NUM> discloses a technique of supporting rehabilitation of a patient.

Patent literature <NUM>: <CIT> <CIT> discloses a rehabilitation support apparatus that provides rehabilitation comfortable for a user includes a detector that detects a direction of a head of a user wearing a head mounted display, a first display controller that generates, in a virtual space, a rehabilitation target object to be visually recognized by the user and displays the target object on the head mounted display in accordance with the direction of the head of the user detected by the detector, and a second display controller that displays, on the head mounted display, a notification image used to notify the user of a position of the target object.

<CIT> discloses a rehabilitation system for performing rehabilitation of higher brain dysfunction, the rehabilitation system comprising: an image processing device which executes an application for presenting a patient with a problem for rehabilitation based on images using virtual reality, augmented reality, or mixed reality, and stores, as rehabilitation history information, the patient's problem solving history; a practitioner-side terminal which receives the rehabilitation history information from the image processing device; a server which stores the rehabilitation history information sent from the practitioner-side terminal; and a doctor-side terminal which receives rehabilitation history information from the server and displays the implementation status of the rehabilitation of the patient on the basis of the rehabilitation history information.

<CIT> discloses a system configured to perform active target updating according to a rehabilitation action of a user, in which a first rehabilitation action of the user is detected, and an avatar image that moves in accordance with the detected first rehabilitation action and a target image representing a target of the first rehabilitation action are displayed. The rehabilitation ability of the user is evaluated by comparing the first rehabilitation action and a target position represented by the target image, and the target position is updated in accordance with an evaluation result. Furthermore, a second rehabilitation action of the user during the first rehabilitation action is detected, and the rehabilitation ability is evaluated based on both the first rehabilitation action and the second rehabilitation action. In addition, when at least a predetermined evaluation is made only for the first rehabilitation action, the rehabilitation ability is evaluated based on both the first rehabilitation action and the second rehabilitation action.

<CIT> discloses a battery of three or more sensory and cognitive challenge tasks which actively or dynamically challenge the brain to monitor its state for assessment of injury, disease, or compound effect, among others. The system analyzes and assesses a personalized biometric brain health signature by integrating the use of electroencephalography (EEG), somato-sensory, neuropsychological, and/or cognitive stimulation, and novel signal processing and display. The system also provides for early detection of dementia, including Alzheimer's disease (AD), vascular dementia (VAD), mixed dementia (AD and VAD), MCI, and other dementia-type disorders, as well as brain injury states such as mild Traumatic Brain Injury and can provide some or all of the following improvements over conventional systems and methods, including: (<NUM>) Increased sensitivity, specificity, and overall accuracy; (<NUM>) early detection of disease and injury; and (<NUM>) enhanced portability with remote data acquisition capability.

However, in the technique described in the above literature, paragraph [<NUM>] describes "it is possible to intuitively grasp the progress of treatments, that is, the recovery of motor functions or reflect it on a current action as needed", and therefore, the purpose of treatments is "recovery of motor functions". For this reason, in the conventional technique, cognitive function determination is not performed.

The present invention enables to provide a technique of solving the above-described problem.

According to the invention, there is provided the subject matter of the independent claims. Some further aspects are defined in the dependent claims. The embodiments that do not fall under the scope of the claims are to be interpreted as examples useful for understanding the disclosure.

According to the present invention, it is possible to accurately evaluate the cognitive ability of a user.

Example embodiments will now be described in detail with reference to the drawings. It should be noted that the relative arrangement of the components, the numerical expressions and numerical values set forth in these example embodiments do not limit the scope of the present invention.

There are a lot of indices for evaluating a cognitive level, including MMSE (Mini-Mental State Examination), HDS-R (Hasegawa's Dementia Scale-Revised), MoCA (Montreal Cognitive Assessment), TMT (Trail-Making Test), and FAB (Frontal Assessment Battery).

However, these indices require questions or tasks on paper to a user and therefore are evaluation indices requiring language elements or including many elements such as literacy and writing ability, which are irrelevant to pure cognition.

Aiming at correctness, a time of <NUM> to <NUM> or more is needed for examinations. If examinations are repeated in a short period, the user learns answers to questions. Alternatively, if the condition is mild, there exists a so-called ceiling effect and, for example, the cognitive function is hardly reflected on the examination result. As described above, these methods can be used only to evaluate the cognitive level in a specific range.

Hence, there is a demand for a cognitive ability evaluation method that does not depend on literacy, writing ability, or a language, can be performed in a short period, can estimate various cognitive ability levels, and can avoid a situation where a subject knows answers due to repetitive examinations.

If such a method is established, training according to the estimated cognitive function can continuously be conducted, and this is expected to be effective for establishment of function improvement of cognitive ability (https://www. neurology-jp. org/guidelinem/degl/degl_2017_02.

A cognitive ability estimation apparatus <NUM> according to the first example embodiment will be described with reference to <FIG>. The cognitive ability estimation apparatus <NUM> is an apparatus for estimating the cognitive ability level of a user.

As shown in <FIG>, the cognitive ability estimation apparatus <NUM> includes a display controller <NUM>, a setter <NUM>, and an estimator <NUM>.

The display controller <NUM> generates and displays, in a virtual space <NUM>, a target object <NUM> configured to urge a user <NUM> to do a three-dimensional body action.

As the attributes of the target object <NUM> in the virtual space <NUM>, the setter <NUM> can set at least one of the moving speed, the number of displayed objects, the size, the display position, and the display interval.

If the user <NUM> achieves the body action to the target object <NUM>, the estimator <NUM> estimates the cognitive ability level of the user <NUM> in accordance with the attributes of the target object <NUM>. As for the estimation of the cognitive ability level, one body action is defined as one task, and the estimation may be performed based on one task, or may be done based on the results of a plurality of tasks.

With the above-described configuration, it is possible to estimate the cognitive ability level of the user more accurately.

A rehabilitation support system <NUM> according to the second example embodiment will be described with reference to <FIG> is a view for explaining the configuration of the rehabilitation support system <NUM> according to this example embodiment.

As shown in <FIG>, the rehabilitation support system <NUM> includes a rehabilitation support apparatus <NUM>, two base stations <NUM> and <NUM>, a head mounted display <NUM>, and two controllers <NUM> and <NUM>. A user <NUM> sitting on a chair <NUM> twists the upper half body or stretches the hands in accordance with display on the head mounted display <NUM>, thereby making a rehabilitation action. In this example embodiment, a description will be made assuming rehabilitation performed while sitting on a chair. However, the present invention is not limited to this. The rehabilitation action may be made while standing or walking, or on a bed, at a supine position, or at a prone position. Alternatively, the rehabilitation action may be made while running or making another specific action or motion. In addition, the controllers may be held on or attached to body parts other than hands, such as feet or trunk.

The two base stations <NUM> and <NUM> sense the motion of the head mounted display <NUM> and the motions of the controllers <NUM> and <NUM>, and send these to the rehabilitation support apparatus <NUM>. The rehabilitation support apparatus <NUM> performs display control of the head mounted display <NUM> based on the motion of the head mounted display <NUM>. The rehabilitation support apparatus <NUM> also evaluates the rehabilitation action of the user <NUM> based on the motions of the controllers <NUM> and <NUM>. Note that the head mounted display <NUM> can be of a non-transmissive type, a video see-through type, an optical see-through type, or a spectacle type. In this example embodiment, a virtual space of VR (Virtual Reality) is presented to the user. However, a physical space and a virtual space may be displayed in a superimposed manner, like AR (Augmented Reality), physical information may be reflected on a virtual space, like MR (Mixed Reality), or a hologram technology may be used as an alternative means.

In this example embodiment, as an example of a sensor configured to detect the position or action of the hand or head of the user, the controllers <NUM> and <NUM> held in the hands of the user <NUM>, and the base stations <NUM> and <NUM> have been described. However, the present invention is not limited to this. A camera (including a depth sensor) configured to detect the positions or action of the hands themselves of the user by image recognition processing, or a sensor configured to detect the positions of the hands of the user by a temperature may be used. A wristwatch-type wearable terminal put on an arm of the user, a motion capture device, or the like can also be applied to the present invention interlockingly with an action detector <NUM>. That is, using a three-dimensional tracking device such as Kinect® or an action analysis device or attaching a marker to the body is also one example embodiment.

The rehabilitation support apparatus <NUM> includes the action detector <NUM>, display controllers <NUM> and <NUM>, a feedback unit <NUM>, an evaluation updater <NUM>, a task set database <NUM>, a setter <NUM>, an estimator <NUM>, and a storage unit <NUM>.

The action detector <NUM> acquires, via the base stations <NUM> and <NUM>, the positions of the controllers <NUM> and <NUM> held in the hands of the user <NUM>, and detects the rehabilitation action of the user <NUM> based on changes in the positions of the hands of the user <NUM>.

The display controller <NUM> generates and displays, in a virtual space <NUM>, a target object <NUM> configured to urge the user <NUM> to do a three-dimensional body action. In particular, the display controller <NUM> generates, in the virtual space, avatar objects <NUM> that move in accordance with a detected rehabilitation action and the target object <NUM> representing the target of the rehabilitation action. The display controller <NUM> displays, on a display screen (in the virtual space <NUM>), the images of the avatar objects <NUM> and the target object <NUM> in accordance with the direction and position of the head mounted display <NUM> detected by the action detector <NUM>. The images of the avatar objects <NUM> and the target object <NUM> are superimposed on a background image <NUM>. Here, the avatar objects <NUM> have the same shape as the controllers <NUM> and <NUM>. However, the present invention is not limited to this, and the size, shape, or color may be changed on the left and right sides. The avatar objects <NUM> move in the virtual space <NUM> in accordance with the motions of the controllers <NUM> and <NUM>. The controllers <NUM> and <NUM> are each provided with at least one button and configured to perform various kinds of settings including initial settings such as origin setting by operating the button. The button can be disabled, or instead of arranging the button, all settings may be executed separately using an external operation unit. The background image <NUM> is cut out from a virtual space including a horizontal line <NUM> and a ground surface image <NUM>.

The display controller <NUM> generates the target object <NUM> in the virtual space and moves this downward from above the user <NUM>. Accordingly, on the head mounted display <NUM>, the target object is displayed such that the display position and size gradually change (for example, gradually becomes large and then becomes small). The user <NUM> moves the controllers <NUM> and <NUM> to make the avatar objects <NUM> in the screen close to the target object <NUM>. Note that as for the moving direction of the target object, for example, the target object may be displayed such that it rises from the floor surface to above the head. Also, including movement in the depth direction or the left-and-right direction in addition to the movement in the vertical direction, any three-dimensional movement may occur, or the target object may be fixed at a specific coordinate position without moving.

The user <NUM> moves the controllers <NUM> and <NUM> to bring the avatar objects <NUM> in the screen close to the target object <NUM>. If the avatar object <NUM> hits the target object <NUM>, the display controller <NUM> causes the target object <NUM> to disappear, and the feedback unit <NUM> determines that the target action is achieved, and displays a message. More specifically, if the shortest distance between the target object <NUM> and a sensor object included in the avatar object <NUM> falls within a predetermined range, the target is achieved, and the target object <NUM> disappears. At this time, if the shortest distance between the target object <NUM> and the sensor object (for example, a spherical object including the center point of the avatar object <NUM>) included in the avatar object <NUM> is equal to or less than a predetermined threshold S1, the target is completely achieved. In a case of complete achievement, "excellent" is displayed, and a corresponding voice is output simultaneously to make feedback. At the same time, the controller <NUM> or <NUM> held by the hand that has achieved the task may be vibrated, or stimuli may be imparted to the sense of smell or the sense of taste. The rehabilitation action may be evaluated in three or more levels depending on how much the distance between the sensor object and the target object <NUM> decreases. Also, two or more target objects may be generated simultaneously in the virtual space and displayed. Notifications of task achievement for five sense stimulation may be combined in any way in accordance with the action type or the like. If the shortest distance between the target object <NUM> and the sensor object included in the avatar object <NUM> is not less than the threshold S1 and not more than a threshold S2, "well done" is displayed because of the achievement of the target, and a corresponding voice is output to make feedback. Note that the output voice need not be the same as the displayed message, and a nonverbal sound effect, for example, "dingdong" may be used.

The display controller <NUM> displays a radar screen image <NUM> on the display screen of the head mounted display <NUM>. The radar screen image <NUM> is a notification image used to notify the user of generation of the target object <NUM>. The radar screen image <NUM> notifies the user of the direction in which the generated target object <NUM> is located relatively with respect to a reference direction (initially set in the front direction of the user who is sitting straight on a chair) in the virtual space. The radar screen image <NUM> also notifies the user how far the position of the generated target object <NUM> is apart from the user <NUM>. Note that the notification image is not limited to the radar screen image, and the notification may be made using characters, an arrow, a symbol, an illustration, or a type, intensity, blinking, or the like of light or a color. The notification method is not limited to the image, and may use a voice, a vibration, or a combination of some of a voice, a vibration, and an image. Independently of the direction of the head of the user <NUM>, the display controller <NUM> displays the radar screen image <NUM> at the center (for example, within the range of -<NUM>° to <NUM>°) of the display screen of the head mounted display <NUM>. However, the display portion is not limited to the center, and may be, for example, an arbitrary place on the four corners, the upper end, the lower end, the left end, and the right end of the screen.

The radar screen image <NUM> includes a head image <NUM> representing the head of the user viewed from above, a block image <NUM> obtained by dividing the periphery of the head image <NUM> into a plurality of blocks, and a fan-shaped image <NUM> as a visual field image representing the visual field of the user. A target position image representing the position of a target object is shown by coloring, blinking, or lighting a block in the block image <NUM>. This allows the user <NUM> to know whether the target object exists on the left side or the right side with respect to the direction in which he/she faces. Note that in this example embodiment, the block image <NUM> is fixed, and the fan-shaped image <NUM> moves. However, the present invention is not limited to this, and the block image <NUM> may be moved in accordance with the direction of the head while fixing the fan-shaped image <NUM> and the head image <NUM>. More specifically, if the head turns to the left, the block image <NUM> may rotate to right.

The feedback unit <NUM> preferably changes the message type via the display controller <NUM> in accordance with evaluation of the rehabilitation action. For example, if the sensor object contacts the center of the target object <NUM>, "excellent" is displayed. If the sensor object contacts only the peripheral portion of the center of the target object <NUM>, "well done" is displayed. The size of the target object <NUM> and the size of the peripheral portion can be set by the setter <NUM>. The size of the sensor object can also be set by the setter <NUM>.

The feedback unit <NUM> performs feedback to impart stimuli to two or more of the five senses (sense of sight, sense of hearing, sense of touch, sense of taste, and sense of smell) of the user who has virtually touched the target object <NUM>. The feedback is performed almost at the same time as the timing at which the sensor object enters a predetermined distance from the center of the target object <NUM> or the timing at which the sensor object contacts the target object <NUM> (real-time multi-channel biofeedback). The effect is large if a delay from the timing to feedback is, for example, <NUM> sec or less. The shorter the interval between the operation timing of the user and the timing of feedback is (the smaller the delay is), the larger the effect is. While performing feedback of giving stimuli to the sense of sight of the user by an image "excellent! ", the feedback unit <NUM> simultaneously performs feedback of giving stimuli to the sense of hearing of the user by a voice output from a speaker <NUM>. The stimulating feedback to the five senses (sense of sight, sense of hearing, sense of touch, sense of taste, and sense of smell) is performed to notify the user of the presence/absence of achievement in each task.

Also, the feedback unit <NUM> may simultaneously output feedback of giving stimuli to the sense of sight of the user <NUM> by the image "excellent!", feedback of giving stimuli to the sense of hearing of the user <NUM> by the voice output from the speaker, and feedback of giving stimuli to the sense of touch of the user <NUM> by causing the controller <NUM> to vibrate. Alternatively, the feedback unit <NUM> may simultaneously output only two types of feedback, including feedback of giving stimuli to the sense of sight of the user <NUM> by the image "excellent! ", and feedback of giving stimuli to the sense of touch of the user <NUM> by causing the controller <NUM> to vibrate. The feedback unit <NUM> may simultaneously output only two types of feedback, including feedback of giving stimuli to the sense of hearing of the user <NUM> by a voice "excellent!", and feedback of giving stimuli to the sense of touch of the user <NUM> by causing the controller <NUM> to vibrate.

The action of the user <NUM> moving the controllers <NUM> and <NUM> is a rehabilitation action, and display of a target object for urging the user <NUM> to do one rehabilitation action that he/she should perform is called a task. Information (task data) representing one task includes at least the moving speed, the number of displayed objects, the size, the display position, and the display interval as the attributes of the target object <NUM> in the virtual space <NUM>. The task data may also include the appearance direction of the target object (right <NUM>°, right <NUM>°, front, left <NUM>°, and left <NUM>° with respect to the front direction of the chair), the distance to the target object, the shape (size) of the target object, the appearance position (the depth direction distance from the user), the appearance interval (time interval), the moving speed in falling, rising, or the like, the color of the target object, which one of the left and right controllers should be used to acquire, the number of target objects that simultaneously appear, the size of the sensor object, and the like. The distance from the user <NUM> to the fall position of the target object <NUM> in the depth direction may continuously be set, or may be set to one of, for example, three stages. For example, a change can be made such that the target object falls quite near the user <NUM> or falls to a position that the user <NUM> cannot reach unless largely inclining the body forward. This can control an exercising load to be given to the user, and a load to a spatial cognitive ability or a spatial comprehension ability.

The evaluation updater <NUM> evaluates the rehabilitation action of the user in accordance with the amount and quality of the task achieved by the user <NUM> and adds a point. Here, the quality of the achieved task includes "well done" or "excellent", that is, how close the avatar object could be brought to the target object. The evaluation updater <NUM> adds different points to achieved tasks (a high point to a far object, and a low point to a close object). The evaluation updater <NUM> can update a task in accordance with the integrated point. For example, a task (the attribute of a target object) may be updated using a task achievement ratio (the number of achieved targets/the number of tasks). The evaluation updater <NUM> compares the rehabilitation action detected by the action detector <NUM> and a target position represented by the target object displayed by the display controller <NUM>, and evaluates the rehabilitation capability of the user <NUM>. More specifically, it is decided, by comparing the positions in the three-dimensional virtual space, whether the target object <NUM> and the avatar object <NUM> that moves in correspondence with the rehabilitation action detected by the action detector <NUM> overlap. If these overlap, it is evaluated that one rehabilitation action is cleared, and a point is added. The display controller <NUM> can make the target object <NUM> appear at different positions (for example, positions of three stages) in the depth direction. The evaluation updater <NUM> gives different points (a high point to a far object, and a low point to a close object).

The evaluation updater <NUM> updates a target task in accordance with the integrated point. For example, a target task may be updated using a task achievement ratio (the number of achieved targets/the number of tasks).

The task set database <NUM> stores a set of a plurality of tasks. A task represents one rehabilitation action that the user should perform. More specifically, as information representing one task, the position of a target object that appears, its speed and size, and the size of an avatar object at that time, and the like are stored. The task set database <NUM> stores a task set that decides the order of providing the plurality of tasks to the user. For example, task sets may be stored as templates for each hospital, or a history of executed task sets may be stored for each user. The rehabilitation support apparatus <NUM> may be configured to be communicable with another rehabilitation support apparatus via the Internet. In this case, one task set may be executed by the same user in a plurality of places, or various templates may be shared by a plurality of users in remote sites.

As the attributes of the target object <NUM> in the virtual space <NUM>, the setter <NUM> sets at least one of the moving speed, the number of displayed objects, the size, the display position, and the display interval. If the user <NUM> can achieve a body action to the target object <NUM>, the estimator <NUM> estimates the cognitive ability level of the user <NUM> in accordance with the attributes of the target object <NUM>. The setter <NUM> can set a delay time from the timing of notifying the generation of the target object <NUM> to the timing of generating the target object <NUM>, thereby controlling a cognitive load given to the user <NUM>. That is, the user needs to continuously memorize and hold an action that he/she should perform during the time after he/she knows, by the radar screen image <NUM> or the like, a position in the virtual space where the target object is generated (a position representing in which direction the head mounted display should directed to display the target object) until actual generation of the target object. The "memorization time" is a cognitive load for the user. Also, the setter <NUM> may control the cognitive load by changing the time not "until the timing of generating the target object <NUM>" but "until the target object <NUM> approaches the range the user <NUM> can reach". The setter <NUM> may give a cognitive load to the user <NUM> by displaying the background image <NUM> other than the target object <NUM> on the head mounted display <NUM>.

Note that when changing the cognitive load, it is preferable to notify the user in advance that the cognitive load is to be increased or decreased. As for the notification method, the notification may be done by visually using characters or a symbol, by a voice, or by touching a part of the body, for example, by tapping a shoulder, an elbow, an arm, or a foot.

<FIG> is a view showing an example of a screen (operation panel) <NUM> to be operated by an operator. The setter <NUM> displays the operation panel <NUM>. In this example embodiment, seven parameters (distance, height, angle, size, speed, sensitivity, and interval) are set by the intuitive operation panel <NUM>, thereby generally evaluating a posture balance ability and a dual task type cognitive processing ability. Measurement can be done by one of a manual mode (a method of setting parameters of each task and performing an operation on a task basis), a template mode (a method of setting parameters for a plurality of task sets in advance), and an auto mode in which a device is caused to automatically generate a task, or a combination thereof. Note that it is also possible to, on the operation panel, confirm the basic information of the user and various kinds of cognitive and motor functions evaluation indices and examination results, create a template, and set and instruct the auto mode.

The display that displays the operation panel <NUM> can be any device, and may be an external display connected to the rehabilitation support apparatus <NUM> or a display incorporated in the rehabilitation support apparatus <NUM>. The operation panel <NUM> includes a user visual field display region <NUM>, various parameter setting regions <NUM> to <NUM>, a score display region <NUM>, and a re-center button <NUM>. For a descriptive convenience, <FIG> includes a region <NUM> showing the actual state of the user <NUM>. However, the operation panel <NUM> need not include the region <NUM>.

The user visual field region <NUM> shows an image actually displayed on the head mounted display <NUM>. A reference direction in the virtual space is displayed in the user visual field region <NUM>. As described with reference to <FIG>, the radar screen image <NUM> is displayed at the center (for example, within the viewing angle range of -<NUM>° to <NUM>°) of the user visual field region <NUM>. The radar screen image <NUM> shows the relative direction of the position of the target object <NUM> that appears next with respect to the reference direction in the virtual space. In this example, the coloring position in the block image <NUM> represents that the target object <NUM> appears at the farthest position on the left side with respect to the reference direction <NUM> in the virtual space. Based on the position of the fan-shaped image <NUM> and the direction of the head image <NUM>, it can be seen that the user already faces left.

The various parameter setting regions <NUM> to <NUM> are screens configured to set a plurality of parameters for defining a task. The setter <NUM> can accept inputs to the various parameter setting regions <NUM> to <NUM> from an input device (not shown). The input device may be a mouse, a ten-key pad, or a keyboard, or may be various kinds of controllers, a joystick for game, or a touch panel, and can use any technical component.

The various parameter setting regions <NUM> to <NUM> include the input region <NUM> that decides the sizes of left and right target objects, the input region <NUM> that decides the size of the sensitivity range of the avatar object <NUM>, the input region <NUM> that decides the moving speed of the target object, and the input region <NUM> that decides the position of a target object that appears next. The operation panel <NUM> also includes a check box <NUM> that sets whether to accept an operation of a target object appearance position by an input device (hot key).

The input region <NUM> can set, on each of the right and left sides, the radius (visual recognition size) of a visual recognition object that makes the target object position easy for the user to see, and the radius (evaluation size) of a target object that reacts with the avatar object <NUM>. That is, in the example shown in <FIG>, the user can see a circle with a radius of <NUM>. Actually, the task is correctly completed only when he/she has touched a ball with a radius of <NUM> located at the center of the circle. If the visual recognition size is small, it is difficult for the user to find the target object. If the visual recognition size is large, the user can easily find the target object. If the evaluation size is large, the allowable amount of the deviation of the avatar object <NUM> is large. If the evaluation size is small, the allowable amount of the deviation of the avatar object <NUM> is small, and a rehabilitation action can be evaluated more severely. The visual recognition sizes and the evaluation sizes may be made to match.

In the input region <NUM>, the left and right sensor sizes of the avatar object <NUM> (the size of the sensor range of the sensor object) can separately be set. If the sensor size is large, a task is achieved even if the position of a hand largely deviates from the target object. Hence, the difficulty of the rehabilitation action is low. Conversely, if the sensor size is small, it is necessary to correctly move the hand to the center region (evaluation size) of the target object. Hence, the difficulty of the rehabilitation action is high. In the example shown in <FIG>, the sensor sizes are <NUM> on the left and right sides.

In the input region <NUM>, the speed of the target object <NUM> moving in the virtual space can be defined on each of the left and right sides. In this example, the speed is set to <NUM>/s.

The input region <NUM> is an image used to input the position of the next position (the distance to the task and the angle), and has the shape of the enlarged radar screen image <NUM>. Since the check box <NUM> has a check mark, if an operation of clicking or tapping one of a plurality of blocks in the input region <NUM> is performed, the target object <NUM> is generated at a position in the virtual space corresponding to the position of the block for which the operation is performed.

That is, in the example shown in <FIG>, the task for the user <NUM> is to bring an avatar object (controller) including a sensor portion with a size of <NUM> into contact with a target object (ball) having a radius of <NUM> that falls at a speed of <NUM>/s in a far place on the left side in a good timing in the virtual space. <FIG> shows a state in which the target object <NUM> appears in the user visual field region <NUM>. In this state, when the user <NUM> stretches the left arm, as shown in the region <NUM>, the avatar object <NUM> appears in the user visual field region <NUM>. A visual recognition object <NUM> that raises the visibility of the target object <NUM> is displayed around the target object <NUM>. The visual recognition size set in the input region <NUM> is the radius of the visual recognition object <NUM> with a doughnut shape.

At the point of time when the avatar object <NUM> contacts the visual recognition object <NUM>, the target object <NUM> does not disappear, and correct achievement of the task is not obtained (a predetermined point is added, and good evaluation is done). Correct achievement of the task (perfect evaluation) is obtained only when the avatar object <NUM> contacts the target object <NUM>.

On the other hand, the total number of tasks at each position, and a count point representing how many times a task has been achieved are shown in the score display region <NUM>. The point may be expressed as a fraction or a percentage, or a combination thereof. After a series of rehabilitation actions decided by one task set, the feedback unit <NUM> derives a rehabilitation evaluation point using values in the score display region <NUM>.

An example of calculation of the rehabilitation evaluation point is as follows. Rehabilitation evaluation point = <NUM> × ([determination score × count of Short] + [determination score × count of Middle × <NUM>] + [determination score × count of Long × <NUM>] + [count of five continuous Perfect catch × <NUM>]), determination score: Perfect (excellent) = <NUM>, Good (well done) = <NUM>, Failure = <NUM>.

The re-center button <NUM> is a button that accepts, from the operator, a reconstruction instruction for reconstructing the virtual space in accordance with the position of the user <NUM>. If the re-center button <NUM> is operated, the display controller <NUM> sets the position of the head mounted display <NUM> at the instant to the origin, and reconstructs the virtual space having, as the reference direction, the direction of the head mounted display <NUM> at that instant.

<FIG> is a view showing a task table <NUM> stored in the task set database <NUM>. In the task table <NUM>, a time (task generation timing) <NUM>, a task interval <NUM> from the end of an immediately preceding task, a task type <NUM>, a task angle <NUM>, and a task distance (intensity) <NUM>, and a task distance (intensity) <NUM> are stored in linkage with a task ID. Also, in the task table <NUM>, a target object speed <NUM>, a perfect determination (excellent evaluation) reference size <NUM>, a good determination (well done evaluation) reference size <NUM>, a sensor object size <NUM>, a task achievement result, and the like are stored in linkage with a task ID. In addition to these, a delay time (predetermined time) from task generation notification to task generation may be set for each task.

<FIG> are views showing examples of display on the head mounted display <NUM> according to this example embodiment. In <FIG>, an image showing an inro <NUM> as a target object is displayed on a background image <NUM> expressing a street in the Edo period. In addition, under the inro <NUM>, a sen-ryo-bako <NUM> is displayed as an item to be protected by the user, and a ninja <NUM> gradually comes close from the far side. The speed of the ninja <NUM> is the speed set in the input region <NUM> of the operation panel <NUM> (the speed here has the same meaning as a time limit). A circle <NUM> serving as a visual recognition assisting image is displayed on the inro <NUM>. If the user touches the inro <NUM> by the sensor object (the center of the tip of the avatar object <NUM>) before the ninja <NUM> reaches the sen-ryo-bako <NUM>, the task is achieved. Two types of circles, that is, red and blue circles are prepared as the circle <NUM>. The task for the inro <NUM> surrounded by the red circle <NUM> is to operate the red avatar object <NUM> on the right side, which corresponds to the controller <NUM> held in the right hand, and bring it into contact with the inro <NUM>. On the other hand, the task for the inro <NUM> surrounded by the blue circle <NUM> is to operate the blue avatar object <NUM> on the left side, which corresponds to the controller <NUM> held in the left hand, and bring it into contact with the inro <NUM>.

The inro <NUM> is displayed at a position (depth and angle) set in the input region <NUM> of the operation panel <NUM>. The position of the inro <NUM> does not change until the user makes the avatar object <NUM> touch it in the virtual space. That is, the inro <NUM> is a target object fixed in the space (also called a horizontal task because the user is required to horizontally stretch the body). Such a fixed target object is very effective as rehabilitation for a disease such as cerebellar ataxia. That is, an image of a limited body motion can be imprinted, by feed forward, in the brain of a patient who has forgotten how to move his/her body. If the distance of the inro <NUM> in the depth direction is increased, the motion intensity can be changed. Also, if multi-channel biofeedback by five sense stimulation for notifying the achievement of each task is combined, the memory can easily be fixed in the brain, and exercise ability greatly improves. Furthermore, according to such a horizontal task, chronic pains can be improved along with the reconstruction of cerebral cortex. Alternatively, it is possible to recover cognitive impairment called chemobrain or a phenomenon that the position sense of a cancer patient using an anticancer drug lowers due to neuropathy. The place where the target object appears may be notified in advance to give a hint and reduce the cognitive load. Touch notices by touching the body without any language is effective to reduce the cognitive load. a plurality of repeated language notices are also effective to reduce the cognitive load. As for the method of language informing, the cognitive load may be reduced by giving a simple short instruction close to the imperative form. Alternatively, a more complex instruction may be given in a questioning format, for example, using "blue? (take by the right hand)". Language informing may be given in a form including a cognitive task such as calculation, for example, "take by the right hand if you hear a number divisible by <NUM>". Note that not only the horizontal position and depth where the inro <NUM> is generated but also a height may be set.

<FIG> is a view showing an example (vertical task) of a screen for performing a task in which a target object moves vertically. In <FIG>, on a background image <NUM> representing a field, an image of a person representing a farmer is displayed as a trigger object <NUM> serving as a trigger of target object appearance. That is, the display controller <NUM> displays the trigger object <NUM> as a notification image used to notify the user of generation of a target object <NUM>. When a predetermined time elapses after the trigger object <NUM> throws up the target object <NUM> in the shape of a potato, a target object <NUM> having the shape of a large potato appears from the upper side of the screen, as shown in <FIG>. When the falling target object <NUM> is received by moving an avatar object <NUM> having the shape of a basket, the task is achieved. The left and right avatar objects <NUM> move on the screen in synchronism with the motions of the controllers <NUM> and <NUM>.

The setter <NUM> can set the delay time from the timing at which the trigger object <NUM> throws up the target object <NUM> and notifies the generation of the target object <NUM> until generation of the target object <NUM>. Thus, the cognitive load given to the user can be adjusted. Note that in synchronism with the motion of the trigger object <NUM>, generation of the target object may be notified at the same timing using the radar chart type notification image <NUM>, or a notification by a voice may be combined.

In this way, the setter <NUM> can give a cognitive load to the user not only by a task of a background including only a horizontal line as shown in <FIG> but by a task of a background including a large quantity of information as shown in <FIG> and <FIG>. That is, it is made difficult to memorize that the target object <NUM> has appeared, and the position to which the target object <NUM> is expected to fall, thereby giving a load closer to a cognitive load necessary in a real life to the user of rehabilitation.

In particular, the setter <NUM> changes the mode of the task and changes at least a part of the background image <NUM> along with time, thereby giving a cognitive load to the user <NUM>. In the example shown in <FIG>, for example, in the background image <NUM>, a cloud <NUM> may be moved, plants <NUM> may be shaken, or an animal (not shown) irrelevant to the target object may be made to appear. This can impede concentration to the target object <NUM> and make it more difficult for the user <NUM> to memorize the position to which the target object <NUM> is expected to fall. More technically, it can be said that information irrelevant to the task is displayed on the background image to prepare an environment in which it is difficult to concentrate to the target object and intentionally cause an attention disorder (more specifically, a selective attention disorder, a divided attention disorder, a conversion attention disorder, or a sustained attention disorder), thereby making memorization difficult and controlling the cognitive load.

<FIG> is a view showing another example (vertical task) of display on the display screen according to this example embodiment. In <FIG>, in a background image <NUM> like woods, a trigger object <NUM> representing a monkey and a target object <NUM> representing an apple are displayed. When the trigger object <NUM> representing a monkey drops the target object <NUM> representing an apple from a tree, and the target object <NUM> approaching the user is received by moving an avatar object <NUM> representing a basket, the task is achieved. In this case as well, the setter <NUM> starts dropping the target object <NUM> after the elapse of a predetermined time from the timing at which the trigger object <NUM> shakes the tree and notifies the generation of the target object <NUM>, thereby giving a cognitive load to the user <NUM> while causing an attention disorder.

Also, the setter <NUM> causes at least two to five target objects <NUM> to simultaneously exist in the three-dimensional virtual space <NUM> and displays these on the display screen, thereby giving a cognitively stronger load to the user <NUM>. In other words, the setter <NUM> generates the at least two target objects <NUM> at different positions in the left-and-right direction in the three-dimensional virtual space.

In particular, if the at least two target objects <NUM> are generated at a plurality of positions in a direction (the left-and-right direction in <FIG>) different from the moving direction (the falling direction in <FIG>) of the target object <NUM>, a larger cognitive load can be given. That is, since the user <NUM> needs to move the controllers <NUM> and <NUM> in consideration of the movement in the vertical direction, the difference between the generation positions in the left-and-right direction, and the difference in the falling position in the depth direction, the spatial cognitive ability is also tested. As described above, the type, number, size, spatial spread, position, amount, and the like of information included in a notification image including a trigger object or a notification sound are adjusted in addition to the change of the predetermined time of the task. It is therefore possible to quantitatively adjust and control the complexity of information to be memorized and held, that is, a cognitive load that should be subjected to information processing by the brain of the user.

The evaluation updater <NUM> evaluates the cognitive ability of the user using information such as whether the avatar object has reached, in a good timing, a three-dimensional target position represented by the target object, the time interval from target object generation notification to generation and the number of target objects, and the degree of the load that causes the attention disorder on the background image.

<FIG> is a flowchart showing the procedure of cognitive ability estimation processing of the rehabilitation support apparatus <NUM>. In step S901, as calibration processing, the target of a rehabilitation action is initialized in accordance with the user. More specifically, each user is first made to do a work for acquiring an action enable range as calibration. The range is set to the initial value, and the target is initialized in accordance with the user. Also, more specifically, the operator reads out a task set (a horizontal task set or a vertical task set) formed by a plurality of tasks from the task set database. Here, as an example, size of the target object is <NUM>, the display interval of the target object is <NUM> sec, target objects generated as one task include one to five objects, and the generation ranges are set to three patterns, that is, <NUM>°, <NUM>° to <NUM>°, and <NUM>° to <NUM>°. Three task sets are generated at an interval of <NUM> sec.

Next, in step S903, a task (that is, display of the target object and evaluation of achievement by an avatar object by the display controller <NUM>) is started.

In step S905, if all the tasks in the three continuous task sets obtain perfect determination, it is determined that the task sets are achieved, and the process advances to step S907. In accordance with the attribute of the target object in the task sets, the estimator <NUM> calculates the cognitive ability as, for example, a cognitive age. The attribute parameters used by the estimator <NUM> can variously be selected. The cognitive ability level of the user is calculated based on at least the moving speed of the target object and the number of displayed target objects (for example, one to five) in one task set. The estimator <NUM> calculates the cognitive ability level using different parameters or different parameter coefficients between a case where the target object moves in the vertical direction (vertical task) and a case where the target object does not move in the vertical direction (horizontal task). The estimator <NUM> calculates the cognitive ability level of the user using different parameters or different parameter coefficients in accordance with the background of the screen on which the target object is displayed. That is, the cognitive ability level is evaluated high for the user who has achieved the task on a complex background. Note that for an adult, a high cognitive ability level and a young cognitive age can be considered as equivalent. The higher the moving speed of the target object in the virtual space is in a task, the higher the cognitive ability level estimated by the estimator <NUM> is if the user can achieve the task. The larger the number of target objects generated in one task set is in the virtual space, the higher the cognitive ability level estimated by the estimator <NUM> is if the user can achieve the task. Here, the number of target objects generated in one task set indicates the number of target objects that can simultaneously exist in the virtual space. That is, if the number is two, two target objects are simultaneously displayed in the virtual space at maximum. The larger the number is, the larger the number of targets that the user needs to simultaneously recognize is, and the larger the load of parallel processing in the brain is.

The shorter the interval of the appearance of the target object in the virtual space is, the higher the cognitive ability level estimated by the estimator <NUM> is if the user can achieve the task. The smaller the size of the target object is, the higher the cognitive ability level estimated by the estimator <NUM> is if the user can achieve the task. The wider the appearance range of the target object is, the higher the cognitive ability level estimated by the estimator <NUM> is if the user can achieve the task. By employing such an estimation method, various problems of cognitive function evaluation indices represented by MMSE, HDS-R, MoCA, TMT, and FAB can be solved. More specifically, it is possible to estimate pure cognitive ability while excluding elements irrelevant to the cognitive ability, such as literacy and writing ability, suppress the time needed for examinations to a very short time, and eliminate learning and the ceiling effect caused by repetitive examinations.

The storage unit <NUM> stores an equation or table representing the relationship between the cognitive age and the attribute of the target object for which the user has achieved a body action. Using the equation or the table, the estimator <NUM> estimates the cognitive ability level of the user as the cognitive age. The apparatus may include an updater configured to update the equation or the table stored in the storage unit <NUM> based on big data that associates an actual age with a result of cognitive ability estimation. Machine learning may be performed using the actual age as supervisory data. If the data of population becomes large, the cognitive ability level estimation accuracy becomes high.

The setter <NUM> can also set the attributes of the target object to be displayed next in accordance with the cognitive ability level of the user estimated by the estimator <NUM>.

The estimator <NUM> can calculate the cognitive age as the cognitive ability using, for example, the following expressions. (<NUM>) Estimation accuracy priority (adjusted contribution, Adjusted R-squared: <NUM>, Akaike Information Criterion AIC = <NUM>): parameters are introduced as many as possible to obtain the highest estimation accuracy → the accuracy is so high that learning is unnecessary <MAT> <MAT>.

<FIG> shows a table <NUM> in which the cognitive age calculated by the above-described equation is applied to automobile driving risk evaluation. In a case of the background as shown in <FIG>, the age for the horizontal task in the table is decreased by <NUM>. Similarly, in a case of the background as shown in <FIG>, the age is decreased by <NUM>, and in a case of the background as shown in <FIG>, the age is decreased by <NUM>. According to <FIG>, a falling risk evaluation can also be performed in a similar way. Using the table shown in <FIG>, the estimator <NUM> evaluates the driving risk or falling risk of the user. In addition, based on the cognitive ability level of the user estimated by the estimator <NUM>, the setter <NUM> proposes, to the operator, a treatment program for improving the cognitive ability level, reducing the driving risk, or reducing the falling risk.

If the cognitive age is <NUM> years or more, the cognitive function is low, and it is evaluated that driving is dangerous. If the cognitive age is <NUM> to <NUM> years, the cognitive function is in decline, and it is evaluated that attention is needed in driving. On the other hand, if the cognitive age is <NUM> years or less, the cognitive function has no problem, and it is evaluated that the user is sufficiently capable of driving. Similarly, a falling risk in daily life may be evaluated. Here, thresholds are set to <NUM> years and <NUM> years. However, the thresholds may be changed in accordance with the required action. For example, if the user drives in a region of little traffic, it may be evaluated that driving is possible even if the cognitive age is <NUM> years.

Note that it is found that if the rehabilitation support system <NUM> according to this example embodiment is used, the cognitive function level can be improved and maintained (the effect of one training continues for three weeks, and the effect can be fixed to an almost constant level if the training is performed three times), and it is considered that for a user judged to have a risk in driving, the evaluation may be changed to "driving OK".

When estimating the cognitive age, the setter <NUM> may display a message concerning control of the cognitive load (attribute parameter). For example, a message "reduce the speed and test again" can be considered. A recommended numerical value may be presented by, for example, "set the speed to <NUM>/sec". Also, to correctly calculate the cognitive function level, various kinds of parameters may be controlled automatically.

If the rehabilitation support system <NUM> according to this example embodiment is used, it is possible to recognize the type of the lowered cognitive function and effectively conduct rehabilitation more correctly. For example, a user evaluated to be <NUM> years old in a vertical task at <NUM>/sec can do spatial recognition. If the speed is low (<NUM>/sec), he/she can handle a multitask (four targets). For such a user, training to gradually increase the speed is effective.

As described above, in this example embodiment, the cognitive function is estimated by adjusting two or more of parameters including the time limit (falling speed or the like), the size of the target, the motion of the target, the number of targets, the display position of the target in the three-dimensional space, and background bb information for intentionally causing an attention disorder. The system according to this example embodiment is a system that needs color identification. Also, the system according to this example embodiment is a system configured to judge the type of attention disorder or treat the attention disorder. Also, the system according to this example embodiment is a system configured to evaluate or treat a falling risk or a driving risk.

According to this example embodiment, it is also possible to judge which one of a sustained attention disorder (for example, the user cannot continuously concentrate to an object), a conversion attention disorder (for example, if two target objects are generated, the user diverts attention to the next object before he/she completely takes one object), a divided attention disorder (for example, the user cannot take a target object outside the visual field where the space can be recognized (cannot see a notification image)), and a selective attention disorder (for example, the user cannot concentrate to an object as soon as a background is displayed (the user cannot discriminate an object that should be seen)) the user has.

While the disclosure has been particularly shown and described with reference to example embodiments thereof, the invention is not limited to these example embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein, the scope of the present invention being defined by the claims.

Claim 1:
A cognitive ability estimation apparatus comprising:
an action detector (<NUM>) that is configured to detect a three-dimensional body action of a user;
a display controller (<NUM>) that is configured to generate and display, in a virtual space, an avatar object that moves in accordance with the detected body action and a target object for urging the user to perform the three-dimensional body action;
a setter (<NUM>) that is configured to set, as attributes of the target object in the virtual space, a moving speed, the number of displayed target objects, a size, a display position, and a display interval;
a feedback unit (<NUM>) that is configured to determine that the body action for the target object is achieved in case the avatar object hits the target object; and
an estimator (<NUM>) that is configured to estimate a cognitive ability level of the user in accordance with the attributes of the target object in a case where the body action for the target object can be achieved by the user or not,
characterized in that
the estimator (<NUM>) is configured to calculate the cognitive ability level using a value according to the moving speed of the target object in the virtual space, a value according to complexity of a background in the virtual space in which the target object is generated, a value according to a moving direction of the target object in the virtual space, a value according to the size of the target object in the virtual space, a value according to a size of an appearance range of the target object in the virtual space, a value according to whether the target object moves in a vertical direction, a value indicating a number of the target objects that simultaneously appear in the virtual space and an interval of appearance of the target object in the virtual space.