Patent Description:
In the above technical field, patent literature <NUM> discloses a system configured to support rehabilitation.

However, in the technique described in the above literature, it is impossible to control a load to be given to the cognitive function of a user in rehabilitation.

The present invention enables to provide a technique of solving the above-described problem.

Aspects of the invention provide a rehabilitation support apparatus, a rehabilitation support method and a rehabilitation support program for recovering one of cognitive impairment and a higher brain dysfunction, as specified by the appended claims.

According to the present invention, it is possible to control a load to be given to the cognitive function of a user in rehabilitation.

Example embodiments of the present invention will now be described in detail with reference to the drawings. It should be noted that the relative arrangement of the components, the numerical expressions and numerical values set forth in these example embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.

A rehabilitation support apparatus <NUM> according to the first example embodiment of the present invention will be described with reference to <FIG>.

The rehabilitation support apparatus <NUM> includes a detector <NUM>, a display controller <NUM>, a notifier <NUM>, and a cognitive load controller <NUM>.

The detector <NUM> detects the direction of the head of a user <NUM> who wears a head mounted display <NUM>, and a three-dimensional rehabilitation action.

The display controller <NUM> generates, in a three-dimensional virtual space <NUM>, avatar objects <NUM> that move in accordance with a detected rehabilitation action and a target object <NUM> to be visually recognized by the user <NUM>. The display controller <NUM> then displays the objects on the head mounted display <NUM> in accordance with the direction of the head of the user <NUM> detected by the detector <NUM>.

The notifier <NUM> notifies the user of the generation of the target object <NUM>. For example, in <FIG>, the notifier <NUM> displays, as a notification image <NUM>, a radar screen image used to notify the user of the generation position of the target object <NUM>. The radar screen image indicates the direction in which the target object <NUM> is located relatively with respect to a reference direction (initially set in the front direction of the user who is sitting straight on a chair) in the virtual space. The radar screen image also indicates the direction in which the head of the user <NUM> is located relatively with respect to the reference direction.

The cognitive load controller <NUM> generates the target object <NUM> after the elapse of a predetermined time from the timing of notifying the generation of the target object <NUM>, thereby giving a cognitive load to the user <NUM>.

As described above, according to this example embodiment, the user needs to continuously memorize and hold a rehabilitation action that he/she should perform after notification of generation of the target object until actual generation of the target object. It is therefore possible to quantitatively adjust and control a cognitive load that should be subjected to information processing by the brain and then give it to the user. For example, when the type, number, size, spatial spread, position, amount, and the like of information included in a notification image or a notification sound are adjusted, a cognitive load that has increased the complexity of information to be memorized and held can be given.

A rehabilitation support system <NUM> according to the second example embodiment of the present invention will be described with reference to <FIG> is a view for explaining the configuration of the rehabilitation support system <NUM> according to this example embodiment.

As shown in <FIG>, the rehabilitation support system <NUM> includes a rehabilitation support apparatus <NUM>, two base stations <NUM> and <NUM>, a head mounted display <NUM>, and two controllers <NUM> and <NUM>. A user <NUM> sitting on a chair <NUM> twists the upper half body or stretches the hands in accordance with display on the head mounted display <NUM>, thereby making a rehabilitation action. In this example embodiment, a description will be made assuming rehabilitation performed while sitting on a chair. However, the present invention is not limited to this. Rehabilitation may be performed while standing, walking, running, or making another specific action. In addition, the controllers may be held on or attached to body parts other than hands, such as feet or trunk.

The two base stations <NUM> and <NUM> sense the motion of the head mounted display <NUM> and the motions of the controllers <NUM> and <NUM>, and send these to the rehabilitation support apparatus <NUM>. The rehabilitation support apparatus <NUM> performs display control of the head mounted display <NUM> while evaluating the rehabilitation action of the user <NUM>. Note that the head mounted display <NUM> can be of a non-transmissive type, a video see-through type, or optical see-through type. In this example embodiment, a virtual space of VR (Virtual Reality) is presented to the user. However, a physical space and a virtual space may be displayed in a superimposed manner, like AR (Augmented Reality), or physical information may be reflected on a virtual space, like MR (Mixed Reality).

In this example embodiment, as an example of a sensor configured to detect the position or action of the hand or head of the user, the controllers <NUM> and <NUM> held in the hands of the user <NUM>, and the base stations <NUM> and <NUM> have been described. However, the present invention is not limited to this. A camera (including a depth sensor) configured to detect the positions or action of the hands of the user by image recognition processing, a sensor configured to detect the positions of the hands of the user by a temperature, a wristwatch-type wearable terminal put on an arm of the user, a motion capture device, or the like can be applied to the present invention.

The rehabilitation support apparatus <NUM> includes an action detector <NUM>, display controllers <NUM> and <NUM>, an evaluator <NUM>, an updater <NUM>, a task set database <NUM>, and an operation unit <NUM>.

The action detector <NUM> acquires, via the base stations <NUM> and <NUM>, the positions of the controllers <NUM> and <NUM> held in the hands of the user <NUM>, and detects the rehabilitation action of the user <NUM> based on changes in the positions of the hands of the user <NUM>.

The display controller <NUM> generates, in a virtual space, avatar objects <NUM> that move in accordance with a detected rehabilitation action and a target object <NUM> representing the target of the rehabilitation action. The display controller <NUM> displays, on a display screen <NUM>, the images of the avatar objects <NUM> and the target object <NUM> in accordance with the direction and position of the head mounted display <NUM> detected by the action detector <NUM>. The images of the avatar objects <NUM> and the target object <NUM> are superimposed on a background image <NUM>. Here, the avatar objects <NUM> have the same shape as the controllers <NUM> and <NUM>. However, the present invention is not limited to this, and the size, shape, or color may be changed on the left and right sides. The avatar objects <NUM> move in the display screen <NUM> in accordance with the motions of the controllers <NUM> and <NUM>. The controllers <NUM> and <NUM> are each provided with at least one button and configured to perform various kinds of settings by operating the button. The background image <NUM> is cut out from a virtual space including a horizontal line <NUM> and a ground surface image <NUM>.

The display controller <NUM> displays the target object <NUM> while gradually changing its display position and size such that it falls downward from above the user <NUM>. The user <NUM> moves the controllers <NUM> and <NUM> to make the avatar objects <NUM> in the screen close to the target object <NUM>. Note that as for the moving direction of the target object, for example, the target object may be displayed such that it rises from the floor surface to above the head, or movement in the depth direction may occur in addition to the movement in the vertical direction.

If the shortest distance between the target object <NUM> and a sensor object (not shown here) included in the avatar object <NUM> falls within a predetermined range, the target is achieved, and the target object <NUM> disappears. At this time, if the shortest distance between the target object <NUM> and the sensor object included in the avatar object <NUM> is equal to or less than a first threshold, "excellent" is displayed because of the complete achievement of the target, and a corresponding voice is output to make feedback. The controllers <NUM> and <NUM> may simultaneously be vibrated.

On the other hand, if the shortest distance between the target object <NUM> and the sensor object included in the avatar object <NUM> is not less than the first threshold and not more than a second threshold, "well done" is displayed because of the achievement of the target, and a corresponding voice is output to make feedback. The controllers <NUM> and <NUM> may simultaneously be vibrated.

The action of the user <NUM> to move the controllers <NUM> and <NUM> is the rehabilitation action, and the display of the target object that urges the user <NUM> to do one rehabilitation action he/she should make is called a task. Information (task data) representing one task includes the appearance direction of the target object, the shape, the appearance position (the distanced from the user), the appearance interval (time interval), the moving speed in falling or rising, the size, the color, which one of the left and right controllers should be used to acquire a target object, the number of target objects that appear simultaneously, the size of the sensor object, and the like.

That is, the display controller <NUM> can also change (for example, in three levels), in accordance with task data, the distance from the user <NUM> to the fall position of the target object <NUM> in the depth direction. For example, a change can be made such that the target object falls quite near the user <NUM> or falls to a position that the user <NUM> cannot reach unless largely inclining the body forward. This can control an exercising load to be given to the user.

The display controller <NUM> displays a radar screen image <NUM> on the display screen <NUM> of the head mounted display <NUM>. The radar screen image <NUM> is a notification image used to notify the user of generation of the target object <NUM>. The radar screen image <NUM> notifies the user of the direction in which the generated target object <NUM> is located relatively with respect to a reference direction (initially set in the front direction of the user who is sitting straight on a chair) in the virtual space. The radar screen image <NUM> also notifies the user how far the position of the generated target object <NUM> is apart from the user <NUM>. Note that the notification image is not limited to the radar screen image, and the notification may be made using characters, an arrow, a symbol, an illustration, or a type, intensity, blinking, or the like of light or a color. The notification method is not limited to the image, and may use a voice, a vibration, or a combination of some of a voice, a vibration, and an image.

Independently of the direction of the head of the user <NUM>, the display controller <NUM> displays the radar screen image <NUM> at the center (for example, within the range of -<NUM>° to <NUM>°) of the display screen <NUM> of the head mounted display <NUM>. However, the display portion is not limited to the center, and may be, for example, an arbitrary place on the four corners, the upper end, the lower end, the left end, and the right end of the screen.

The radar screen image <NUM> includes a head image <NUM> representing the head of the user viewed from above, a block image <NUM> obtained by dividing the periphery of the head image <NUM> into a plurality of blocks, and a fan-shaped image <NUM> as a visual field image representing the visual field of the user. A target position image representing the position of a target object is shown by coloring, blinking, or lighting a block in the block image <NUM>. This allows the user <NUM> to know whether the target object exists on the left side or the right side with respect to the direction in which he/she faces. Note that in this example embodiment, the block image <NUM> is fixed, and the fan-shaped image <NUM> moves. However, the present invention is not limited to this, and the block image <NUM> may be moved in accordance with the direction of the head while fixing the fan-shaped image <NUM> and the head image <NUM>. More specifically, if the head turns to the left, the block image <NUM> may rotate to right.

The evaluator <NUM> evaluates the rehabilitation action of the user in accordance with the amount and quality of the task achieved by the user <NUM> and adds a point. Here, the quality of the achieved task depends on "well done" or "excellent", that is, how close the avatar object could be brought to the target object. The evaluator <NUM> adds different points to achieved tasks (a high point to a far object, and a low point to a close object).

The updater <NUM> updates a target task in accordance with the integrated point. For example, a target task may be updated using a task achievement ratio (the number of achieved targets/the number of tasks).

The task set database <NUM> stores a set of a plurality of tasks. The task set database <NUM> stores a task set that decides the order of providing the plurality of tasks to the user.

For example, task sets may be stored as templates for each hospital, or a history of executed task sets may be stored for each user. The rehabilitation support apparatus <NUM> may be configured to be communicable with another rehabilitation support apparatus via the Internet. In this case, one task set may be executed by the same user in a plurality of places, or various templates may be shared by a plurality of users in remote sites.

The operation unit <NUM> is provided to operate display control in the display controller <NUM> and the display controller <NUM>.

A cognitive load controller <NUM> generates the target object <NUM> after the elapse of a predetermined time from the timing of notifying the generation of the target object <NUM>, thereby giving a cognitive load to the user <NUM>. That is, the user needs to continuously memorize and hold an action that he/she should perform after he/she knows that the target object will be generated until actual generation of the target object. The "request to memorize" is a cognitive load for the user. The cognitive load controller <NUM> may control the cognitive load by changing a predetermined time from the timing of notifying the generation of the target object <NUM> to the timing of generating the target object <NUM>.

Also, the cognitive load controller <NUM> may control the cognitive load by changing the time not "until the timing of generating the target object <NUM>" but "until the target object <NUM> approaches the range the user <NUM> can reach".

The cognitive load controller <NUM> may give a cognitive load to the user <NUM> by displaying the background image <NUM> other than the target object <NUM> on the head mounted display <NUM>.

<FIG> is a view showing a task table <NUM> stored in the task set database <NUM>. In the task table <NUM>, a time (task generation timing) <NUM>, a task interval <NUM>, a task position <NUM>, a task angle <NUM>, and a task distance (intensity) <NUM> are stored in linkage with a task ID. Also, in the task table <NUM>, a target object speed <NUM>, a perfect determination "excellent evaluation" criterion <NUM>, a good determination (well done evaluation) criterion <NUM>, a sensor object size <NUM>, and the like are stored in linkage with a task ID. In addition to these, a delay time (predetermined time) from task generation notification to task generation may be set for each task.

<FIG> are views showing examples of display on the display screen <NUM> according to this example embodiment. In <FIG> and <FIG>, on a background image <NUM> representing a field, an image of a person representing a farmer is displayed as a trigger object <NUM> serving as a trigger of target object appearance. That is, the display controller <NUM> displays the trigger object <NUM> as a notification image used to notify the user of generation of the target object <NUM>.

When a predetermined time elapses after the trigger object <NUM> throws up a target object <NUM> in the shape of a potato, a target object <NUM> having the shape of a large potato appears from the upper side of the screen, as shown in <FIG>. When the falling target object <NUM> is received by moving an avatar object <NUM> having the shape of a basket, the task is achieved. The left and right avatar objects <NUM> move on the screen in synchronism with the motions of the controllers <NUM> and <NUM>.

The cognitive load controller <NUM> generates the target object <NUM> after the elapse of a predetermined time from the timing at which the trigger object <NUM> throws up the target object <NUM> and notifies the generation of the target object <NUM>, thereby giving a cognitive load to the user <NUM>. Note that in synchronism with the motion of the trigger object <NUM>, generation of the target object may be notified at the same timing using a radar chart type notification image, or a notification by a voice may be combined.

In this way, the cognitive load controller <NUM> displays not a background including only a horizontal line as shown in <FIG> but a background image including a large quantity of information as shown in <FIG> and <FIG>, thereby giving a cognitive load to the user. That is, it is made difficult to memorize that the target object <NUM> has appeared, and the position to which the target object <NUM> is expected to fall, thereby giving a load closer to a cognitive load necessary in a real life to the user of rehabilitation.

In particular, the cognitive load controller <NUM> changes at least a part of the background image <NUM> along with time, thereby giving a cognitive load to the user <NUM>. In the example shown in <FIG>, for example, in the background image <NUM>, a cloud <NUM> may be moved, plants <NUM> may be shaken, or an animal (not shown) irrelevant to the target object may be made to appear. This can impede concentration to the target object <NUM> and make it more difficult for the user <NUM> to memorize the position to which the target object <NUM> is expected to fall. More technically, it can be said that information irrelevant to the task is displayed on the background image to prepare an environment in which it is difficult to concentrate to the target object and intentionally cause an attention disorder (more specifically, a selective attention disorder, a divided attention disorder, an alternating attention disorder, or a sustained attention disorder), thereby making memorization difficult and controlling the cognitive load.

<FIG> is a view showing another example of display on the display screen <NUM> according to this example embodiment. In <FIG>, in a background image <NUM> like woods, a trigger object <NUM> representing a monkey and a target object <NUM> representing an apple are displayed. When the trigger object <NUM> representing a monkey drops the target object <NUM> representing an apple from a tree, and the target object <NUM> approaching the user is received by moving an avatar object <NUM> representing a basket, the task is achieved. In this case as well, the cognitive load controller <NUM> starts dropping the target object <NUM> after the elapse of a predetermined time from the timing at which the trigger object <NUM> shakes the tree and notifies the generation of the target object <NUM>, thereby giving a cognitive load to the user <NUM> while causing an attention disorder.

Also, the cognitive load controller <NUM> causes at least two target objects <NUM> to exist in the three-dimensional virtual space and display these on the display screen <NUM>, thereby giving a cognitively stronger load to the user <NUM>. In other words, the cognitive load controller <NUM> generates the at least two target objects <NUM> at different positions in the left-and-right direction in the three-dimensional virtual space. That is, since the user <NUM> needs to memorize a plurality of points concerning at which position the trigger object <NUM> shakes the tree, a cognitively stronger load is given.

In particular, since the at least two target objects <NUM> are generated at a plurality of positions in a direction (the left-and-right direction in <FIG>) different from the moving direction (the falling direction in <FIG>) of the target object <NUM>, a larger cognitive load can be given. That is, since the user <NUM> needs to move the controllers <NUM> and <NUM> in consideration of the movement in the vertical direction, the difference between the generation positions in the left-and-right direction, and the difference in the falling position in the depth direction, the spatial cognitive ability is also tested. As described above, when the type, number, size, spatial spread, position, amount, and the like of information included in a notification image including a trigger object or a notification sound are adjusted in addition to the change of the predetermined time of the task, it is possible to quantitatively adjust and control the complexity of information to be memorized and held, that is, a cognitive load that should be subjected to information processing by the brain of the user.

The evaluator <NUM> evaluates the cognitive ability of the user based on whether the avatar object has reached, in a good timing, a three-dimensional target position represented by the target object, the time interval from target object generation notification to generation and the number of target objects, the degree of the attention disorder caused by the background image, and the like.

As described above, according to this example, it is possible to effectively control the load to be given to the cognitive function of the user in rehabilitation. More specifically, the memory and the cognitive processing ability of the user are tested quite naturally in a very easy task to receive a ball, a potato, or an apple, instead of memorizing something in accordance with an instruction for memorizing. For this reason, the psychological burden is small, and a rejection or an adverse reaction to rehabilitation, or a depressive reaction in case of a failure hardly occurs. By the rehabilitation action according to this example embodiment, it is possible to recover a so-called cognitive ability and a higher brain function (disorder) including a memory, a cognitive processing ability, a spatial cognitive ability, and an attention function (disorder) together with an exercise capacity and make a guidance such that the user can live his/her everyday life comfortably.

Note that in the example embodiment, a head mounted display is used. However, an eyeglass type display suffices. In place of the head mounted display, a hologram or a large TV type display may be arranged on the periphery of the user to urge the user to make a rehabilitation action.

Claim 1:
A rehabilitation support apparatus (<NUM>, <NUM>) for recovering one of cognitive impairment and a higher brain dysfunction, comprising:
a detector (<NUM>, <NUM>) that detects a three-dimensional rehabilitation action of a user;
a display controller (<NUM>, <NUM>) that generates, in a three-dimensional virtual space, an avatar object that moves in accordance with the detected rehabilitation action and a target object to be visually recognized by the user and displays the avatar object and the target object on a display;
an evaluator (<NUM>) that evaluates the rehabilitation action of the user in accordance with how close the avatar object could be brought to the target object;
a notifier (<NUM>, <NUM>) that notifies a generation position, in the three-dimensional virtual space, of the target object; and
a cognitive load controller (<NUM>, <NUM>) that gives a cognitive load to the user by generating the target object after an elapse of a predetermined time from the notification of the generation position of the target object and changes the predetermined time to control the cognitive load to be given to the user.