Patent Description:
In the above technical field, patent literature <NUM> discloses a system configured to assist rehabilitation performed for a hemiplegia patient suffering apoplexy or the like.

In the technique described in the above patent literature <NUM>, however, it is impossible to perform active target updating according to an action of a user, and the same load needs to be repeated for any user.

The present invention enables to provide a technique of solving the above-described problem.

The dependent claims define embodiments of the invention. The invention as claimed is best understood in light of the embodiment described in the context of <FIG>. Other embodiments described herein do not necessarily describe the complete combination of features as claimed, but are useful for understanding the invention as claimed.

According to the present invention, it is possible to perform active target updating according to the rehabilitation action of a user.

Example embodiments will now be described in detail with reference to the drawings.

A rehabilitation assistance system <NUM> according to the first example embodiment of the present invention will be described with reference to <FIG>.

As shown in <FIG>, the rehabilitation assistance system <NUM> includes an action detector <NUM>, a display controller <NUM>, an evaluator <NUM>, and an updater <NUM>.

The action detector <NUM> detects a rehabilitation action of a user <NUM>. The display controller <NUM> displays an avatar image that moves in accordance with the detected rehabilitation action and a target image representing the target of the rehabilitation action.

The evaluator <NUM> evaluates the rehabilitation ability of the user in accordance with the difference between the rehabilitation action and a target position represented by the target image. The updater <NUM> updates the target position in accordance with the evaluation result by the evaluator <NUM>.

The action detector <NUM> further detects a second rehabilitation action of the user during a first rehabilitation action. When a predetermined or more evaluation is made only for the first rehabilitation action, the evaluator <NUM> evaluates the rehabilitation ability based on both the first rehabilitation action and the second rehabilitation action. This makes it possible to perform active and proper target updating according to the rehabilitation action of the user.

A rehabilitation assistance system <NUM> according to the second example embodiment of the present invention will be described next with reference to <FIG> is a view for explaining the arrangement of the rehabilitation assistance system according to this example embodiment.

As shown in <FIG>, the rehabilitation assistance system <NUM> includes a rehabilitation assistance server <NUM>, two base stations <NUM> and <NUM>, a head mounted display <NUM>, and two controllers <NUM> and <NUM>. Note that the head mounted display <NUM> can be any one of a nontransparent type, a video see-through type, and an optical see-through type.

In addition, the rehabilitation assistance server <NUM> includes an action detector <NUM>, a display controller <NUM>, an evaluator <NUM>, an updater <NUM>, a voice input/output unit <NUM>, a target database <NUM>, and a background image + question/answer database <NUM>.

The action detector <NUM> acquires the positions of the controllers <NUM> and <NUM> in the hands of a user <NUM> and the position of the head mounted display <NUM> via the base stations <NUM> and <NUM>, and detects the rehabilitation action of the user <NUM> based on changes in the positions.

The display controller <NUM> causes the head mounted display <NUM> to display an avatar image that moves in accordance with the detected rehabilitation action and a target image representing the target of the rehabilitation action. <FIG> is a view showing an example of avatar images <NUM> and <NUM> in a screen <NUM> displayed on the head mounted display <NUM>. The avatar images <NUM> and <NUM> are displayed on a background image <NUM> in a superimposed manner. In this example, the avatar images <NUM> and <NUM> have the same shapes as the controllers <NUM> and <NUM> and move in the screen <NUM> in accordance with the motions of the controllers <NUM> and <NUM>. Additionally, a background image <NUM> changes depending on the position and orientation of the head mounted display <NUM>. As shown on the avatar images <NUM> and <NUM>, buttons are prepared on the controllers <NUM> and <NUM>, and the controllers <NUM> and <NUM> are configured to be able to do various kinds of setting operations and the like. Here, a landscape video (for example, a movie obtained by capturing a street in New York) obtained by capturing an actual landscape is displayed as the background image <NUM>. As the landscape video, a video of a road around the rehabilitation facility may be used. This makes the user feel to take a walk in a foreign country or feel to stroll in a familiar place. When the landscape video is superimposed, training in an enormous information amount can be implemented while entertaining the patient.

In addition, for example, as shown in <FIG>, the display controller <NUM> displays an object <NUM> superimposed on the background image <NUM> in screens <NUM> to <NUM> of the head mounted display <NUM>. The object <NUM> is displayed while gradually changing its display position and size such that it appears to be falling downward from overhead of the user <NUM>. The user <NUM> moves the controllers <NUM> and <NUM> to bring the avatar image <NUM> in the screen close to the object <NUM>. When the avatar image <NUM> hits the object <NUM>, the object <NUM> disappears. In the screens <NUM> to <NUM>, characters "left" near the avatar image <NUM> of the sensor means touching the object <NUM> with the left hand.

The evaluator <NUM> compares the rehabilitation action detected by the action detector <NUM> and the target position represented by the target image displayed by the display controller <NUM>, and evaluates the rehabilitation ability of the user. More specifically, the evaluator <NUM> decides, by comparing the positions in a three-dimensional virtual space, whether the avatar image <NUM> that moves in correspondence with the rehabilitation action detected by the action detector <NUM> overlaps the object <NUM> serving as the target image. As a result, if these overlap, the evaluator <NUM> evaluates that one rehabilitation action is completed, and adds a point. As for the position of the object <NUM> in the depth direction, various steps (for example, three steps) are prepared and set to different points (a high point for a far object, and a low point for a close object), respectively.

The updater <NUM> updates the target task in accordance with the integrated point. For example, the target task may be updated using a task achievement ratio (number of achieved targets/number of tasks) or the like.

<FIG> is a flowchart showing the procedure of processing in the rehabilitation assistance server <NUM>. In step S501, as calibration processing, the target of the rehabilitation action is initialized in accordance with the user. More specifically, each patient is first caused to do a work in an action range as calibration, it is set to the initial value, and the target is initialized in accordance with the user.

In addition, a target according to the attribute information (for example, whether the user is an athlete or suffers from the Parkinson disease) of the user is set by referring to the target database <NUM>. For example, in a case of an injured athlete, an initial value not to make the injury worse is set. In a case of a user suffering from the Parkinson disease, an exercise to make the disease progress slow is set to the initial value. Furthermore, each patient is first caused to do a work in an action range, it is set to the initial value, and the target is initialized in accordance with the user.

Next, in step S503, the avatar images <NUM> and <NUM> are displayed in accordance with the positions of the controllers <NUM> and <NUM> detected by the action detector <NUM>. Furthermore, in step S505, the object <NUM> is displayed at a position and speed according to the set task.

In step S507, the motions of the avatar images <NUM> and <NUM> and the motion of the object <NUM> are compared, and it is determined whether the task is completed. If the task is not completed, the process directly returns to step S505, and the next object is displayed without changing the difficulty of the task.

If the task is completed, the process advances to step S509 to calculate an accumulated point, a task achievement probability, and the like. The process further advances to step S511 to compare the accumulated point, the task achievement probability, or the like with a threshold T. If the accumulated point, the task achievement probability, or the like exceeds the predetermined threshold T, the process advances to step S513 to update the exercise intensity of the task. If the accumulated point, the task achievement probability, or the like does not reach the threshold T, the process returns to step S505, and the next object is displayed without changing the difficulty of the task.

For example, when the achievement level in a short range exceeds <NUM>% (or a count such as <NUM> times may be used), the display frequency of an object in a middle range is raised. When the achievement level of the object in the middle range exceeds <NUM>% (or a count such as <NUM> times may be used), the display frequency of an object in a long range is raised. Conversely, if the achievement level is low, the target value may be set to the short range.

As for the task updating here as well, the task is changed in accordance with the attribute of the user (for example, whether the user is an injured athlete or a patient suffering from the Parkinson disease). As the task updating method, a method of switching the background image is also conceivable.

After the task is updated, the process advances to step S515, and the fatigue level of the user is calculated and compared with a threshold N. If the fatigue level exceeds the predetermined threshold, the "stop condition" is satisfied, and the processing is ended. For example, (fatigue level = <NUM> - collection ratio of closest objects) can be calculated. Alternatively, (fatigue level = <NUM>/eye motions) may be calculated. If it is obvious that the user is not concentrating (for example, the user is not searching for an object at all or does not move the head), it would be meaningless to continue the rehabilitation any more, and the user takes a break. In addition, the fatigue level may be calculated by detecting a decrease in the speed (acceleration) of stretching out the hand.

Additionally, for example, when the accumulated point exceeds a predetermined threshold, which one of the two, left and right controllers <NUM> and <NUM> should be used to touch the object <NUM> (right here) is instructed, as indicated by a character image <NUM> shown in <FIG>. This requires a cognitive function of recognizing a character, and also, the difficulty of the action rises, and an advanced motor function is necessary. That is, a dual task for the cognitive function and the motor function is required.

Note that in <FIG>, the instruction is made using a character. However, the present invention is not limited to this, and the instruction may be made by an arrow, a color, or a voice. As described above, in this example embodiment, the load is updated in accordance with the evaluation of the rehabilitation action.

An able-bodied person makes two or more actions simultaneously, for example, "walks while talking" in a daily life. Such "an ability to make two actions simultaneously" declines with age. For example, "stop when talked to during walking" occurs. It is considered that an elderly person falls not only because of "the deterioration of the motor function" but also because of involvement of such "decline in the ability to make two actions simultaneously". In fact, there are many elderly persons who are judged to have sufficiently recovered the motor function by rehabilitation but fall after returning to the home. One factor responsible to this is that the rehabilitation is performed in a state in which the environment and conditions to allow a person to concentrate on the rehabilitation action are organized. That is, a living environment includes factors that impede concentration on an action, and an action is often made under a condition that, for example, the view is poor, an obstacle exists, or consciousness is turned to a conversation.

Hence, it is considered that it is important to perform such rehabilitation that makes the user distract attention. It is preferable to give a specific dual task and perform training. Such a dual task training is an effective program not only to prevent a fall of an elderly person but also to prevent dementia.

The dual task training includes not only a training that combines a cognitive task and an exercise task but also a training that combines two types of exercise tasks.

As cognitive task + exercise task, a training such as walking while subtracting one by one from <NUM> can be performed. As exercise task + exercise task, a training such as walking without spilling water from a glass can be performed.

In a case in which the walking speed is lower about <NUM>% in a dual task walking test than in simple walking, the evaluator <NUM> evaluates that the risk of fall is high, and notifies the display controller <NUM> to repeat the dual task.

Note that the dual task is readily more effective to "a person having a relatively high moving ability". For example, for an elderly person who cannot move without a stick even indoors, strengthening the balance ability (muscle power, sense of equilibrium, and the like) is given higher priority than the dual task ability. Roughly judging, it can be expressed that the dual task ability is important for a person requiring support, and the balance ability other than the dual task ability is important for a person requiring care. A time-series change in calibration is displayed, and the improvement of the exercise range of the user is visually displayed.

For a patient expected to normally improve (a patient suffering from an orthopedic disease such as a bone fracture and assumed to completely improve), hardest rehabilitation actions are set to speed up the improvement.

For a patient whose degree of improvement changes individually (in a case of brain infarction or the like, a paralysis of a different form occurs depending on the morbid portion), the load of a task is improved to some extent, and the improvement of the load is stopped at a certain level.

In a case of a patient suffering from hypofunction in principle due to the Parkinson disease or the like, periodically evaluating the current exercise enable state is useful.

<FIG> is a view showing another example of an image for dual task training. A loser (bomb) is mixed among objects, thereby requiring the cognitive function. Alternatively, as shown in <FIG>, a question image (for example, multiplication, here) may be displayed on the background screen in a superimposed manner, and only acquisition of an object on which a correct answer is displayed may be evaluated. One of rock, scissors, and paper may be displayed on the background screen, and the user may be requested to collect an object on which a mark to win is displayed.

In addition, a number may be simply displayed on each object, and only acquisition of an object of a large number may be evaluated. Alternatively, a traffic signal may be displayed in the background image <NUM>, and when the user acquires an object at red light, the evaluator <NUM> may decrement the point.

According to this example embodiment, since the task is updated in accordance with the achievement level (for example, achievement probability) of the rehabilitation action, a load according to the degree of progress of rehabilitation can be given to the user. In addition, when the background image <NUM> is displayed, the patient can enjoy and also perform rehabilitation in a situation in which he/she turns consciousness to the periphery, and can implement a safer life when returning to the physical world.

<FIG> is a view showing still another example of dual task training. As shown in <FIG>, the voice input/output unit <NUM> outputs a question voice concerning the background image to a headphone <NUM> and acquires an answer to the question via a microphone <NUM> provided on the head mounted display <NUM>. The evaluator <NUM> performs voice recognition processing for the answer acquired as voice information, compares the answer with an answer prepared in advance, and evaluates the rehabilitation ability of the user in accordance with the comparison result.

<FIG> is a view showing an example of the contents of the background image + question/answer database <NUM>. A question voice, an answer, and a point are stored in association with a background movie.

As a reaction of the user, a result that the object collection ratio lowers is expected in a dual task. A result that the object collection achievement ratio does not change even when the dual task is displayed is expected as a goal. The object collection ratio or object reach ratio in a single task is compared with that in a dual task, and training is repetitively performed until the difference falls within a predetermined range.

Dual task training that simultaneously requires a motor function and a cognitive function has been described above. However, the present invention is not limited to this, and dual task training that simultaneously requires two motor functions may be performed.

For example, as shown in <FIG>, the user may be required to pick up the object <NUM> while getting out of the way of a flying object <NUM>. Whether the user has dodged the object <NUM> well can be determined by detecting the position of a sensor provided on the head mounted display <NUM>. Evaluation and task updating are performed based on the achievement points (for example, achievement ratios) of both of the two rehabilitation actions.

Additionally, for example, as indicated by an image <NUM> shown in <FIG>, glass images <NUM> and <NUM> with water may be displayed as avatar images that move in accordance with the actions of the controllers <NUM> and <NUM>, and the object <NUM> may be collected by moving the glass images <NUM> and <NUM>. However, as indicated by an image <NUM>, when a glass image <NUM> is tilted, and water spills, a point cannot be obtained even when the object <NUM> is collected by a glass image <NUM>. A point is added only when the object <NUM> is collected without spilling water from the glass images <NUM> and <NUM>, as indicated by an image <NUM>.

In addition, it can be considered that the user is required to cause the avatar image on the reverse side of the avatar image on the side of collecting the object to always touch a designated place. The user may be required to collect the object while pressing a designated one of the buttons provided on the controllers <NUM> and <NUM> a predetermined number of times. In addition, when a sensor configured to acquire the motion of a foot of the user is provided, the user may be required to move a designated foot.

A rehabilitation assistance system according to the third example embodiment of the present invention will be described next with reference to <FIG>. <FIG> is a view for explaining the outline of the operation of the second example embodiment of the rehabilitation assistance system. <FIG> is a view for explaining the outline of the operation of the rehabilitation assistance system according to the second example embodiment. The rehabilitation assistance system according to this example embodiment is different from the above-described second example embodiment in that a visual recognition support image that improves the recognizability (for example, visibility) of a target image is displayed. The rest of the components and operations is the same as in the second example embodiment. Hence, the same reference numerals denote the same components and operations, and a detailed description thereof will be omitted.

In the second example embodiment, the moving distance of an avatar image <NUM>, that is, an exercise distance <NUM> of a user <NUM> is measured based on the distance between a reference <NUM> and the sensor of the avatar image <NUM> (the head portion of the avatar image <NUM>). A target distance <NUM> that is a distance required to move an arm or the like by the user <NUM> is decided based on the distance between the reference <NUM> and a reference line <NUM> of an object <NUM> serving as a target image. As a rehabilitation exercise, the user <NUM> moves the avatar image <NUM> and brings it close to the object <NUM>.

However, as shown in <FIG>, when the avatar image <NUM> touches an apex <NUM> of the object <NUM>, the system judges that one of rehabilitation actions of the user <NUM> has ended, and displays the new object <NUM> as the next target.

The system provider side wants the avatar image <NUM> to touch the object <NUM> when the user <NUM> completely stretches out the arm as the rehabilitation exercise. However, if the size of the object <NUM> is large (the distance between the apex and the reference line <NUM> is long), it is determined that the avatar image <NUM> touches the object <NUM> when it just touches an edge of the object <NUM>. Hence, since the user <NUM> cannot move the arm by the initially assumed distance, the expected rehabilitation effect is difficult to obtain.

In addition, since the user <NUM> can touch the object <NUM> before he/she completely stretches out the arm, the feeling of achievement or feeling of satisfaction cannot sufficiently be obtained, and the motivation to rehabilitation may lower.

In this case, the exercise distance <NUM> that is the distance the avatar image <NUM> has actually moved deviates from the target distance <NUM> that is the distance the user <NUM> should move. For this reason, the user <NUM> cannot do the exercise through the exercise distance <NUM> set before the start of the rehabilitation, and the effect obtained by the rehabilitation is less than the expected effect.

For example, the length of one side of the object <NUM> is set to <NUM>, and a diameter <NUM> of the sensor portion of the avatar image <NUM> (the head portion of the avatar image <NUM>) is set to <NUM>. In this case, when the user <NUM> makes the avatar image <NUM> touch not the reference line <NUM> but the apex <NUM> of the object <NUM>, an error of about <NUM> is generated between the target distance <NUM> and the exercise distance <NUM>.

For this reason, since the user <NUM> does not move the avatar image <NUM> by the exercise distance <NUM> assumed before the start of the rehabilitation, the effect of the rehabilitation the user <NUM> should enjoy decreases.

On the other hand, if the object <NUM> is made small such that the user <NUM> can touch the object <NUM> by completely stretching out the arm, it becomes difficult for the user <NUM> to visually recognize the position of the object <NUM> in the screen. If the object <NUM> cannot be visually recognized, the rehabilitation cannot hold.

In this example embodiment, the sensor portion (reactive portion) of the avatar image <NUM> is formed into a region smaller than the head portion of the avatar image <NUM>. This can decrease the deviation (error) between the target distance <NUM> and the exercise distance.

<FIG> is a view for explaining the outline of the operation of the rehabilitation assistance system according to this example embodiment. In this example embodiment, the gradation at the center of the object <NUM> is darkened to form a reactive portion so no deviation occurs between the assumed target distance <NUM> and the exercise distance <NUM> of the avatar image <NUM>. Then, the gradation of the portion around the reactive portion of the object <NUM> is lightened. That is, the size of the object <NUM> shown in <FIG> and <FIG> is made small, and the object <NUM> is surrounded by a visual recognition support image <NUM> larger than the object <NUM>. That is, the object <NUM> and the visual recognition support image <NUM> are displayed in a superimposed manner.

Viewed from the user <NUM>, when the size of the object <NUM> is made small, the object <NUM> is difficult to see (the visibility lowers). However, to compensate for the decrease in the visibility, the visual recognition support image <NUM> is arranged around the object <NUM> that has become small.

For example, the length of one side of the object <NUM> is set to <NUM>, the length of one side of the visual recognition support image <NUM> is set to <NUM>, and the diameter of a sensor portion <NUM> of the avatar image <NUM> is set to <NUM>. Then, the error (deviation) between the target distance <NUM> and the exercise distance <NUM> decreases to about <NUM>.

This can make it possible to decrease the deviation (error) between the target distance <NUM> and the exercise distance <NUM> while preventing the visibility of the object <NUM> from lowering due to the gradation difference and the size difference between the object <NUM> and the visual recognition support image <NUM>. Additionally, as a secondary effect, the degree of experience obtained by bringing the avatar image <NUM> into contact with the object <NUM> increases. That is, the sensation of touching the object <NUM> is clear for the user <NUM>, and the joy in achieving the target of the rehabilitation also increases.

<FIG> is a view for explaining the arrangement position of a visual recognition support image in the rehabilitation assistance system according to this example embodiment. In <FIG>, the object <NUM> serving as the target image is displayed so as to be included in the visual recognition support image <NUM>, and is also arranged near the center of the visual recognition support image <NUM>.

However, as shown in <FIG>, the object <NUM> may be arranged near the lower side of the visual recognition support image <NUM> on the near side. That is, the object <NUM> may be arranged on the near side viewed from the user <NUM>. In this way, the object <NUM> can be arranged at any position in the visual recognition support image <NUM> as long as it is displayed inside the visual recognition support image <NUM>. When the size of the object <NUM> is made small, and the deviation between the target distance <NUM> and the exercise distance <NUM> is decreased, the visibility of the object <NUM> lowers. Hence, to improve the visibility of the object <NUM>, the visual recognition support image <NUM> larger than the object <NUM> is displayed around the object <NUM>, thereby compensating for the decrease in the visibility of the object <NUM>. Note that the visual recognition support image <NUM> used to improve the visibility of the object <NUM> is not limited to a cube, as shown here, obtained by increasing the magnification of the cubic object <NUM>.

Other shapes of the visual recognition support image <NUM> will be described next with reference to <FIG>. <FIG> is a view for explaining another example of the visual recognition support image in the rehabilitation assistance system according to this example embodiment. <FIG> is a view for explaining still another example of the visual recognition support image in the rehabilitation assistance system according to this example embodiment. <FIG> is a view for explaining still another example of the visual recognition support image in the rehabilitation assistance system according to this example embodiment. <FIG> is a view for explaining still another example of the visual recognition support image in the rehabilitation assistance system according to this example embodiment. <FIG> is a view for explaining still another example of the visual recognition support image in the rehabilitation assistance system according to this example embodiment.

As shown in <FIG>, a visual recognition support image <NUM> may have, for example, an arrow shape representing the existence position of the object <NUM>. The object <NUM> is not included in the arrow-shaped visual recognition support image <NUM>. That is, the object <NUM> serving as the target image and the visual recognition support image <NUM> are not displayed in a superimposed manner, and the visual recognition support image <NUM> is displayed outside the object <NUM>. In this way, when the arrow-shaped visual recognition support image <NUM> is used, the user <NUM> can easily recognize that the object <NUM> exists at the tip of the arrow.

As shown in <FIG>, a visual recognition support image <NUM> may have a shape for attracting the attention of the user <NUM>. Note that the shape for attracting the attention of the user <NUM> is not limited to the shape shown in <FIG> and may be, for example, a star shape, a cross shape, a polygonal shape, or the like. In addition, a vertical line <NUM> and a horizontal line <NUM> may be displayed together to indicate that the object <NUM> is arranged at the intersection of the vertical line <NUM> and the horizontal line <NUM>.

As shown in <FIG>, a visual recognition support image <NUM> may be an alternate long and short dashed line extending from the sensor portion <NUM> of the avatar image <NUM> to the object <NUM>. Note that the visual recognition support image <NUM> is not limited to the alternate long and short dashed line and may be, for example, a straight line, an alternate long and two short dashed line, a dotted line, or the like.

Using the alternate long and short dashed line of the visual recognition support image <NUM> as a guideline, the user <NUM> moves the line of sight along the alternate long and short dashed line and visually recognizes the object <NUM>, thereby recognizing the existence position of the object <NUM>. Furthermore, when the avatar image <NUM> is moved along the alternate long and short dashed line, the user can make the avatar image <NUM> touch the object <NUM>. Note that when the visual recognition support image <NUM> is displayed together with a visual recognition support image <NUM>, the visibility of the object <NUM> further improves.

As shown in <FIG>, the visual recognition support image <NUM> may have a plurality of arrows arranged on a straight line from the sensor portion <NUM> to the object <NUM>. Using the plurality of arrows as a guideline, the user <NUM> moves the line of sight along the plurality of arrows and visually recognizes the object <NUM>, thereby recognizing the existence position of the object <NUM>. Furthermore, when the avatar image <NUM> is moved along the plurality of arrows, the user can make the avatar image <NUM> touch the object <NUM>. Note that when the cubic visual recognition support image <NUM> is displayed together with the visual recognition support image <NUM>, the visibility of the object <NUM> further improves.

As shown in <FIG>, a plurality of spherical visual recognition support images <NUM> are arranged at positions on the upper, lower, left, and right sides of the object <NUM>. That is, in <FIG>, the plurality of spherical visual recognition support images <NUM> are arranged around the object <NUM> such that the object <NUM> is arranged at the center of the four visual recognition support images <NUM>. Note that the shape of the visual recognition support image <NUM> is not limited to the spherical shape and may be, for example, a triangular shape, a rectangular shape, a polygonal shape, a star shape, or the like.

<FIG> is a view for explaining still another example of the visual recognition support image in the rehabilitation assistance system according to this example embodiment. The rehabilitation assistance server changes the size of the visual recognition support image <NUM> displayed on a display unit <NUM> in accordance with the degree of progress of rehabilitation of the user <NUM>. For example, the rehabilitation assistance server displays the large visual recognition support image <NUM> at the initial stage of the rehabilitation. In a state in which the rehabilitation of the user <NUM> has progressed, the size of the visual recognition support image <NUM> is reduced in accordance with the degree of progress of rehabilitation.

In addition, in an embodiment not falling under the scope of the claims, the rehabilitation assistance server may change the size of the visual recognition support image <NUM> not in accordance with the degree of progress of rehabilitation of the user <NUM> but in accordance with, for example, the eyesight of the user <NUM>. That is, the rehabilitation assistance server displays the large visual recognition support image <NUM> for the user <NUM> with poor eyesight, and displays the small visual recognition support image <NUM> for the user <NUM> with relatively good eyesight. In this way, the rehabilitation assistance server may display the visual recognition support image having a size according to the eyesight of the user <NUM>.

Additionally,, for example, if the user <NUM> has dementia, the rehabilitation assistance server may display the visual recognition support image <NUM> having a size according to the degree of progress of dementia or the cognitive function. Note that the size of the visual recognition support image <NUM> may be changed automatically by the rehabilitation assistance server, or may be changed manually by an operator such as a doctor who operates the rehabilitation assistance system and changed by the user <NUM>.

<FIG> is a block diagram for explaining the arrangement of the rehabilitation assistance system according to this example embodiment. A rehabilitation assistance system <NUM> includes a rehabilitation assistance server <NUM> and the display unit <NUM>. Note that the elements included in the rehabilitation assistance system <NUM> are not limited to these. The rehabilitation assistance server <NUM> includes an action detector <NUM>, a display controller <NUM>, an evaluator <NUM>, and an updater <NUM>.

The action detector <NUM> acquires the position of a controller in the hand of the user <NUM> and the position of a head mounted display or the like worn by the user <NUM>, and detects the motion (rehabilitation action) of the user <NUM> based on changes in the acquired positions.

The display controller <NUM> causes the display unit <NUM> to display the avatar image <NUM> that moves in accordance with the detected rehabilitation action, the target image representing the target of the rehabilitation action, and at least one visual recognition support image <NUM> used to improve the visibility of the target image.

The display controller <NUM> displays the target image and the visual recognition support image <NUM> in a superimposed manner. For example, the size of the target image is made smaller than the size of the visual recognition support image <NUM>, and the target image is displayed such that it is included in the visual recognition support image <NUM>.

The display controller <NUM> may display the target image, for example, near the center of the visual recognition support image <NUM>. In addition, the display controller <NUM> may display the target image not near the center of the visual recognition support image <NUM> but at a position included in the visual recognition support image <NUM> and on a side close to the avatar image <NUM>, that is, on the near side when viewed from the user <NUM>.

The display controller <NUM> may identifiably display the object <NUM> and the visual recognition support image <NUM>. More specifically, for example, the gradation of the object <NUM> is displayed darker than the gradation of the visual recognition support image <NUM>. Since the object <NUM> is displayed darker, a contrast difference is generated with respect to the visual recognition support image <NUM> displayed lighter, and the user <NUM> can reliably recognize the object <NUM>. Note that how to apply gradation to the object <NUM> and the visual recognition support image <NUM> is not limited to the method described here. For example, gradation may be applied such that even the user <NUM> with poor eyesight can reliably identify the object <NUM> and the visual recognition support image <NUM>.

In addition, the display controller <NUM> displays the object <NUM> and the visual recognition support image <NUM> in different colors so as to identifiably display the object <NUM> and the visual recognition support image <NUM>. The display controller <NUM> applies, for example, a dark color to the object <NUM> and a light color to the visual recognition support image <NUM>. However, the combination (pattern) of applied colors is not limited to this. For example, a combination of colors that allow even the user <NUM> with color anomaly (color blindness) to reliably identify the object <NUM> and the visual recognition support image <NUM> may be used. Furthermore, the display controller <NUM> may perform coloring capable of coping with the users <NUM> of various types such as weak eyesight, narrowing of visual field, and color anomaly. Note that the colors to be applied to the object <NUM> and the visual recognition support image <NUM> may be selected by the user <NUM> or may be selected by an operator such as a doctor.

Note that the gradations and colors of the object <NUM> and the visual recognition support image <NUM> have been described here. The gradations and colors may similarly be changed for the other visual recognition support images <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> as well.

Furthermore, the display controller <NUM> controls the change of the display of the visual recognition support image <NUM> in accordance with at least one of the eyesight of the user <NUM> and the evaluation result of the evaluator <NUM>. For example, the display controller <NUM> changes the size of the visual recognition support image <NUM> in accordance with the eyesight of the user <NUM>, the degree of progress of the rehabilitation of the user <NUM>, the degree of progress of the dementia of the user <NUM>, or the like.

The evaluator <NUM> compares the rehabilitation action detected by the action detector <NUM> and the target position represented by the object <NUM> serving as the target image displayed by the display controller <NUM> and evaluates the rehabilitation ability of the user <NUM>.

The updater <NUM> updates the target position represented by the object <NUM> in accordance with the evaluation result of the evaluator <NUM>.

The display unit <NUM> displays the target image, the visual recognition support image, and the like under the control of the display controller <NUM>. The display unit <NUM> is a head mounted display, a display, a screen, or the like but is not limited to these.

<FIG> is a view for explaining an example of a patient table provided in the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. A patient table <NUM> stores attribute information <NUM>, a rehabilitation target <NUM>, a current level <NUM>, and a rehabilitation menu <NUM> in association with a patient ID (Identifier) <NUM>. The patient ID <NUM> is an identifier used to identify a patient. The attribute information <NUM> is information representing attributes such as the age and sex of the patient. The rehabilitation target <NUM> is data representing which part of the body of the patient is the target of rehabilitation, for example, data representing a body part such as an arm or a leg.

The current level <NUM> is data representing the current rehabilitation level of the patient. That is, the current level <NUM> is data representing the degree of progress or the like of the rehabilitation of the patient. The data is data dividing rehabilitation stages from the initial stage to the final stage into a plurality of ranks, for example, A rank, B rank, and the like. Note that the rehabilitation level division method is not limited to this. The rehabilitation menu <NUM> is information concerning the menu of rehabilitation that the patient should undergo.

Next, <FIG> is a view for explaining an example of a display parameter table provided in the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. A display parameter table <NUM> stores a target image ID <NUM>, a visual recognition support image ID <NUM>, and a display parameter <NUM> in association with the rehabilitation menu <NUM>.

The target image ID <NUM> is an identifier used to identify the object <NUM> to be displayed on the display unit <NUM>. The visual recognition support image ID <NUM> is an identifier used to identify the visual recognition support image <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> to be displayed on the display unit <NUM>. The display parameter <NUM> is a parameter necessary for displaying the object <NUM> or the visual recognition support image <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> on the display unit <NUM>. The display parameter <NUM> includes, for example, pieces of information such as a position and a magnification. However, the pieces of information included in the display parameter <NUM> are not limited to these.

<FIG> is a view for explaining an example of an image table provided in the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. An image table <NUM> stores image data <NUM>, a display position <NUM>, and a magnification <NUM> in association with an image type <NUM>. Note that the items stored in the image table <NUM> are not limited to these.

The image type <NUM> is information for discriminating whether the image to be displayed is a target image or a visual recognition support image. The image data <NUM> is the image data of the object <NUM> or the visual recognition support image <NUM> to be displayed on the display unit <NUM> and includes image data of various image file formats. The display position <NUM> is data representing a position in the display unit <NUM> at which an image should be displayed, and is, for example, the data of a set of (X-coordinate position, Y-coordinate position, Z-coordinate position). The magnification <NUM> is data used to decide the size to display the object <NUM>, the visual recognition support image <NUM>, or the like on the display unit <NUM>.

The rehabilitation assistance server <NUM> refers to the tables <NUM>, <NUM>, and <NUM> and displays the visual recognition support images <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> on the display unit <NUM>.

<FIG> is a block diagram for explaining the hardware arrangement of the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. A CPU (Central Processing Unit) <NUM> is a processor for arithmetic control and executes a program, thereby implementing the functional components of the rehabilitation assistance server <NUM> shown in <FIG>. A ROM (Read Only Memory) <NUM> stores permanent data such as initial data and a program, and other programs. A network interface <NUM> communicates with another device or the like via a network. Note that the CPU <NUM> is not limited to one CPU and may include a plurality of CPUs or a GPU (Graphics Processing Unit) for image processing. In addition, the network interface <NUM> preferably includes a CPU independent of the CPU <NUM> and writes or reads transmission/reception data in or from an area of a RAM (Random Access Memory) <NUM>. In addition, it is preferable to provide a DMAC (Direct Memory Access Controller) (not shown) configured to transfer data between the RAM <NUM> and a storage <NUM>. In addition, an input/output interface <NUM> preferably includes a CPU independent of the CPU <NUM> and writes or reads input/output data in or from an area of the RAM <NUM>. Hence, the CPU <NUM> recognizes that data is received from or transferred to the RAM <NUM> and processes the data. In addition, the CPU <NUM> prepares a processing result in the RAM <NUM> and leaves subsequent transmission or transfer to the network interface <NUM>, the DMAC, or the input/output interface <NUM>.

The RAM <NUM> is a random access memory used by the CPU <NUM> as a work area for temporary storage. In the RAM <NUM>, an area to store data necessary for implementation of this example embodiment is allocated. Patient data <NUM> is data concerning a patient who undergoes rehabilitation using the rehabilitation assistance system. Image data <NUM> is the data of the object <NUM> serving as a target image or the visual recognition support image <NUM> to be displayed on the display unit <NUM>. A display position <NUM> is data representing a position in the display unit <NUM> at which the object <NUM> or the visual recognition support image <NUM> should be displayed. A magnification <NUM> is data representing the size to display an image such as the object <NUM> or the visual recognition support image <NUM> on the display unit <NUM>. These data are read out from, for example, the patient table <NUM>, the display parameter table <NUM>, and the image table <NUM>.

Input/output data <NUM> is data input/output via the input/output interface <NUM>. Transmission/reception data <NUM> is data transmitted/received via the network interface <NUM>. In addition, the RAM <NUM> includes an application execution area <NUM> used to execute various kinds of application modules.

The storage <NUM> stores databases, various kinds of parameters, and following data and programs necessary for implementation of this example embodiment. The storage <NUM> stores the patient table <NUM>, the display parameter table <NUM>, and the image table <NUM>. The patient table <NUM> is a table that manages the relationship between the patient ID <NUM> and the attribute information <NUM> and the like shown in <FIG>. The display parameter table <NUM> is a table that manages the relationship between the rehabilitation menu <NUM> and the display parameter <NUM> and the like shown in <FIG>. The image table <NUM> is a table that manages the relationship between the image type <NUM> and the image data <NUM> and the like shown in <FIG>.

The storage <NUM> further stores an action detection module <NUM>, a display control module <NUM>, an evaluation module <NUM>, and an updating module <NUM>.

The action detection module <NUM> is a module configured to detect the rehabilitation action of the user <NUM>. The display control module <NUM> is a module configured to display the avatar image <NUM>, the object <NUM> serving as a target image, the visual recognition support image <NUM> used to improve the visibility of the object <NUM>, and the like on the display unit <NUM>. The evaluation module <NUM> is a module configured to evaluate the rehabilitation ability of the user <NUM>. The updating module <NUM> is a module configured to update the target position represented by the target image in accordance with the evaluation result. The modules <NUM> to <NUM> are loaded into the application execution area <NUM> of the RAM <NUM> and executed by the CPU <NUM>. A control program <NUM> is a program configured to control the entire rehabilitation assistance server <NUM>.

The input/output interface <NUM> interfaces input/output data to/from an input/output device. A display unit <NUM> and an operation unit <NUM> are connected to the input/output interface <NUM>. In addition, a storage medium <NUM> may further be connected to the input/output interface <NUM>. Furthermore, a speaker <NUM> that is a voice output unit, a microphone that is a voice input unit, or a GPS (Global Positioning System) position determiner may be connected. Note that programs and data concerning general-purpose functions or other implementable functions of the rehabilitation assistance server <NUM> are not illustrated in the RAM <NUM> and the storage <NUM> shown in <FIG>.

<FIG> is a flowchart for explaining the processing procedure of the rehabilitation assistance server included in the rehabilitation assistance system according to this third example embodiment. <FIG> is a flowchart for explaining the processing procedure of visual recognition support image display of the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. These flowcharts are executed by the CPU <NUM> using the RAM <NUM> and implement the functional components of the rehabilitation assistance server <NUM> shown in <FIG>.

In step S1701, the rehabilitation assistance server <NUM> causes the display unit <NUM> or the like to display a visual recognition support image.

In step S1721, the rehabilitation assistance server <NUM> acquires patient information representing the attribute of the patient who undergoes rehabilitation using the rehabilitation assistance system <NUM> and what kind of rehabilitation menu the patient should undergo.

In step S1723, the rehabilitation assistance server <NUM> acquires display parameters necessary for displaying, on the display unit <NUM>, the visual recognition support image <NUM> and the like to be displayed on the display unit <NUM>. The display parameters to be acquired are parameters concerning the position and magnification of the visual recognition support image <NUM> and the like.

In step S1725, the rehabilitation assistance server <NUM> acquires image data of the visual recognition support image <NUM>. In step S1727, the rehabilitation assistance server <NUM> displays the visual recognition support image <NUM> and the like on the display unit <NUM>.

In step S1729, the rehabilitation assistance server <NUM> judges whether the display of the visual recognition support image <NUM> and the like needs to be changed. If the display change is not needed (NO in step S1729), the rehabilitation assistance server <NUM> ends the processing. If the display change is needed (YES in step S1729), the rehabilitation assistance server <NUM> advances to the next step.

In step S1731, the rehabilitation assistance server <NUM> changes the size of the visual recognition support image <NUM> in accordance with the eyesight of the user <NUM> or the evaluation result of the rehabilitation ability of the user <NUM>.

According to this example embodiment, even if the size of the target image is made small to reduce the deviation between the target distance and the exercise distance, the effect of rehabilitation can be increased by making the target distance and the exercise distance close while maintaining the visibility of the target image. In addition, since the sensation of touching the target image is clear for the user, the user can experience feeling of satisfaction in achieving the target.

A rehabilitation assistance system according to the fourth example embodiment of the present invention will be described next with reference to <FIG>. <FIG> is a block diagram for explaining the arrangement of the rehabilitation assistance system according to this example embodiment. The rehabilitation assistance system according to this example embodiment is different from the above-described third example embodiment in that the rehabilitation assistance system includes a sound output unit. The rest of the components and operations is the same as in the second example embodiment and the third example embodiment. Hence, the same reference numerals denote the same components and operations, and a detailed description thereof will be omitted.

A rehabilitation assistance system <NUM> includes a rehabilitation assistance server <NUM> and a sound output unit <NUM>. The rehabilitation assistance server <NUM> includes a sound output controller <NUM>. The sound output controller <NUM> controls output of a sound in accordance with the positional relationship between an object <NUM> serving as a target image and an avatar image <NUM>. The sound whose output is controlled by the sound output controller <NUM> is output from the sound output unit <NUM>.

For example, when the object <NUM> falls downward from above, the sound output controller <NUM> outputs a sound based on the distance, that is, the positional relationship between the object <NUM> and the avatar image <NUM>.

The output sound may be changed to a sound of a higher frequency as the distance between the object <NUM> and the avatar image <NUM> decreases, that is, the object <NUM> moves close to the avatar image <NUM>. In addition, similarly, the output sound may be changed to a sound of a lower frequency as the distance between the object <NUM> and the avatar image <NUM> increases, that is, the object <NUM> moves away from the avatar image <NUM>. That is, an acoustic effect like the Doppler effect for observing a difference in the frequency of the sound (wave) in accordance with the distance between the object <NUM> (sound source) and the avatar image <NUM> (user <NUM> (observer)) may be expressed. Note that instead of changing the frequency of the output sound, the volume of the output sound may be increased/decreased in accordance with the distance between the object <NUM> and the avatar image <NUM>.

In addition, the position of the object <NUM> may be instructed to the user <NUM> by outputting a sound from the sound output controller <NUM>. That is, the position of the object <NUM> is instructed using the sense of hearing of the user <NUM>.

For example, consider a case in which the user <NUM> wears a headphone when using the rehabilitation assistance system <NUM>. When the object <NUM> serving as a target image is located on the right side of the avatar image <NUM> (user <NUM>), the rehabilitation assistance server <NUM> outputs a sound from the right ear side of the headphone. Similarly, when the object <NUM> is located on the left side of the avatar image <NUM> (user <NUM>), the rehabilitation assistance server <NUM> outputs a sound from the left ear side of the headphone. This allows the user <NUM> to judge, based on the direction of the sound, whether the object <NUM> is located on the right side or left side of the user <NUM>. In addition, when the object <NUM> is located in front of the avatar image <NUM> (user <NUM>), the rehabilitation assistance server <NUM> outputs a sound from both sides of the headphone.

In the above description, the position of the object <NUM> is instructed using the sense of sight or the sense of hearing of the user <NUM>. However, one of the five senses other than the sense of sight and the sense of hearing, for example, the sense of taste, the sense of touch, or the sense of smell may be used to instruct the position of the object <NUM> to the user <NUM>.

For example, a sensor is placed on the tongue of the user <NUM> to cause the user <NUM> to feel a taste according to the position of the object <NUM>. Alternatively, the controller in the hand of the user <NUM> or the headphone or head mounted display worn by the user <NUM> may be vibrated. That is, the position of the object <NUM> may be instructed using the sense of touch of the user <NUM>.

<FIG> is a view for explaining an example of a sound table provided in the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. A sound table <NUM> stores sound data <NUM> in association with an image type <NUM>. The rehabilitation assistance server <NUM> controls the sound to be output by referring to the sound table <NUM>.

<FIG> is a view for explaining the hardware arrangement of the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. A RAM <NUM> is a random access memory used by a CPU <NUM> as a work area for temporary storage. In the RAM <NUM>, an area to store data necessary for implementation of this example embodiment is allocated. Sound data <NUM> is data concerning a sound to be output. This data is read out from, for example, the sound table <NUM>.

A storage <NUM> stores databases, various kinds of parameters, and following data and programs necessary for implementation of this example embodiment. The storage <NUM> stores the sound table <NUM>. The sound table <NUM> is the table that manages the relationship between the image type <NUM> and the sound data <NUM> shown in <FIG>.

The storage <NUM> further stores a sound output control module <NUM>. The sound output control module <NUM> is a module configured to control output of a sound in accordance with the positional relationship between the object <NUM> serving as a target image and the avatar image <NUM>. The module <NUM> is loaded into an application execution area <NUM> of the RAM <NUM> and executed by the CPU <NUM>. Note that programs and data concerning general-purpose functions or other implementable functions of the rehabilitation assistance server <NUM> are not illustrated in the RAM <NUM> and the storage <NUM> shown in <FIG>.

<FIG> is a flowchart for explaining the processing procedure of the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. <FIG> is a flowchart for explaining the processing procedure of sound output control of the rehabilitation assistance server included in the rehabilitation assistance system according to this example embodiment. These flowcharts are executed by the CPU <NUM> using the RAM <NUM> and implement the functional components of the rehabilitation assistance server <NUM> shown in <FIG>.

In step S2101, the rehabilitation assistance server <NUM> controls output of a sound. In step S2121, the rehabilitation assistance server <NUM> acquires the position of the avatar image <NUM>. In step S2123, the rehabilitation assistance server <NUM> acquires the position of the object <NUM>. In step S2125, the rehabilitation assistance server <NUM> determines the positional relationship between the avatar image <NUM> and the object <NUM>. In step S2127, the rehabilitation assistance server <NUM> controls the output of a sound in accordance with the determined positional relationship.

According to this example embodiment, since the rehabilitation is executed using the sense of hearing in addition to the sense of sight of the user, the user can more easily visually recognize the object, and the effect obtained by the rehabilitation can further be enhanced. In addition, the user can grasp the position of the object not only by the sense of sight but also by the sense of hearing. Furthermore, since a sound is output, even a user with poor eyesight can undergo the rehabilitation according to this example embodiment.

A system according to the fifth example embodiment of the present invention will be described next with reference to <FIG>. A rehabilitation assistance system according to this example embodiment is different from the above-described third example embodiment in that a target is made definite by a plurality of parameters. The rest of the components and operations is the same as in the third example embodiment. Hence, the same reference numerals denote the same components and operations, and a detailed description thereof will be omitted.

<FIG> is a view showing the contents of a target DB <NUM> according to this example embodiment in detail. As shown in <FIG>, a target to be currently achieved in rehabilitation is set for each patient. First, the exercise level and cognitive level of a patient are individually determined as the attributes of the patient. If the exercise level or cognitive level is high, the level is evaluated as A. A low level is evaluated as C, and a medium level is evaluated as B. For example, in a case of patient ID <NUM>, the exercise level is high, but the cognitive level is low. In this case, the distance up to the object, that is, the distance to stretch out the hand at maximum is long (here, for example, level <NUM> in five levels), the object appearance range is narrow to some extent (here, for example, level <NUM>), and the speed of the motion of the object is low (here, for example, level <NUM>). In addition, the object appearance interval is long (here, for example, level <NUM>), and both the object size and the sensor size are large (here, for example, level <NUM>).

On the other hand, in a case of patient ID <NUM>, the exercise level is low, but the cognitive level is high. In this case, the distance to the object, that is, the distance to stretch out the hand at maximum is short (here, for example, level <NUM> in five levels), the object appearance range is wide (here, for example, level <NUM>), and the speed of the motion of the object is low (here, for example, level <NUM>). On the other hand, the object appearance interval is short (here, for example, <NUM> in five levels), and both the object size and the sensor size are small (here, for example, <NUM> in five levels).

In a case of patient ID <NUM>, both the exercise level and the cognitive level are low. In this case, the distance to the object, that is, the distance to stretch out the hand at maximum is short (here, for example, level <NUM> in five levels), the object appearance range is narrow (here, for example, level <NUM>), and the speed of the motion of the object is low (here, for example, level <NUM>). In addition, the object appearance interval is long (here, for example, <NUM> in five levels), and both the object size and the sensor size are large (here, for example, <NUM> in five levels).

In this way, the parameters are variously changed in accordance with the attributes of the patient.

In general, the relationship between the motor function, the cognitive function, and various kinds of parameters is expected as shown in <FIG>. However, the rehabilitation assistance system according to the present invention does not set parameters limited to this relationship, and can search for a rehabilitation intensity suitable for each patient by changing various kinds of parameters (distance, range, speed, interval, and size) in accordance with the state and ability of the patient.

<FIG> shows a screen example <NUM> that a display controller <NUM> displays on a head mounted display <NUM> in this example embodiment. The display controller <NUM> displays an object <NUM> superimposed on a background image <NUM>. The display controller <NUM> displays the object <NUM> having the shape of a sweet potato while gradually changing its display position and size such that the object <NUM> seems to fall downward from above the user <NUM>. Here, an image <NUM> of a state in which a farmer is bending forward is displayed as a preliminary state to the appearance of the object <NUM>. Upon recognizing the farmer <NUM> bending forward, the user predicts that the object <NUM> then appears from the direction of the farmer. In addition, since a farmer <NUM> throws the object <NUM> upward, the user spontaneously performs an operation of following the object <NUM> with eyes and looking up. That is, it is possible to make not a linguistic instruction but an instruction that makes the user to be spontaneously conscious of the upper side.

After that, the user <NUM> moves controllers <NUM> and <NUM> in accordance with the position of the falling object <NUM> to move an avatar image <NUM> (not shown in <FIG>) having the shape of a basket. When the falling object <NUM> enters the basket, the mission is completed, and the requested rehabilitation action is completed. In a case in which the object <NUM> cannot be put in the basket, a child helping collection of the sweet potato may be displayed to relieve the mental shock or stress of the user. Note that an auxiliary indicator <NUM> may be displayed to show the appearance position of the farmer to the user.

In this example as well, it is possible to set a rehabilitation intensity appropriate for the user by changing various kinds of parameters (the distance to the falling sweet potato, the range of appearance of the farmer, the falling speed of the sweet potato, the interval to throw the sweet potato by the farmer, and the size of the basket) in accordance with the motor function and the cognitive function of the user.

<FIG> shows another screen example <NUM> that the display controller <NUM> displays on the head mounted display <NUM> in this example embodiment. Here, the display controller <NUM> displays an object <NUM> superimposed on a background image <NUM>. In this example, the display controller <NUM> displays the object <NUM> having the shape of an apple while gradually changing its display position and size such that the object <NUM> seems to fall downward from above the user <NUM>. An image <NUM> of a monkey shaking a tree is displayed as a preliminary state to the fall of the object <NUM> having the shape of an apple. Upon recognizing the monkey, the user predicts that the object <NUM> then falls from the direction of the monkey. After that, the user <NUM> moves the controllers <NUM> and <NUM> in accordance with the position of the falling object <NUM> to move the avatar image <NUM> (not shown in <FIG>) having the shape of a basket. When the falling object <NUM> enters the basket, the mission is completed, and the requested rehabilitation action is completed. In a case in which the object <NUM> cannot be put in the basket as well, a child helping collection of the apple may be displayed to relieve the mental shock or stress of the user.

<FIG> shows still another screen example <NUM> that the display controller <NUM> displays on the head mounted display <NUM> in this example embodiment. Here, the display controller <NUM> displays an object <NUM> superimposed on a background image <NUM>. In this example, the display controller <NUM> displays the object <NUM> having the shape of Dracula while gradually changing its display position and size such that the object <NUM> approaches from the far side to the user <NUM>. The user <NUM> moves the controllers <NUM> and <NUM> in accordance with the position of approaching Dracula to move an image <NUM> having the shape of a cross. When the cross hits Dracula, the mission is completed, and the requested rehabilitation action is completed. In a case in which the cross cannot hit Dracula as well, a helping child may be displayed to relieve the mental shock or stress of the user.

According to the above-described examples, it is possible to give a task to both the motor function and the cognitive function of the user. For example, a task to the cognitive function of the user can be given by displaying a preliminary state such as a farmer bending forward or a monkey appearing, and a task to the motor function of the user can be given by changing the distance, direction, speed, and the like of an object. That is, the patient is caused to perform both a motor rehabilitation action of stretching out an arm and a cognitive rehabilitation action of predicting the next appearance position of an object and moving the line of sight. This makes it possible to perform more effective rehabilitation.

Note that the visual recognition support image described in the third example embodiment may be additionally displayed for the object in each of <FIG>. In this case, the size of the outline of the visual recognition support image may be changed in accordance with the cognitive function of the patient. In addition, stepwise evaluation (good when touching only the outline and very good when touching the center) may be done in a case in which the avatar image serving as a sensor touches only the visual recognition support image (outline) and in a case in which the avatar image touches the object.

While the invention has been described with reference to example embodiments thereof, the invention is not limited to these example embodiments. For example, the display device is not limited to the head mounted display but may be a large screen. The controller is not limited to a grip type but may be a wearable sensor.

While the invention has been particularly shown and described with reference to example embodiments thereof, the invention is not limited to these example embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the claims.

Claim 1:
A rehabilitation assistance system comprising:
an action detector (<NUM>) configured to detect a first rehabilitation action of a user:
a display controller (<NUM>) configured to control a display unit to display an avatar image (<NUM>) that moves in a three-dimensional virtual space in accordance with the detected first rehabilitation action, further configured to control the display unit to display a target object (<NUM>) representing a target of the first rehabilitation action in the three-dimensional virtual space and further configured to control the display unit to display a visual recognition support image (<NUM>) that improves the recognizability of the target object by being displayed such that the target object is surrounded by the visual recognition support image being larger than the target object;
an evaluator (<NUM>) configured to evaluate a rehabilitation ability of the user by comparing the position of the avatar with the position of the target object, wherein the first rehabilitation action is judged as completed when the avatar touches the target object, and
an updater (<NUM>) configured to update the position of the target object in accordance with an evaluation result by said evaluator,
wherein the size of the visual recognition support image changes in accordance with the evaluation result,
wherein said action detector is further configured to detect a second rehabilitation action during the first rehabilitation action, and wherein
said evaluator is further configured to evaluate the rehabilitation ability based on both the first rehabilitation action and the second rehabilitation action.