Patent Description:
A walking assist device may be used to assist a user who experiences inconvenience in walking more readily. Such an inconvenience in walking may be attributed to various reasons, for example, diseases or accidents, and the user cannot walk readily on her/his own due to such reasons. In such a case, the walking assist device may be used to assist the user during a walking exercise as a part of a rehabilitation. In addition, a recent issue of aging societies has contributed to a growing number of people who experience inconvenience and pain from reduced muscular strength or joint problems due to aging. Thus, there is a growing interest in walking assist devices that enable elderly users or patients with reduced muscular strength or joint problems to walk with less effort.

A walking assist device may be worn on a body of a user to provide the user with power needed to walk, and assist the user with walking in a normal gait pattern. The walking assist device may interact directly with the body of the user, and thus a high level of safety may be needed.

The following publications concern a walking assist device:.

The present invention relates to a non-therapeutic method of controlling a walking assist device configured to be worn by a user and a corresponding device that are defined in the appended claims.

Hereinafter, some example embodiments will be described in detail with reference to the accompanying drawings. Regarding the reference numerals assigned to the elements in the drawings, it should be noted that the same elements will be designated by the same reference numerals, wherever possible, even though they are shown in different drawings. Also, in the description of embodiments, detailed description of well-known related structures or functions will be omitted when it is deemed that such description will cause ambiguous interpretation of the present disclosure.

It should be understood, however, that there is no intent to limit this disclosure to the particular example embodiments disclosed. On the contrary, example embodiments are to cover all modifications, equivalents, and alternatives falling within the scope of the example embodiments. Like numbers refer to like elements throughout the description of the figures.

In addition, terms such as first, second, A, B, (a), (b), and the like may be used herein to describe components. It should be noted that if it is described in the specification that one component is "connected," "coupled," or "joined" to another component, a third component may be "connected," "coupled," and "joined" between the first and second components, although the first component may be directly connected, coupled or joined to the second component.

Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosure of this application pertains. Terms, such as those defined in commonly used dictionaries, are to be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art, and are not to be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In the drawings, the thicknesses of layers and regions are exaggerated for clarity.

<FIG> is a diagram illustrating a walking assist device worn on a user according to at least one example embodiment.

A walking assist device <NUM>, or a gait assist device, may assist a user <NUM> wearing the walking assist device <NUM> in walking readily. The walking assist device <NUM> may assist or support a portion of a leg or an entire leg of the user <NUM> to help the user <NUM> walk more readily. The walking assist device <NUM> may be provided in a wearable exoskeleton type as illustrated in <FIG>, and configured to assist or support muscular strength of the user <NUM> when the user <NUM> walks to improve a walking movement or a gait of the user <NUM> or enable the user <NUM> to walk normally. For example, when a general user or an elderly user with reduced muscular strength wears the walking assist device <NUM>, the walking assist device <NUM> may increase a walking ability of the user by enabling the user to walk for a longer period of time as compared to when the user does not wear the walking assist device <NUM>, and enable the user to walk independently by providing or adding power needed for the user to walk. The walking assist device <NUM> may also be used in such fields as rehabilitation and walking correction by improving abnormal walking of a patient. The type of the walking assist device <NUM> illustrated in <FIG> is provided as an example, and example embodiments described herein may thus be applicable to other types of walking assist devices.

Referring to <FIG>, the walking assist device <NUM> generates a torque, or a rotational force, at left and right hip joints <NUM> and 120R under the control of a controller <NUM>, and the generated torque provides legs of the user <NUM> with power for flexion and extension through transferrers <NUM> and 140R disposed above knees of the user <NUM>. The walking assist device <NUM> measures a walking movement, or a gait, of the user <NUM> through a sensor thereof, and estimates a gait state or a gait phase in a gait cycle of the user <NUM> based on the measured gait. The walking assist device <NUM> determines a direction in which power is to be provided to each of the legs and an amount of the power to be provided, at a current point in time, based on the estimated gait phase.

The walking assist device <NUM> is worn on a body of the user <NUM> such that the walking assist device <NUM> applies power to the user <NUM> while receiving power applied by the user <NUM>. Thus, there is an interaction between the user <NUM> and the walking assist device <NUM>. In such interaction, a reaction to a physical intervention of the walking assist device <NUM> may vary from individual to individual. That is, individuals may react differently to a same physical intervention.

Thus, one or more example embodiments, may individualize the walking assist device <NUM> by determining a rule for adjusting a control variable of the walking assist device <NUM> to be suitable for each individual user. The foregoing term "individualization" indicates that a control policy or rule of the walking assist device <NUM> may be applied differently to individual users based on a difference in individual gait characteristic, physical state, muscular strength, and the like. The individualization may be achieved by learning the control policy of the walking assist device <NUM> based on a measured gait state of a user.

The example embodiments to be described hereinafter provide a safe and consistent walking assistance performance through the individualization of the walking assist device <NUM> based on a state variable and a torque control variable to which a walking movement, or a gait, of a user is applied. For the safe and consistent walking assistance performance, a smoother and/or a delayer may be used, and reinforcement learning that improves (or, alternatively, optimizes) parameters for the control policy may be performed. Through the reinforcement learning, it is possible to improve assistance transfer power by increasing a rate indicating how much an assist torque provided to the user <NUM> by the walking assist device <NUM> is to be useful for the user <NUM> in walking, and to reduce a mismatch in walking assistance by decreasing a rate indicating how much assist power provided to the user <NUM> by the walking assist device <NUM> hinders the user <NUM> in walking. In addition, it is possible to provide a walking assistance customized (or, alternatively, optimized) for an individual user, reduce an amount of energy needed for walking, and improve levels of walking regularity, walking safety, and walking stability. Hereinafter, the example embodiments will be described in detail with reference to the accompanying drawings.

<FIG> is a diagram illustrating a configuration of a walking assist device according to at least one example embodiment.

Referring to <FIG>, a walking assist device <NUM> includes a sensor <NUM>, a control device <NUM>, and a driver <NUM>. The control device <NUM> may be a device for controlling the walking assist device <NUM>. According to an example, the walking assist device <NUM> may further include a support member to support a body of a user wearing the walking assist device <NUM> and a fixing member to be fastened to the body of the user. For example, the walking assist device <NUM> may be the walking assist device <NUM> of <FIG> such that the control device <NUM> corresponds to the controller <NUM> of <FIG>.

The sensor <NUM> may include various sensors. The sensor <NUM> may include a sensor configured to sense information associated with a walking movement, or a gait, of the user, and a sensor configured to sense information needed to control an operation of the walking assist device <NUM>. For example, the sensor <NUM> may include a sensor (e.g., acceleration sensor, inertial sensor, and gyro sensor) configured to measure a gait of the user, a torque sensor configured to measure an assist torque transferred by the driver <NUM>, a current/voltage sensor, and the like.

In an example, the gait of the user may be measured through a sensor sensing angle information or motion information of both hip joints corresponding to positions of the hip joints of both legs of the user. The angle information of the hip joints may include at least one set of information associated with angles of the hip joints, a difference between the angles of the hip joints, motion directions of the hip joints, and angular velocities of the hip joints.

The control device <NUM> configured to control the walking assist device <NUM> includes a controller <NUM> and a memory <NUM>. The memory <NUM> is connected to the controller <NUM>, and configured to store instructions to be executed by the controller <NUM>, and data to be processed by the controller <NUM> and/or data having been processed by the controller <NUM>. For example, the memory <NUM> stores parameters corresponding to an assist torque control signal output by the controller <NUM>. The memory <NUM> may include a non-transitory computer-readable storage medium, for example, a high-speed random-access memory (RAM) and/or a nonvolatile computer-readable storage medium (e.g., at least one disk storage device, flash memory device, or other nonvolatile solid-state memory devices).

The controller <NUM> may be implemented using processing circuitry such as hardware including logic circuits, a hardware/software combination such as a processor executing software; or a combination thereof. For example, the processing circuitry may include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a System-on-Chip (SoC) a programmable logic unit, a microprocessor, or an application-specific integrated circuit (ASIC), etc..

The controller <NUM> generates a control signal to control the walking assist device <NUM>. For example, the controller <NUM> generates a torque control signal to control an assist torque to be provided by the walking assist device <NUM> based on the gait measured by the sensor <NUM>. The controller <NUM> controls the driver <NUM> configured to generate the assist torque in the walking assist device <NUM> based on the generated control signal.

As discussed in more detail below, the controller <NUM> may be programmed as a special purpose processor to perform reinforcement learning such that the walking assist device <NUM> may improve assist torque transfer power by increasing a rate at which the assist torque provided to the user by the walking assist device <NUM> is useful for the user when walking, and reduce the number of mismatches in walking assistance by decreasing a rate at which the assist torque provided to the user by the walking assist device <NUM> hinders the user from walking.

The driver <NUM> operates an actuator of the walking assist device <NUM> based on the torque control signal generated by the controller <NUM>. The driver <NUM> provides the assist torque to the gait of the hip joints of the user through the actuator. The actuator converts electrical energy to kinetic energy, and applies the kinetic energy to the body of the user to provide the user with power needed for the user to walk. The actuator is disposed on a portion corresponding to the positions of the hip joints of the user, and generates the assist torque for flexion and extension of the legs of the user to assist the user in walking.

The controller <NUM> determines a state variable indicating a gait state of the user based on the gait of the user, and controls the walking assist device <NUM> based on the determined state variable. The controller <NUM> sets a parameter to control the assist torque based on the state variable, and outputs a periodic assist torque control signal to assist the user in walking based on the set parameter.

In an example, the controller <NUM> controls an assist torque to be provided by the walking assist device <NUM> based on a state variable, and determines whether to individualize a control signal used to control the assist torque based on the state variable. The controller <NUM> sets a gain to adjust an intensity of the assist torque and sets a time delay to adjust an output time of the assist torque, and defines the state variable based on the set gain and the set time delay. By adjusting the gain and the time delay, the walking assist device <NUM> may more stably respond to a sudden motion of the user, a sudden stop of the user, or a change in environment, and to improve levels of walking regularity and safety. A first state variable, a second state variable, and a third state variable to be described hereinafter may be used to define the assist torque control signal that is used to control the operation of the walking assist device <NUM>.

In an example, the controller <NUM> determines a first state variable for a gait sate of the user based on a walking movement, or a gait, of the user. The controller <NUM> smooths the first state variable and time-delays the smoothed first state variable to obtain a second state variable which is smoothed and time-delayed from the first state variable. In this example, the term "time-delay" may indicate a process performed using a time delay which is a time value set (or, alternatively, preset) by the user or based on a control design rule. For example, when a walking speed of the user is greater than a first reference value, the controller <NUM> sets a time delay to be relatively small. Conversely, when a walking speed of the user is less than the first reference value, the controller <NUM> sets a time delay to be relatively large. For another example, the controller <NUM> determines a delay based on a walking acceleration of the user. In this example, when a walking acceleration of the user is greater than a second reference value, the controller <NUM> sets a time delay to be small. Conversely, when a walking acceleration of the user is less than the second reference value, the controller <NUM> sets a time delay to be large. In the foregoing examples, the walking speed and the walking acceleration of the user may be determined based on sensing information obtained from the sensor <NUM>.

The controller <NUM> obtains a third state variable by applying a torque control variable to the second state variable. The controller <NUM> determines a gait phase in a gait cycle of the user based on the first state variable, and determines the torque control variable when the determined gait phase corresponds to a defined or (alternatively, a predefined) gait phase.

The torque control variable is determined in different ways in a learning mode and in a normal mode. The user may select the learning mode or the normal mode. The user may select the learning mode to optimize or individualize the walking assist device <NUM>. Through the learning mode, parameters associated with controlling an assist torque to be provided by the walking assist device <NUM> may be adjusted based on a gait state or a gait style of the user.

When the normal mode is selected, the controller <NUM> determines a desired (or, alternatively, an optimal) torque control variable by applying a state variable value corresponding to a current gait phase of the user to a torque control variable determining function based on parameters derived from a previous learning result. The controller <NUM> determines the determined torque control variable to be the torque control variable to be applied to the second state variable.

When the learning mode is selected, the controller <NUM> determines, to be the torque control variable, a retrieval torque control variable generated based on a probability function. In an example, in the learning mode, the controller <NUM> determines a score of a previous torque control variable based on a walking assist power index determined from a previous gait of the user. The walking assist power index includes a first walking assist power index indicating a magnitude of walking assistance power transferred by the walking assist device <NUM>, and a second walking assist power index indicating a degree of hindrance to the user in walking by the walking assist device <NUM>. The first walking assist power index is a positive measurement result value for walking assistance, and the second walking assist power index is a negative measurement result value for walking assistance. For example, the controller <NUM> calculates the score in a form of weighted sum of the first walking assist power index and the second walking assist power index.

The walking assist power index may be calculated in real time, and a control policy of the walking assist device <NUM> may be learned based on the walking assist power index. In the learning mode, a parameter of the probability function may be determined such that the first walking assist power index increases and the second walking assist power index decreases, and the retrieval torque control variable may be determined based on the probability function. When the score determined based on the walking assist power index does not satisfy a set (or, alternatively, a preset) condition, the controller <NUM> generates the retrieval torque control variable based on the probability function, and determines the generated retrieval torque control variable to be the torque control variable to be applied to the second state variable. The set (or, alternatively, preset) condition may indicate that the score is greater than a threshold value, or the number of times calculating the score reaches a certain number of times.

The controller <NUM> obtains a third state variable by smoothing the determined torque control variable and applying the smoothed torque control variable to the second state variable. The controller <NUM> generates a control signal to determine the assist torque to be provided by the walking assist device <NUM> based on the obtained third state variable. The controller <NUM> generates an assist torque control signal by applying, to the third state variable, a gain for adjusting an intensity of the assist torque, and controls the assist torque to be provided by the walking assist device <NUM> based on the generated assist torque control signal.

Through the operations described above, it is possible to reduce the number of mismatches in walking assistance provided by the walking assist device <NUM>, improve an assist torque transfer efficiency, and/or improve a level of walking regularity. For example, through reinforcement learning, the walking assist device <NUM> may improve assist torque transfer power by increasing a rate at which the assist torque provided to the user by the walking assist device <NUM> is useful for the user when walking, and reduce the number of mismatches in walking assistance by decreasing a rate at which the assist torque provided to the user by the walking assist device <NUM> hinders the user from walking.

According to an example, a remote controller (not shown) configured to remotely control the walking assist device <NUM> may be provided. The remote controller may control an overall operation of the walking assist device <NUM> in response to an input from the user. For example, the remote controller may initiate or suspend the operation of the walking assist device <NUM>. In addition, the remote controller may generate an assist torque control signal to control a walking assistance operation of the walking assist device <NUM>, and transmit the generated assist torque control signal to the walking assist device <NUM>. The remote controller may provide a user interface (UI) that facilitates manipulation or operation of the walking assist device <NUM>. Through the UI, the user may directly set a gain associated with an intensity of an assist torque to be provided by the walking assist device <NUM> or a delay in an output time of the assist torque.

<FIG> and <FIG> are flowcharts illustrating a method of controlling a walking assist device according to at least one example embodiment. The method of controlling a walking assist device will be hereinafter simply referred to as a control method, and the control method may be performed by a device for controlling a walking assist device which will be hereinafter simply referred to as a control device, which may be, for example, the control device <NUM>.

Referring to <FIG>, in operation <NUM>, a control device determines a first state variable for a gait state of a user wearing a walking assist device based on a gait of the user. For example, the control device determines the first state variable based on hip joint angle information of the user that is measured by a sensor of the walking assist device. The first state variable may indicate a gait or a movement of the user in association with the gait state of the user, and may be defined by angle information of left and right hip joints as represented by Equation <NUM>.

In Equation <NUM>, y<NUM>(t) denotes a first state variable based on time t. qr(t) and ql(t) denote an angle of a right hip joint and an angle of a left hip joint, respectively, based on time t.

In operation <NUM>, the control device obtains a second state variable which is smoothed and time-delayed from the first state variable. Operation <NUM> may be performed in detail as described hereinafter with reference to <FIG>. Referring to <FIG>, in operation <NUM>, the control device smooths the first state variable. Through the smoothing, noise in the first state variable may be reduced. The control device may smooth the first state variable using a smoothing filter including, for example, a low-pass filter (LPF). The control device may obtain the smoothed first state variable by performing low-pass filtering based on a first state variable determined for a current gait cycle and a first state variable determined for a previous gait cycle, as represented by Equation <NUM>.

In Equation <NUM>, y denotes a smoothed first state variable, and α denotes a defined (or, alternatively, a predefined) constant having a value between <NUM> and <NUM>. yprev denotes a first state variable determined for a previous gait cycle, and yraw denotes a first state variable determined for a current gait cycle.

In operation <NUM>, the control device obtains the second state variable by time-delaying the smoothed first state variable obtained in operation <NUM>. In another example, the time-delaying may be performed prior to the smoothing. In this example, the second state variable may be obtained by performing the smoothing, such as, for example, filtering, on the time-delayed first state variable obtained through the time-delaying. In the time-delaying process, a time delay value applied to the first state variable may be a defined (or, alternatively, a predefined) value or a value set by the user.

In operation <NUM>, the control device determines whether a gait phase of the user corresponds to a defined (or, alternatively, a predefined) gait phase. The control device calculates a current gait phase based on the smoothed first state variable obtained in operation <NUM> as represented by Equation <NUM>.

In Equation <NUM>, φ denotes a gait phase, and y denotes a smoothed first state variable. The calculated gait phase may be periodic based on a gait of the user.

In operation <NUM>, when the gait phase of the user corresponds to the defined (or, alternatively, a predefined) gait phase, the control device determines a torque control variable. The torque control variable may be used to more finely adjust a point in time at which an assist torque control signal is to be applied. In a mode which is not a learning mode, a previously determined torque control variable or a defined (or, alternatively, a predefined) torque control variable may be used. However, in the learning mode, a desired (or, alternatively, an optimal) torque control variable may be retrieved based on a gait of the user. The learning mode may be a process for individualizing the walking assist device to be suitable for the user, and whether to enter the learning mode or not may be selected by the user. Hereinafter, how to determine a torque control variable will be described in detail with reference to <FIG>.

In operation <NUM>, the control device smooths the torque control variable determined in operation <NUM>. In an example, the torque control variable determined in operation <NUM> may change discontinuously and such a discontinuous change of the torque control variable may generate a relatively great jerk motion, which may pose a threat to the safety of the user and the walking assist device. Thus, a consistent reaction may not be readily derived from the user. To inhibit (or, alternatively, prevent) this, the control device may smooth the torque control variable such that the torque control variable is gradually applied. By the smoothed torque control variable, an assist torque to be provided to the walking assist device may continuously and smoothly change. Thus, the jerk motion may not be generated in the walking assist device, and safe and stable interactive learning may occur between the user and the walking assist device.

When the gait phase of the user does not correspond to the defined (or, alternatively, a predefined) gait phase as a result of the determination in operation <NUM>, the torque control variable applied to a previous cycle may be applied unchanged to a current cycle.

Referring back to <FIG>, in operation <NUM>, the control device obtains a third state variable by applying the torque control variable to the second state variable. In operation <NUM>, the control device determines an assist torque to be provided by the walking assist device based on the obtained third state variable. The control device generates an assist torque control signal by applying, to the third state variable, a gain for adjusting an intensity of the assist torque, and controls the assist torque to be provided by the walking assist device based on the generated assist torque control signal. The assist torque controlled in such a manner is applied during one gait cycle of the user. When the gait cycle is terminated, operations <NUM> through <NUM> are performed again on a next gait cycle.

According to at least one example embodiment, it is possible to effectively individualize a walking assist device to be suitable to a user of the walking assist device based on a gait characteristic of the user, and to improve safety of both the user and the walking assist device by smoothly controlling an assist torque to be provided by the walking assist device.

<FIG> is a flowchart illustrating a method of determining a torque control variable according to at least one example embodiment.

Referring to <FIG>, when determining the torque control variable in operation <NUM>, the controller may perform operations <NUM> to <NUM>, discussed below.

In operation <NUM>, a control device determines whether a current set mode is a learning mode. In an example, a user may adjust a mode of a walking assist device to be the learning mode or a normal mode.

In operation <NUM>, when the current set mode is the normal mode, the control device determines a desired (or, alternatively, an optimal) torque control variable without a learning process. For example, the control device determines the desired (or, alternatively, optimal) torque control variable by applying a state variable value corresponding to a current gait phase of the user to a torque control variable determining function based on parameters derived from a previous learning result. In an example, when learning is already completed and the normal mode operates currently, the control device sets a control policy suitable to a current gait state of the user using parameters obtained from a previous learning process, and determines the desired (or, alternatively, optimal) torque control variable using the set control policy. In another example, the control device uses a previously determined optimal torque control variable. The control device determines the determined torque control variable to be a torque control variable to be applied to a second state variable.

When the current set mode is the learning mode, the control device performs reinforcement learning including operations <NUM> through <NUM>. Such a learning process may be performed once per gait cycle of the user.

In operation <NUM>, the control device determines a score, for example, a reward, for a previous torque control variable based on a walking assist power index determined for a previous gait of the user. The walking assist power index includes a first walking assist power index indicating a magnitude of walking assistance power transferred from the walking assist device, and a second walking assist power index indicating a degree of hindrance to the user in walking by the walking assist device. In an example, the first walking assist power index indicates how assist power is well transferred to the user for walking. The greater the first walking assist power index the greater the assist power to be transferred. In addition, the second walking assist power index indicates a quantity of hindrance to the user in walking and is indicated by a negative value. The smaller the value the greater the resistance of the hindrance to the user in walking.

In an example, the score may be determined based on a weighted sum of the first walking assist power index and the second walking assist power index as represented by Equation <NUM>.

In Equation <NUM>, MPP (Mean Positive Power) and MNP (Mean Negative Power) denote a first walking assist power index and a second walking assist power index, respectively. In Equation <NUM>, weights applied to MPP and MNP are provided as examples, and the weights may vary according to an example. For example, when one cycle of learning is applied per gait cycle, MPP and MNP indicate a mean value of the first walking assist power index and a mean value of the second walking assist power index, respectively, for a gait during a previous gait cycle, for example, two steps or one stride. Equation <NUM> above may be an objective function for adjusting a learning result to be desirable.

When the score represented by the weighted sum of the first walking assist power index and the second walking assist power index increases, the walking assistance to be provided may be smoother, without hindrance to the user in walking while naturally great assist power is being provided. The learning process may be performed in accordance with a current gait state of the user. For example, the learning process may be performed such that a value of the first walking assist power index increases while an absolute value of the second walking assist power index does not excessively increase.

In operation <NUM>, the control device determines whether a termination condition is satisfied. The termination condition indicates whether the score determined in operation <NUM> exceeds a threshold value, or whether the number of repetitions of the learning process reaches a defined (or, alternatively, a predefined) number of times. In operation <NUM>, when the termination condition is not satisfied, the control device updates a parameter of a probability model for determining a retrieval torque control variable. The control device updates the parameter of the probability model such that the score increases. The updating of the parameter may be performed repetitively on each gait cycle until the termination condition is satisfied.

In operation <NUM>, the control device generates the retrieval torque control variable based on a probability model-based function. The control device generates the retrieval torque control variable based on the probability model-based function, and determines the generated retrieval torque control variable to be the torque control variable to be applied to the second state variable.

A score for a retrieval torque control variable newly determined in each learning process may be measured, and a parameter of the probability model may be updated repetitively until the termination condition is satisfied.

<FIG> is a diagram illustrating a configuration of a control device for controlling a walking assist device according to at least one example embodiment.

Referring to <FIG>, a control device <NUM> of a walking assist device <NUM> may correspond to the control device <NUM> of <FIG>.

The control device <NUM> may include a state variable smoother <NUM>, a state variable delayer <NUM>, a learner <NUM>, a torque control variable smoother <NUM>, and a torque control signal generator <NUM>.

For example, as discussed in more detail below, the processing circuitry included in the control device <NUM> may be configured as a special purpose processor to perform the operations of the state variable smoother <NUM>, the state variable delayer <NUM>, the learner <NUM>, the torque control variable smoother <NUM>, and the torque control signal generator <NUM>.

The control device <NUM> may use such components to safely and effectively individualize walking assistance to be suitable to a user of the walking assist device <NUM>, and train or learn a control policy to be applied to the walking assist device <NUM>. In an example, operations of the state variable smoother <NUM>, the state variable delayer <NUM>, the learner <NUM>, the torque control variable smoother <NUM>, and the torque control signal generator <NUM> may be performed by the controller <NUM> described above with reference to <FIG>.

A first state variable for a gait state of the user is defined based on information associated with a gait of the user measured by a sensor of the walking assist device <NUM>. Referring to <FIG>, the sensor of the walking assist device <NUM> senses a left hip joint angle ql and a right hip joint angle qr associated with a current gait of the user. To the control device <NUM>, left hip joint angle information and right hip joint angle information sensed by the sensor are transferred. The control device <NUM> then determines the first state variable based on the left hip joint angle information and the right hip joint angle information. For example, the control device <NUM> calculates the first state variable based on the left hip joint angle information and the right hip joint angle information as represented by Equation <NUM> above.

Referring back to <FIG>, the state variable smoother <NUM> smooths the first state variable determined based on the sensed hip joint angle information to change the first state variable to a smoother signal. Such a smoothing process may include, for example, low-pass filtering as represented by Equation <NUM> above. Referring to <FIG>, a first state variable which includes noise or is not smooth is input to the state variable smother <NUM>, and the state variable smoother <NUM> changes the input first state variable to a smoother signal and outputs the smoother signal using a filter.

Referring back to <FIG>, the state variable delayer <NUM> time-delays the smoothed first state variable, and outputs a second state variable as a result of performing the time-delaying. The second state variable corresponds to a result of smoothing the first state variable calculated as represented by Equation <NUM> above and then time-delaying the smoothed first state variable. Referring to <FIG>, a smoothed first state variable y(t) is input to the state variable delayer <NUM>, and the state variable delayer <NUM> applies a time delay Δt to the input first state variable y(t) and outputs a second state variable y(t-Δt). The time delay Δt determines an output time of a increased (or, alternatively, a maximum) assist torque and may be a value set by the user, for example, a constant.

Referring back to <FIG>, the learner <NUM> estimates a current gait phase of the user based on the smoothed first state variable. For example, the learner <NUM> estimates a gait phase based on Equation <NUM>.

In Equation <NUM>, phase denotes a gait phase of a user, and y(t) denotes a smoothed first state variable. Δt denotes a time delay, and c denotes a defined (or, alternatively, a predefined) constant as a scaling factor for scaling. An example of a change of the gait phase calculated based on Equation <NUM> above over time will be described hereinafter with reference to <FIG>.

<FIG> illustrates a graph indicating a change of a smoothed first state variable <NUM> over time and a change of a gait phase <NUM> determined based on the first state variable <NUM> over time. Both the first state variable <NUM> and the gait phase <NUM> have periodicity.

Referring back to <FIG>, the learner <NUM> determines whether a current gait phase corresponds to a defined (or, alternatively, a predefined) gait phase. When the current gait phase corresponds to the defined (or, alternatively, a predefined) gait phase, the learner <NUM> determines a torque control variable. When a set mode is a learning mode, the learner <NUM> retrieves a torque control variable to improve (or, alternatively, optimize) a point in time at which an assist torque control signal is to be applied by performing reinforcement learning. When the reinforcement learning is performed, the learner <NUM> calculates a score based on a first walking assist power index and a second walking assist power index. The learner <NUM> determines parameters of a probability model-based function based on the calculated score and the first state variable, and generates a retrieval torque control variable based on the probability model-based function. The retrieval torque control variable may be used to finely adjust a time delay applied by the state variable delayer <NUM>. Herein, a learning process may be a process of finely adjusting the time delay based on the retrieval torque control variable, and discovering the retrieval torque control variable such that the score calculated as a result of the adjusting is increased (or, alternatively, maximized). Such learning process may be performed on a certain gait phase.

In a normal mode which is not the learning mode, the learner <NUM> determines a desired (or, alternatively, an optimal) torque control variable without performing the learning process. For example, the learner <NUM> determines the desired (or, alternatively, optimal) torque control variable by applying a state variable value corresponding to a current gait phase of the user to a torque control variable determining function based on parameters derived from a previous learning result. The desired (or, alternatively, optimal) torque control variable may also be used to finely adjust a time delay applied by the state variable delayer <NUM>.

The torque control variable smoother <NUM> smooths the retrieval torque control variable or the optimal torque control variable. Referring to <FIG>, the torque control variable smoother <NUM> performs such smoothing process such that a torque control variable changes gradually around a time point to which the torque control variable is applied and around a time point in which the application is terminated. Thus, it is possible to inhibit (or, alternatively prevent) an assist torque provided by the walking assist device <NUM> from rapidly changing.

Referring back to <FIG>, the torque control variable smoother <NUM> generates a third state variable by applying the smoothed torque control variable to the second state variable output from the state variable delayer <NUM>, and outputs the generated third state variable. The torque control signal generator <NUM> generates an assist torque control signal based on the third state variable, and transmits the generated assist torque control signal to the walking assist device <NUM>. The torque control signal generator <NUM> generates the assist torque control signal by applying, to the third state variable, a gain for adjusting an intensity of the assist torque. The assist torque control signal may be maintained for one gait cycle of the user. In an example, the torque control signal generator <NUM> inverts the assist torque control signal to generate the inverted assist torque control signal. The assist torque control signal may be applied to assist a left leg of the user with walking, and the inverted assist torque control signal may be applied to assist a right leg of the user with walking.

<FIG> is a diagram illustrating a configuration of a system for controlling a physical interactive device according to at least one example embodiment. The system for controlling a physical interactive device will be hereinafter simply referred to as a control system.

Referring to <FIG>, the example embodiments described above with reference to <FIG> are expandable and applicable to control a physical interactive device <NUM> which is physically interactive with humans. The physical interactive device <NUM> includes, for example, a wheel-based mobile robot, an articulated robotic limb, a biped walking humanoid robot, and the like.

When there is a physical interaction between a user and the physical interactive device <NUM>, a control device <NUM> configured to control the physical interactive device <NUM> may smooth or intentionally time-delay a control signal used to control the physical interactive device <NUM> for the safety of the user and the physical interactive device <NUM>. Similar to the control device <NUM> of the walking assist device <NUM> illustrated in <FIG>, the control device <NUM> includes a state variable smoother <NUM>, a state variable delayer <NUM>, a learner <NUM>, a power control variable smoother <NUM>, and a power control signal generator <NUM>. Respective operations of such components of the control device <NUM> respectively correspond to the operations of the components of the control device <NUM>, and thus detailed descriptions of the operations will be omitted here for brevity. However, the control device <NUM> of <FIG> may differ from the control device <NUM> of <FIG> in that the control device <NUM> generates a power control signal to control a driver of the physical interactive device <NUM> through the power control signal generator <NUM>, whereas the control device <NUM> generates an assist torque control signal to control an assist torque of the walking assist device <NUM>.

The learner <NUM> determines whether a defined (or, alternatively, a predefined) event occurs based on information measured by a sensor of the physical interactive device <NUM>, and performs a learning process in response to the event occurring. For example, when there is a user-robot mutual contact or interaction, the learner <NUM> learns a user-customized control operation of learning a reaction that is most preferred by the user, for example. A method of calculating a state variable, a score, and the like to be used for the learning may vary as described in the example of a walking assist device. In addition, the learner <NUM> may perform the learning process using other optimization or learning algorithms, such as, for example, evolutionary computation, in addition to reinforcement learning.

As described above, the control device <NUM> may facilitate a smoother and safer action-reaction interaction between the user and the physical interactive device <NUM>.

<FIG> are diagrams illustrating different examples of a control system for controlling a physical interactive device according to at least one example embodiment.

Referring to <FIG>, a control system for controlling a physical interactive device <NUM> may be embodied by selective combinations of the physical interactive device <NUM>, a state variable smoother <NUM>, a state variable delayer <NUM>, a learner <NUM>, a power control variable smoother <NUM>, and a power control variable delayer <NUM>. In addition, a connection of components or elements in each selective combination may be embodied in various ways.

The units and/or modules described herein may be implemented using hardware components and software components. For example, the hardware components may include microphones, amplifiers, band-pass filters, audio to digital convertors, and processing devices. A processing device may be implemented using one or more hardware device configured to carry out and/or execute program code by performing arithmetical, logical, and input/output operations. The processing device(s) may include a processor, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a field programmable array, a programmable logic unit, a microprocessor or any other device capable of responding to and executing instructions in a defined manner. The processing device may run an operating system (OS) and one or more software applications that run on the OS. The processing device also may access, store, manipulate, process, and create data in response to execution of the software. For purpose of simplicity, the description of a processing device is used as singular; however, one skilled in the art will appreciated that a processing device may include multiple processing elements and multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. In addition, different processing configurations are possible, such a parallel processors.

Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM discs, DVDs, and/or Blue-ray discs; magnetooptical media such as optical discs; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory (e.g., USB flash drives, memory cards, memory sticks, etc.), and the like.

Claim 1:
A walking assist device (<NUM>, <NUM>, <NUM>) comprising:
a sensor (<NUM>) configured to measure hip joint angle information of a user (<NUM>) wearing the walking assist device (<NUM>, <NUM>, <NUM>);
a controller (<NUM>, <NUM>) configured to generate an assist torque control signal to control an assist torque based on the measured hip joint angle information; and
a driver (<NUM>) configured to operate an actuator of the walking assist device (<NUM>, <NUM>, <NUM>) to provide the assist torque to the user (<NUM>) based on an assist torque control signal,
wherein the controller (<NUM>, <NUM>) is further configured to:
determine a first state variable (<NUM>) for a gait phase (<NUM>) in a gait cycle of the user (<NUM>) based on the hip joint angle information,
obtain a second state variable by smoothing and time-delaying the first state variable (<NUM>),
obtain a smoothed torque control variable by determining a torque control variable and smoothing the determined torque control variable in response to a current gait phase (<NUM>) in the gait cycle of the user (<NUM>) corresponding to a set gait phase,
obtain a third state variable by applying the smoothed torque control variable to the second state variable, and
determine the assist torque control signal based on the third state variable.