Method and apparatus for creating virtual joint sensation

An apparatus for creating virtual joint sensation is provided. The apparatus includes: a controlling part for creating control signals for controlling respective user's joints by referring to information on torques to be applied to the respective user's joints, wherein the information on the torques is acquired by analyzing information on forces to be applied to the user's body contacting a virtual object; and a torque-applying part, including one or more torque-applying units worn on the respective user's joints, for giving the torques to the respective user's joints by using the control signals.

FIELD OF THE INVENTION

The present invention relates to an apparatus, and a method for creating virtual joint sensation; and more particularly, to the apparatus, and the method for making a user feel sensation of grabbing an object even though the user actually does not by applying torques to the user's respective joints by controlling interactions between coils and magnets.

BACKGROUND OF THE INVENTION

When a user grabs an object, forces are exerted on a user's body by which the object is touched. That is, the forces are exerted on points of contact in the user's body. For example, when the user grabs the object, the forces work between the object and the user's hand. As the forces are delivered to fingers, the user may feel physical properties of the object. Herein, a force may be generated in a direction normal to a contact surface.

Even though the user actually does not grab the object, if the forces are applied to the user's body from the outside, the user may feel sensation as if the user grabbed the object. In other words, an apparatus for applying the forces to the user's body from the outside may enable the user to feel the sensation as if the user grabbed the object. Such apparatus for enabling the user to feel a virtual sensation may be widely used in a system for implementing a virtual world, etc. Haptic devices have been developed as tools for applying forces to users in the virtual world.

However, most of existing haptic devices apply forces to ends of the body of a user (e.g., fingertips as points of contact). Such existing haptic devices have problems as follows: the volume of the devices is increased and a pose of a hand or a finger becomes unnatural or limited.

Therefore, the present inventor came to develop a technology for applying torques to user's joints instead of applying forces to ends of the user's body by which an object is touched and relatively supporting the user's flexible motion.

SUMMARY OF THE INVENTION

It is an object of the present invention to solve all the aforementioned problems.

It is another object of the present invention to enable a user to feel a sensation of grabbing an object even though the user actually does not grab it by applying torques to the user's joints.

It is still another object of the present invention to provide an apparatus for creating a virtual sensation that does not limit the user's motions or puts relatively few limits on them.

In accordance with one aspect of the present invention, there is provided an apparatus for creating virtual joint sensation, including: a controlling part for creating control signals for controlling respective user's joints by referring to information on torques to be applied to the respective user's joints, wherein the information on the torques is acquired by analyzing information on forces to be applied to the user's body contacting a virtual object; and a torque-applying part, including one or more torque-applying units worn on the respective user's joints, for giving the torques to the respective user's joints by using the control signals.

In accordance with another aspect of the present invention, there is provided a method for creating virtual joint sensation, including steps of: (a) a controlling part creating control signals for controlling respective user's joints by referring to information on torques to be applied to the respective user's joints, wherein the information on the torques is acquired by analyzing information on forces to be applied to the user's body contacting a virtual object; and (b) a torque-applying part, including one or more torque-applying units worn on the respective user's joints, giving the torques to the respective user's joints by using the control signals.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

To allow those skilled in the art to the present invention to be carried out easily, the example embodiments of the present invention by referring to attached diagrams will be explained in detail as follows:

FIG. 1is a drawing explaining a principle of creating joint sensation in accordance with one example embodiment of the present invention.

In accordance with one example embodiment of the present invention, as illustrated in a left drawing110inFIG. 1, when a user grabs an object, forces may be applied to the fingertips of the user's hand, i.e., a part of the hand at which the object is contacted. For example, when the user grabs the object, the forces work between the object and the user's hand, and as the forces are delivered to the fingers, the user may feel physical properties of the object. At the time, the forces may be generated in a direction perpendicular to a contact surface between the object and the user's body. In the left drawing110inFIG. 1, it is assumed that a force F1is applied to the tip of the user's index finger and the other force F2is applied to the tip of the user's thumb.

In accordance with one example embodiment of the present invention, the forces may be delivered to the user's joints. For the delivery, the forces applied to the fingertips may be converted to torques applied to respective joints of the finger. For example, if external forces are applied to ends, e.g., finger tips, of a robot, appropriate torques must be applied to joints of a robot, e.g., joints of the finger, to withstand the external forces.

In accordance with one example embodiment of the present invention, as explained in equations between the left drawing110ofFIG. 1and a central drawing120thereof, the applied forces may be converted into joint torque under the principle of virtual work. The conversion between a force F and torque τ may be made under the following equation 1:
τ=JTF<Equation 1>

where J may be a Jacobian matrix; and T may represent the matrix transpose. Forces F1and F2may be converted to torques τ1and τ2, respectively, under Equations 2 and 3 below. The Jacobian matrix J may be defined by a kinematic motion equation with respect to points of contact, e.g., fingertips, and their corresponding joints. In the kinematic motion equation, a hydrostatic equation expressed as JTunder the principle of virtual work, also, may be obtained. Herein, the hydrostatic equations may be equations between forces at the points of contact and torques at the respective joints.
τ1=J1TF1<Equation 2>
τ2=J2TF2<Equation 3>

In accordance with one example embodiment of the present invention, forces, for example, F1and F2illustrated inFIG. 1, may be six-value vectors consisting of linear forces in three directions and moments in three directions. However, herein, it is assumed that forces are three-value vectors consisting of linear forces in three directions for simplification. Besides, the number of individual finger joints may be decided by a finger model. Because structures of a thumb and an index finger, e.g., the number of joints of fingers, are different, the number of torques to be applied to the joints of fingers may be different depending on fingers. For example, a thumb may be modeled with three degrees of freedom, i.e., three independently moving joints, and the other fingers except the thumb may be modeled with four degrees of freedom, i.e., four independently moving joints. In accordance with another example embodiment of the present invention, a wrist joint, however, may be excluded from being modeled. Accordingly, as illustrated inFIG. 1, the thumb and the index finger may be modeled with two and three degrees of freedom, respectively. In other words, for a sensation generated when an object is grabbed, the thumb may be modeled with two degrees of freedom and the other fingers except the thumb may be enough to be modeled even with three degrees of freedom. As explained above, the forces generated at points of contact may be converted into torques of joints depending on the modeling. For example, for a thumb modeled with two joints, two torques applied respectively to the two joints may be considered and for an index finger modeled with three joints, three torques applied respectively to three joints may be taken account of. So to speak, the number of torques which is same as the number of modeled joints may be considered. Of course, torques corresponding to some joints may be excluded from those to be considered, as the case may be.

In accordance with one example embodiment of the present invention, just as illustrated in the central drawing120inFIG. 1, the torques applied to joints of an index finger may be expressed as three torques τ11, τ12, and τ13and those applied to joints of a thumb may be expressed as two torques τ21and τ22. In other words, torques τ1and τ2may be expressed as torques of the joints of the fingers as shown in equations 4 and 5 below.
τ1=[τ11τ12τ13]T<Equation 4>
τ2=[τ21τ22]T<Equation 5>

In other words, torques τ1of three joints of the index finger may be calculated and torques τ2of two joints of the thumb may be calculated. In case of the index finger, the torques of three joints may be calculated under the Equation 2 as stated above. In addition, τ which represents a matrix transpose in the Equations 4 and 5 is used to show that τ1and τ2are column vectors, respectively.

In accordance with one example embodiment of the present invention, under the hydrostatic equation expressed as JT, if the index finger is modeled with three joints and only three linear forces as external forces are considered, a matrix J1may be a (3×3) matrix and the transpose matrix J1Tmay also a (3×3) matrix. Besides, if the thumb is modeled with two joints and only three linear forces as external forces are considered, a matrix J2may be a (3×2) matrix and the transpose matrix J2Tmay be a (2×3) matrix.

In accordance with one example embodiment of the present invention, even if the user does not actually grab a real object, an effect as if the user grabbed the real object may be provided by applying the torques to the user's joints. In other words, when torques are applied to the user's joints, the user may feel a sensation of grabbing the real object. Besides, a virtual sensation such as the volume and the elasticity of the object may be created by the torques applied to the user's joints.

In accordance with one example embodiment of the present invention, as shown in a right drawing130ofFIG. 1, apparatuses131,132,133,134, and135for creating torques are disposed on joints. The apparatuses for creating the torques may be installed on individual joints and the installed apparatuses may apply torques to the joints respectively by generating pulling or pushing forces at the respective joints. For example, the apparatuses installed on five individual joints may apply torques to the individual joints by generating pulling or pushing forces on upper or lower parts of the joints. An example of generating pulling or pushing forces on upper or lower parts of the joints will be explained later.

FIG. 2is a drawing exemplarily a configuration of an apparatus for creating torques and a method for operating it in accordance with one example embodiment of the present invention.

An apparatus200for creating torques may include a control part230and a torque-applying part240. In accordance with one example embodiment of the present invention, the apparatus200for creating torques may also include a force-calculating part210and a torque-calculating part220.

(i) First of all, in accordance with one example embodiment of the present invention, the force-calculating part210may calculate at least one force vector received from at least one point of contact to a virtual object if the user takes a motion of grabbing the virtual object at a step of S250. In other words, the force-calculating part210may calculate at least one force due to an object. At the time, the applied forces may depend on characteristics of the virtual object such as size or material.

In accordance with one example embodiment of the present invention, the force-calculating part210may be a physics engine or part of it. In addition to the calculation of the forces, the physics engine may handle interactions among virtual objects. For example, the force-calculating part210may calculate the forces by using mass-spring-damper (MSD) models and finite difference methods (FDM) with meshes. Based on the calculation of the forces, the sizes and the direction of the forces may be calculated.

Besides, the force-calculating part210in accordance with one example embodiment of the present invention may use a sensor (non-illustrated) attached to a mechanical device (such as a robot) capable of grabbing an actual object. The force-calculating part210may calculate the sizes and directions of the forces by using information transferred from the sensor when the mechanical device grabs the actual object.

(ii) Next, the torque-calculating part220may calculate torques corresponding to the forces at a step of S260. As explained above inFIG. 1, the forces generated at points of contact may be converted to the torques under the Principle of Virtual Work. In other words, the torques corresponding to the forces may be torques of values created by being converted under the Principle of Virtual Work. The torque-calculating part220may calculate the torques corresponding to the calculated forces by converting the forces to the torques under the Principle of Virtual Work.

The torque-calculating part220may provide information on currents depending on the calculated torques to the control part230.

To calculate the forces and the torques, the force-calculating part210and the torque-calculating part220may use the aforementioned methods or equations by referring toFIG. 1.

(iii) In accordance with one example embodiment of the present invention, the control part230may receive calculation information to be used to create a control signal for controlling the torque-applying part240, including one or more torque-applying units, at a step of S270. For example, the control part230may receive information on currents to be flowed through individual coils310(included in the torque-applying part240that will be explained later) from the aforementioned torque-calculating part220. However, the apparatus200for creating the torques may directly calculate the forces and the torques, but, in another case, the apparatus200may also receive information on the currents corresponding to the forces or the torques from the outside without performing the steps of S250and S260.

(iv) Next, the control part230in accordance with one example embodiment of the present invention may transmit individual control signals to the individual coils310included in the respective torque-applying units installed in individual joints by referring to the received calculation information and then support the individual joints to feel pulling or pushing forces at a step of S280. For example, the control part230may cause interactions between the coils and the magnets (pulling or pushing forces) by controlling the currents corresponding the individual coils310included in the respective torque-applying units by reference to information on the currents received from the torque-calculating parts220. Accordingly, it may apply the torques to the user's joints. The torque-applying units may enable the user to feel a sensation of actually grabbing the object by applying the torques to the respective user's joints.

In accordance with one example embodiment of the present invention, the points of contact, for example, may be the user's fingertips and the joints may be the user's finger joints. The joints to which the torques are applied may be one or more joints moved by the forces applied to the points of contact among all the user's joints.

Meanwhile, as described above, the control part230may control directions and strengths of magnetic fields produced by the coils310to be explained later by controlling currents flowing through the individual coils310included in the respective torque-applying units worn on the individual joints, but it is not necessary for the control part230to control the currents. As another example, if the torques of the individual joints caused by the forces that a body of a user receives from a virtual object has been calculated by a separate computing device, the control part230may also perform a function of allowing the torques to be applied to the individual joints by delivering the control signals corresponding to the calculated torques to the respective torque-applying units.

At the time, as explained below, the respective torque-applying units worn on the individual joints could include respective pairs of coil and magnet but they are not limited to these. They would be certain apparatuses that may receive the control signals from the control part230and may apply the torques to joints depending on the control signals.

In accordance with one example embodiment of the present invention, the torque-applying part240may correspond to apparatuses for applying the aforementioned torques to the individual joints by referring toFIG. 1.

However, for convenience of explanation, explanation will be made below mainly on assumption that the respective torque-applying units include respective pairs of coil and magnet.

In accordance with one example embodiment of the present invention, the torque-applying part240may apply the torques to the user's joints through interactions between magnetic fields produced by the currents flowing through the coils and those generated by the magnets corresponding thereto. A method for applying the torques through the magnetic fields in accordance with one example embodiment of the present invention will be explained in details by referring toFIG. 3as shown below.

There may be multiple torque-applying units. For example, the torque-applying units may be worn on the user's individual fingers. In addition, they may be worn on the individual finger joints. The apparatus200for creating a torque may generate torque to individual fingers of the user, i.e., individual finger joints of the user, through the multiple torque-applying part. The user may feel a sensation of grabbing an object by torque applied to the user's multiple finger joints.

FIG. 3explains a configuration of a torque-applying part in accordance with one example embodiment of the present invention.

In accordance with one example embodiment of the present invention, the torque-applying units included in the torque-applying part240may include a coil310and a magnet320, respectively.

In accordance with one example embodiment of the present invention, the coil310may produce a magnetic field by flowing a specific current therethrough. The torque-applying part240may generate the torques to be applied to the user's joints by using pulling and pushing forces from the interaction of the magnetic fields produced by the currents flowing through the coils and those by the magnets320.

In accordance with one example embodiment of the present invention, a magnetic force of a magnetic field may be controlled by a current. For example, the magnetic force generated by the coil310may be defined as shown in Equation 6 as shown below.
m=I×N<Equation 6>

where m is a magnetic force; I is a current; and N is the number of coil turns. The direction of the magnetic field may be determined under the right hand rule. Accordingly, the pulling or pushing forces acting on the joints may be adjusted depending on the direction of currents flowing through the coils310. For example, the control part230may support the torque-applying units to apply either the torque of the pulling force or that of the pushing force, respectively, by adjusting the directions of the currents flowing through the coils310.

In accordance with one example embodiment of the present invention, a force F may be generated by a magnetic force m1generated by the coil310and a magnetic force m2generated by the permanent magnet under Coulomb's law. Of course, “the permanent magnet” could be replaced with an electromagnet by using another coil, but explanation will be omitted herein. Equation 7 below may represent relationships among m1, m2, and F.

where r may be a rotating radius. The rotating radius may be a distance from a rotation axis to the coil310or to the magnet320.

Besides, τ, the torques working on the joints, in accordance with one example embodiment of the present invention may be calculated under Equation 8 as shown below.
τ=r×F<Equation 8>

In accordance with one example embodiment of the present invention, as illustrated inFIG. 3, each of the torque-applying units may include multiple pairs of coil and magnet. If one of the torque-applying units includes multiple pairs of coil and magnet, the multiple coils and magnets may be used to generate pulling forces or pushing forces. At the time, all the torques may be a total of torque components generated by all the pairs of coil and magnet320. Of course, one of the torque-applying units could be also configured by using only one pair of coil and magnet.

By referring toFIG. 3, it is illustrated that each of the torque-applying units includes two pairs of coil and magnet. Among them, one pair of coil and magnet may be placed on a side where a joint is bent, e.g., a palm side, and the other pair of coil and magnet may be placed on the opposite side, e.g., the back of the hand. The two pairs of coil and magnet may be placed each other in center of a joint. The two pairs of coil and magnet could apply the torques to the corresponding joint in opposite directions with each other. In other words, when the torque-applying unit on the palm side generates a pulling force, and the torque-applying unit on the side of the back of the hand generates a pushing force, the corresponding finger joint could be bent inwardly with a larger value of torque.

In accordance with one example embodiment of the present invention, a device with a proper structure is required to actually apply the torques to the user's joints. In other words, the torque-applying part240may have a structure appropriate to apply the torques to the joints. For instance, the torque-applying part240may have the torque-applying units with a shape of rings or a glove to apply the torques to the user's finger joints. The ring-shaped torque-applying part240may be worn on each finger joint. Or the glove-shaped torque-applying part240may be worn on the user's hand such that the torques are applied to the individual finger joints of the user's hand. After the apparatus200capable of applying the torques to the individual finger joints is worn on the hand or the fingers, if the torques is generated by an interaction between the coils310and the magnets320, the user's hand or fingers may be moved according to the generated torques. Besides, the user may feel a sensation as if the user grabbed an object through the torques.

FIGS. 4A and 4Bexplain how to create torque by controlling a magnetic field due to a direction of current and that generated by a magnet.

FIG. 4Aillustrates that a current flows through the coil310in a certain direction andFIG. 4Bshows that a current flows through the coil310in a direction opposite to that inFIG. 4A.

In accordance with one example embodiment of the present invention, if either of the N and S poles of the magnet is considered as plus (+), and the other pole is as minus (−), signs of m1and m2of the Equation 7 above may be decided. InFIGS. 4A and 4B, the N pole and the S pole are set as plus (+) and minus (−), respectively.

InFIGS. 4A and 4B, it is assumed that, if the magnet320is on the right of the coil310, there are the N pole of the magnet320near the coil310and the S pole thereof far from the coil310. Meanwhile, if the current flows through the coil310, the direction of the magnetic force may be decided depending on a direction in which the coil is wound and under the right hand rule. For example, it can be found inFIG. 4Athat a pushing force is generated between a magnet field formed around a coil and that of the magnet320under the right hand rule and it can be found inFIG. 4Bthat a pulling force is generated.

Accordingly, a size and a direction of a force between the coil310and the magnet320may be adjusted by controlling a strength and a direction of a current flowing through the coil310.

FIG. 5illustrates a ring-shaped torque-applying unit in accordance with one example embodiment of the present invention.

The torque-applying unit could have a shape of ring which is put on a position of the user's finger joint and worn on the joint.

For rotation in center of a rotation axis, the torque-applying part240may include a first ring510and a second ring520. For example, the coil310may be attached to the first ring510and the magnet320may be attached to the second ring520. InFIG. 5, a state before the first ring510and the second ring520are combined with each other is illustrated in the upper drawing and a state after the combination of the first ring510and the second ring520is shown in the lower drawing.

The first ring510and the second ring520may have a bulge, respectively. There may be two bulges, and the two bulges may be formed at opposite sides in center of the rings. The first ring510and the second ring520may be combined with each other through the bulges of the first ring510and the second ring520and the center of the combined bulges may be a central axis of rotation of the first ring510and the second ring520.

FIG. 6illustrates a floor plan of the ring-shaped torque-applying part in accordance with one example embodiment of the present invention.

FIG. 6illustrates a view from the top of the ring-shaped torque-applying part240ofFIG. 5. InFIG. 6, the shapes before the first ring510and the second ring520are combined and those after that are illustrated, respectively, in the upper and lower drawings.

In accordance with one example embodiment of the present invention, as explained above, as the bulges of the first ring510and the second ring520are combined with each other, the center of the combined bulges may play a rotation axis. InFIG. 6, a dashed line passing through the bulges of the first ring510and that of the second ring520may be represented as a rotation axis, and may be a virtual connection line between the first ring510and the second ring520. The combination of the first ring510and the second ring520may do relative rotational movement in center of the rotation axis and the coil310and the magnet320may create a rotational force capable of rotating the first ring510and the second ring520.

In accordance with one example embodiment of the present invention, the coil310and the magnet320may be attached to the outsides of the first ring510and the second ring520, respectively, to prevent them from giving influence over a motion of a first finger. Besides, the coil310and the magnet320may allow the rotation axis to be rotated in a certain direction by generating a pulling force or a pushing force through an interaction.

In accordance with one example embodiment of the present invention, one torque-applying unit may include multiple pairs of the coil310and the magnet320. One pair of the coil310and the magnet320may be attached to the first ring510and the other pair thereof to the second ring520.

In accordance with one example embodiment of the present invention, there may be two pairs of the coil310and the magnet320as well. One of the two pairs of the coil310and the magnet320may be placed on a side where a finger joint is bent, e.g., a palm side, and the other may be placed on the site opposite to the bent side, e.g., the back of the hand. The two pairs of the coil310and the magnet320may be formed to apply the torques in opposition directions with each other against the joint. For example, currents in different directions may be provided to the two different coils. Of course, as mentioned above, the one torque-applying unit may include only one pair of the coil310and the magnet320as well. At the time, the one pair of the coil310and the magnet320could be attached side by side either on the top of the first ring510and the second ring520or on the bottom thereof.

FIG. 7shows a glove-shaped apparatus for creating torques in accordance with one example embodiment of the present invention.

In accordance with one example embodiment of the present invention, the apparatus200may be configured by using a shape of glove. The apparatus200may further include a glove710. The torque-applying units may be attached to the glove710such that the torque-applying units are disposed on the individual finger joints. The respective torque-applying units may be worn on the individual joints. In addition, one of the torque-applying units may have one or more pairs of the coil310and the magnet320.

In accordance with one example embodiment of the present invention, if the apparatus200is glove-shaped, it could reduce inconvenience that might be caused while multiple rings are worn or taken off.

In accordance with the present invention, a user's body may be caused to react as if the user grabbed an object by applying the torques to the user's joints and thus the user may feel sensation as if the user grabbed an object even though the user actually does not.

In accordance with the present invention, the user may feel virtual sensation without severely limiting the user's motions.

The embodiments of the present invention as explained above can be implemented in a form of executable program command through a variety of computer means recordable to computer readable media. The computer readable media may include solely or in combination, program commands, data files, and data structures. The program commands recorded to the media may be components specially designed for the present invention or may be usable to a skilled human in a field of computer software. Computer readable record media include magnetic media such as hard disk, floppy disk, and magnetic tape, optical media such as CD-ROM and DVD, magneto-optical media such as floptical disk and hardware devices such as ROM, RAM, and flash memory specially designed to store and carry out programs. Program commands include not only a machine language code made by a complier but also a high level code that can be used by an interpreter etc., which is executed by a computer. The aforementioned hardware device can work as more than a software module to perform the action of the present invention and they can do the same in the opposite case.

As seen above, the present invention has been explained by specific matters such as detailed components, limited embodiments, and drawings. While the invention has been shown and described with respect to the preferred embodiments, it, however, will be understood by those skilled in the art that various changes and modification may be made without departing from the spirit and scope of the invention as defined in the following claims.

Accordingly, the thought of the present invention must not be confined to the explained embodiments, and the following patent claims as well as everything including variations equal or equivalent to the patent claims pertain to the category of the thought of the present invention.