TACTILE FEEDBACK SYSTEM AND METHOD FOR GENERATING TACTILE FEEDBACK

A tactile feedback system and a method for generating a tactile feedback are provided. The method includes: obtaining, by a tactile feature classification module, a fused tactile signal using 2D image digital data, 3D topography data, and a tactile representative signal corresponding to an object; obtaining, by a tactile signal conversion module, an object surface coordinate from an object surface coordinate processing module, obtaining, by the tactile signal conversion module, a tactile fusion result using the fused tactile signal and the object surface coordinate, and converting, by the tactile signal conversion module, the tactile fusion result into a tactile digital code; and generating, by a tactile feedback actuation module, the tactile feedback using a tactile control signal corresponding to the tactile digital code.

TECHNICAL FIELD

The disclosure relates to a tactile feedback system and a method for generating a tactile feedback.

BACKGROUND

Currently, virtual reality (VR) and extended reality (XR) applications usually only include visual feedback, and there is no effective tactile feedback technology.

SUMMARY

A tactile feedback system and a method for generating a tactile feedback are introduced herein.

A tactile feedback system according to an embodiment of the disclosure includes a tactile feedback control subsystem and a feedback actuation subsystem. The tactile feedback control subsystem includes a tactile feature classification module, a tactile signal database, an object surface coordinate processing module, and a tactile signal conversion module. The tactile signal database stores a tactile representative signal. The feedback actuation subsystem includes a tactile feedback actuation module. The tactile feature classification module obtains a fused tactile signal using 2D image digital data, 3D topography data, and the tactile representative signal corresponding to an object. The tactile signal conversion module obtains an object surface coordinate from the object surface coordinate processing module, the tactile signal conversion module obtains a tactile fusion result using the fused tactile signal and the object surface coordinate, and the tactile signal conversion module converts the tactile fusion result into a tactile digital code. The tactile feedback actuation module generates a tactile feedback using a tactile control signal corresponding to the tactile digital code.

A method for generating a tactile feedback according to an embodiment of the disclosure includes the following steps. A fused tactile signal is obtained by a tactile feature classification module using 2D image digital data, 3D topography data, and a tactile representative signal corresponding to an object. An object surface coordinate is obtained by a tactile signal conversion module from an object surface coordinate processing module, a tactile fusion result is obtained by the tactile signal conversion module using the fused tactile signal and the object surface coordinate, and a tactile fusion result is converted into a tactile digital code by the tactile signal conversion module. The tactile feedback is generated by the tactile feedback actuation module using a tactile control signal corresponding to the tactile digital code.

DETAILED DESCRIPTION OF DISCLOSURED EMBODIMENTS

FIG.1is a schematic diagram of a tactile feedback system1aaccording to an embodiment of the disclosure. In the embodiment, a tactile feedback system1amay include a tactile feedback control subsystem20and a feedback actuation subsystem30. In other embodiments, the tactile feedback system1aof the embodiment of the disclosure may further include a sensing subsystem10. In the embodiment, the tactile feedback control subsystem20may include a tactile feature classification module20-1, a tactile signal database20-2, an object surface coordinate processing module20-3, and a tactile signal conversion module20-4, wherein the tactile signal database20-2may store a tactile representative signal. For example, the tactile representative signal is, for example, a parameter such as an object surface roughness, a surface texture feature, a friction coefficient, an object density, an object hardness, and an object elasticity coefficient. In the embodiment, the feedback actuation subsystem30may include a tactile feedback actuation module31. In other embodiments, the sensing subsystem10may include an image sensing module11, and the tactile feedback control subsystem20may further include a tactile imaging module20-5. The image sensing module11is, for example, a 2D camera or a 3D camera. It should be noted here that the sensing subsystem10, the tactile feedback control subsystem20, and the feedback actuation subsystem30of various embodiments of the disclosure may be software and/or firmware codes executed by a processor. In another embodiment, the sensing subsystem10, the tactile feedback control subsystem20, and the feedback actuation subsystem30of various embodiments of the disclosure may be implemented as circuits. The disclosure does not limit the implementation methods of the sensing subsystem10, the tactile feedback control subsystem20, and the feedback actuation subsystem30.

FIG.2is a flowchart of a method for generating a tactile feedback according to an embodiment of the disclosure, wherein the method may be implemented by the tactile feedback system1ashown inFIG.1. Please refer toFIG.1andFIG.2at the same time.

In Step S210, the tactile feature classification module20-1may obtain a fused tactile signal using 2D image digital data, 3D topography data, and the tactile representative signal corresponding to an object. In an embodiment, the image sensing module11may obtain a touch object real-time image when the object is touched, and may obtain a pre-movement position and a post-movement position of the object, wherein the touch object real-time image may include an object surface texture. Then, the object surface coordinate processing module20-3may obtain an object surface coordinate using the pre-movement position and the post-movement position. On the other hand, the tactile imaging module20-5may execute a 2D image digital operation on the object surface texture to obtain the 2D image digital data. Based on this, the tactile imaging module20-5may obtain digital data with gray level contrast. In addition, the tactile imaging module20-5may obtain the 3D topography data using the touch object real-time image. The 3D topography data is, for example, a surface undulation feature of the object. Then, the tactile imaging module20-5may transmit the 2D image digital data and the 3D topography data to the tactile feature classification module20-1. After the tactile feature classification module20-1receives the tactile representative signal from the tactile signal database20-2, the tactile feature classification module20-1may obtain the fused tactile signal using the 2D image digital data, the 3D topography data, and the tactile representative signal. Then, the tactile feature classification module20-1may transmit the fused tactile signal to the tactile signal conversion module20-4.

In Step S220, the tactile signal conversion module20-4may obtain the object surface coordinate from the object surface coordinate processing module20-3, the tactile signal conversion module20-4may obtain a tactile fusion result using the fused tactile signal and the object surface coordinate, and the tactile signal conversion module20-4may convert the tactile fusion result into a tactile digital code.

In Step S230, the tactile feedback actuation module31may generate the tactile feedback using a tactile control signal corresponding to the tactile digital code. In an embodiment, the tactile feedback control subsystem20may include a tactile feedback modulation module20-9. The tactile feedback modulation module20-9may convert the tactile digital code into the tactile control signal. Then, the tactile feedback actuation module31may generate the tactile feedback using the tactile control signal. In an embodiment, the tactile feedback modulation module20-9may include an electric friction control unit20-9a, a deformation control unit20-9b, and a vibration control unit20-9c, wherein the tactile control signal may include an electric friction control signal (X % of the tactile control signal) corresponding to the electric friction control unit20-9a, a deformation control signal (Y % of the tactile control signal) corresponding to the deformation control unit20-9b, and a vibration control signal (Z % of the tactile control signal) corresponding to the vibration control unit20-9c.

FIG.3Ais a schematic diagram of a tactile feedback system1baccording to a second embodiment of the disclosure. Please refer toFIG.1andFIG.3Aat the same time. The difference between the embodiments ofFIG.3AandFIG.1is that in the embodiment ofFIG.3A, the sensing subsystem10may include an inertial sensing module12and a physiological sensing module13, the tactile feedback control subsystem20may include a physiological signal fusion module20-6and a physiological signal fusion database20-7, and the physiological signal fusion database20-7may store a physiological variable representative signal. For example, the physiological variable representative signal is, for example, a parameter such as a surface temperature, a body temperature, an electromyographic signal, an electrocardiographic signal, an electroencephalographic signal, and a respiratory rate. The inertial sensing module12is, for example, an inertial measurement unit (IMU). On the other hand, the physiological sensing module13is, for example, a temperature sensor, a muscle strength sensor, a heart rate sensor, an electroencephalography sensor, or a respiration sensor. Furthermore, the tactile feedback control subsystem20may include a physiological signal conversion module20-8, a physiological feedback module20-10, and a tactile signal generation module20-11, and the feedback actuation subsystem30may include a physiological feedback actuation module32.

In the embodiment ofFIG.3A, the inertial sensing module12may obtain the pre-movement position and the post-movement position of the object, and the inertial sensing module12may obtain an instantaneous speed and an acceleration of the object using the pre-movement position and the post-movement position. Then, the inertial sensing module12may transmit the instantaneous speed and the acceleration to the object surface coordinate processing module20-3. The object surface coordinate processing module20-3may obtain the object surface coordinate of the object using the pre-movement position, the post-movement position, instantaneous speed, and the acceleration.

On the other hand, in the embodiment ofFIG.3A, the physiological sensing module13may obtain a physiological signal value of the object. Then, the physiological signal fusion module20-6may obtain a fused physiological signal using the physiological signal value (received from the physiological sensing module13), the object surface coordinate (received from the object surface coordinate processing module20-3), and the physiological variable representative signal (received from the physiological signal fusion database20-7). Then, the physiological signal conversion module20-8may convert the fused physiological signal into a physiological digital code. Then, the tactile signal conversion module20-4may obtain the tactile fusion result using the fused tactile signal, the object surface coordinate, and the physiological digital code. Furthermore, after the physiological feedback module20-10obtains the physiological digital code from the physiological signal conversion module20-8, the physiological feedback module20-10may convert the physiological digital code into a physiological control signal. Then, the physiological feedback actuation module32may generate a physiological feedback using the physiological control signal.

It should be noted that in the embodiment ofFIG.3A, the tactile signal generation module20-11may generate the updated tactile representative signal using the fused tactile signal. Then, the tactile signal generation module20-11may store the updated tactile representative signal in the tactile signal database20-2. In other words, if the fused tactile signal received by the tactile signal generation module20-11from the tactile feature classification module20-1is not stored in the tactile signal database20-2in advance, the tactile signal generation module20-11may generate the updated tactile representative signal, and store the updated tactile representative signal in the tactile signal database20-2.

FIG.3Bis a schematic diagram of a tactile feedback system1caccording to a third embodiment of the disclosure. Please refer toFIG.3BandFIG.3Aat the same time. The difference between the embodiments ofFIG.3BandFIG.3Ais that in the embodiment ofFIG.3B, the sensing subsystem10does not include the inertial sensing module12.

FIG.3Cis a schematic diagram of a tactile feedback system1daccording to a fourth embodiment of the disclosure. Please refer toFIG.3CandFIG.3Aat the same time. The difference between the embodiments ofFIG.3CandFIG.3Ais that in the embodiment ofFIG.3C, the sensing subsystem10does not include the physiological sensing module13, the tactile feedback control subsystem20does not include the physiological signal fusion module20-6, the physiological signal fusion database20-7, the physiological signal conversion module20-8, and the physiological feedback module20-10, and the feedback actuation subsystem30does not include the physiological feedback actuation module32.

FIG.4Ais a schematic diagram of a tactile feedback system1eaccording to a fifth embodiment of the disclosure. Please refer toFIG.4AandFIG.3Aat the same time. The difference between the embodiments ofFIG.4AandFIG.3Ais that in the embodiment ofFIG.4A, the tactile feedback control subsystem20does not include the tactile signal generation module20-11.

FIG.4Bis a schematic diagram of a tactile feedback system if according to a sixth embodiment of the disclosure. Please refer toFIG.4BandFIG.3Aat the same time. The difference between the embodiments ofFIG.4BandFIG.3Ais that in the embodiment ofFIG.4B, the tactile feedback control subsystem20does not include the tactile signal generation module20-11and the physiological signal fusion database20-7.

FIG.5Ais a schematic diagram of a tactile feedback system1gaccording to a seventh embodiment of the disclosure. Please refer toFIG.5AandFIG.3Aat the same time. The difference between the embodiments ofFIG.5AandFIG.3Ais that in the embodiment ofFIG.5A, the sensing subsystem10may include an object selection module14. The object selection module14may judge whether an object80is touched. The object80is, for example, a virtual object. When the object selection module14judges that the object80is touched, the tactile feedback system1gshown inFIG.5Amay execute the operation process of the tactile feedback system1bshown inFIG.3A.

FIG.5Bis a schematic diagram of a tactile feedback system1haccording to an eighth embodiment of the disclosure. Please refer toFIG.5BandFIG.3Aat the same time. The difference between the embodiments ofFIG.5BandFIG.3Ais that in the embodiment ofFIG.5B, the sensing subsystem10may include an object selection module14. The object selection module14may judge whether an object80is touched. When the object selection module14judges that the object80is touched, the physiological sensing module13may receive a surface temperature of the object80from the object80. Then, the tactile feedback system1hshown inFIG.5Bmay execute the operation process of the tactile feedback system1bshown inFIG.3A.

FIG.6Ais a schematic diagram of a tactile feedback system1iaccording to a ninth embodiment of the disclosure. Please refer toFIG.6AandFIG.3Aat the same time. The difference between the embodiments ofFIG.6AandFIG.3Ais that in the embodiment ofFIG.6A, the tactile feedback control subsystem20may be disposed in a cloud. On the other hand, the sensing subsystem10and the feedback actuation subsystem30may be disposed on the same local end. For example, the sensing subsystem10may be connected to the tactile feedback control subsystem20via a wireless network, and the tactile feedback control subsystem20may be connected to the feedback actuation subsystem30via a wireless network.

FIG.6Bis a schematic diagram of a tactile feedback system1jaccording to a tenth embodiment of the disclosure. Please refer toFIG.6BandFIG.3Aat the same time. The difference between the embodiments ofFIG.6BandFIG.3Ais that in the embodiment ofFIG.6B, the tactile signal database20-2and the tactile signal generation module20-11in the tactile feedback control subsystem20may be disposed in a cloud. On the other hand, other modules (except for the tactile signal database20-2and the tactile signal generation module20-11) in the tactile feedback control subsystem20, the sensing subsystem10, and the feedback actuation subsystem30may be disposed on the same local end.

FIG.6Cis a schematic diagram of a tactile feedback system1kaccording to an eleventh embodiment of the disclosure. Please refer toFIG.6CandFIG.4Aat the same time. The difference between the embodiments ofFIG.6CandFIG.4Ais that in the embodiment ofFIG.6C, the tactile signal database20-2in the tactile feedback control subsystem20may be disposed in a cloud. On the other hand, other modules (except for the tactile signal database20-2) in the tactile feedback control subsystem20, the sensing subsystem10, and the feedback actuation subsystem30may be disposed on the same local end.

The following will describe implementation examples of the tactile imaging module20-5, the tactile feature classification module20-1, the tactile signal conversion module20-4, and the tactile feedback modulation module20-9in the tactile feedback control subsystem20of the disclosure.

In an embodiment, the tactile imaging module20-5may execute a shadow removal operation on the touch object real-time image when the object is touched to obtain a shadow-removed touch object real-time image, and a gray level co-occurrence matrix (GLCM) may be generated using the shadow-removed touch object real-time image, wherein the gray level co-occurrence matrix may include multiple elements. Then, the tactile imaging module20-5may calculate a gray level value of each of the elements, and may calculate an eigenvalue of each of the elements using the gray level value, wherein the eigenvalue may include an energy value, an entropy value, a contrast value, a correlation value, and a contrast score. The gray level values may be classified into 256 levels (levels 0 to 255). For example, it is assumed that Cijrepresents the gray level value of the element in the i-th row and the j-th column in the gray level co-occurrence matrix. A calculation formula of the energy value may be ΣiΣjCij2, a calculation formula of the entropy value may be −ΣiΣjCijlog Cij, a calculation formula of the contrast value may be ΣiΣj(i−j)Cij, a calculation formula of the correlation value may be

and a calculation formula of the contrast score may be

Then, the tactile imaging module20-5may use the gray level co-occurrence matrix at various angles, such as 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

Then, the tactile feature classification module20-1may calculate a gray level proportion, a gray level value change topography, and a gray level gradient change corresponding to the touch object real-time image using a finger pulp contact area, and may judge an interactive relationship and a degree of smoothness of the object. Furthermore, the tactile feature classification module20-1may judge whether a contact surface of the object is an edge structure using a contact area gray level value corresponding to the finger pulp contact area.

FIG.7Ais a schematic diagram of the tactile feature classification module20-1judging whether the finger is stationary or moving according to an embodiment of the disclosure.FIG.7Bis a schematic diagram of the tactile feature classification module20-1judging whether the object is stationary or moving according to an embodiment of the disclosure. Please refer toFIG.1,FIG.7A, andFIG.7Bat the same time. The tactile feature classification module20-1may adopt a monitoring matrix, and define a pulp contact area size representing one finger as the central position. Then, the tactile feature classification module20-1may calculate the gray level proportion (to judge a texture form), the gray level value change topography, and the gray level gradient change, and may judge the interactive relationship between the finger and the object or may judge the degree of smoothness of the object. On the other hand, the tactile feature classification module20-1may judge whether the contact surface is the edge structure using the contact area gray level value of one-half of a finger pulp area in the touch object real-time image.

Furthermore, the tactile feature classification module20-1may classify tactile features into multiple categories, wherein the categories may include roughness features, smoothness features, soft/hard elasticity features, viscosity features, and temperature features. Furthermore, each category may be further divided into 3 levels. Specifically, the roughness features are, for example, to distinguish between textures or particle sizes of surfaces of objects. The smoothness features are, for example, to distinguish between degrees of smoothness of surfaces of objects. The soft/hard elasticity features are, for example, to distinguish between different degrees of hardness, softness, elasticity, etc. of surfaces of objects. The viscosity features are, for example, to distinguish between degrees of viscosity of surfaces of objects. The temperature features are, for example, to distinguish between cold (less than 20° C.), warm (20° C. to 50° C.), and hot (greater than 50° C.) of surfaces of objects. Then, the tactile feature classification module20-1may add the viscosity features and the temperature features to the gray level co-occurrence matrix to obtain the fused tactile signal. In other words, the general gray level co-occurrence matrix is usually only used for gray value processing of images, while the tactile feature classification module20-1of the disclosure may add the viscosity features and the temperature features to the gray level co-occurrence matrix to obtain the fused tactile signal.

After the tactile feature classification module20-1obtains the fused tactile signal, the tactile signal conversion module20-4may obtain the tactile fusion result using the fused tactile signal and the object surface coordinate, and the tactile signal conversion module20-4may convert the tactile fusion result into the tactile digital code. Then, the tactile feedback modulation module20-9may convert the tactile digital code into the tactile control signal.

FIG.8is a schematic diagram of the tactile feedback modulation module20-9converting the tactile digital code into the tactile control signal according to an embodiment of the disclosure. Please refer toFIG.1andFIG.8at the same time. In the embodiment, the tactile control signal may be associated with frequency and tactile sensation. Specifically, the tactile feedback modulation module20-9may divide the frequency into different intervals using 100 Hz, 300 Hz, and 500 Hz, and may divide the tactile sensation into different intervals using 25%, 50%, 75%, and 100%. Then, the tactile feedback modulation module20-9may obtain 12 types of the tactile control signal such as S1to S12as shown inFIG.8. The tactile control signal may include the electric friction control signal (X % of the tactile control signal) corresponding to the electric friction control unit20-9a, the deformation control signal (Y % of the tactile control signal) corresponding to the deformation control unit20-9b, and the vibration control signal (Z % of the tactile control signal) corresponding to the vibration control unit20-9c. Specifically, the respective ranges of X %, Y %, and Z % above may respectively be 0% to 100%. Then, the tactile feedback actuation module31may generate the tactile feedback using the tactile control signal.

In summary, the tactile feedback system and the method for generating the tactile feedback of the embodiments of the disclosure may obtain the tactile fusion result using the fused tactile signal and the object surface coordinate to convert the tactile digital code, and generate the tactile feedback using the tactile control signal. In particular, the tactile control signal of the embodiment of the disclosure may include the electric friction control signal, the deformation control signal, and the vibration control signal. Therefore, the tactile feedback system and the method for generating the tactile feedback of the embodiments of the disclosure can provide an effective/highly realistic tactile feedback.