Modular electrotactile system and method

This application relates to an electrotactile system and method for delivering tactile stimuli to a skin surface of a user. The system includes one or more electrotactile modules each comprising an array of electrodes electrically connected to an integrated circuit. Each integrated circuit has data processing and current driving capability. The current delivered from the integrated circuit to each electrode is relatively small, preferably less than 4 milliamps. The modules may be connected to a flexible PC board by spring-loaded connectors. In one embodiment a plurality of electrotactile modules may be grouped together to from an electrotactile device. Multiple electrotactile devices may in turn be deployed as part of a wearable article for use in virtual reality, telepresence, telerrobotics or other haptic feedback applications. The system is capable receiving and transmitting tactile data via a communication link, such as a conventional data network. For example, tactile data can be transmitted in a scalable streaming format from a remote site to the system via a data network. The system may form part of a virtual reality entertainment application.

TECHNICAL FIELD

This application relates to an electrotactile system and method for delivering tactile stimuli to a skin surface of a user. The system comprises electrotactile modules each comprising an integrated circuit electrically connectable to an array of electrodes. The modules may be connected together to form an electrotactile device. A plurality of devices may in turn be deployed as part of a wearable article for use in virtual reality, telepresence, telerrobotics or other haptic feedback applications. The system is capable of receiving and transmitting data via a communication link.

BACKGROUND

Interactive haptic feedback systems are known in the prior art for delivering tactile data to a user, such as sensations of force and touch. For example, haptic feedback has been used in the past in various aerospace, surgical and defence applications for the purpose of controlling remote robotic vehicles and manipulators. More recently, haptic feedback has been used in telespresence and virtual reality applications. Telepresence refers to the experience or impression of being present at a real-world location remote from one's own immediate environment. Virtual reality is a similar concept except that its goal is to immerse the user in a simulated, computer-generated world, often for the purposes of entertainment. In all of these applications tactile data is conveyed to a user via human interface devices, such as a gloves or exoskeletans. However, these interface devices are typically bulky or awkward to use or are adapted for very specific applications.

Four different types of mechanoreceptors in the human skin detect tactile stimuli, namely the Meissner corpuscle, the Merkel cell, the Pacinian corpuscle and the Ruffini ending. The Meissner corpuscles and Merkel cells detect light touch and are located beneath the surface of the skin approximately 0.7 mm. Both of these mechanoreceptors transduce very slight inputs of mechanical energy into action potentials. Pacinian corpuscles are located deeper in the skin, typically about 2 mm below the skin surface, and are a type of pressure sensor stimulated by strong pressure. Ruffini endings are a type of mechanoreceptor located around the base of hairs and detect hair movements. Most prior art haptic systems developed thus far do not stimulate human skin mechanoreptors selectively or precisely. Such systems are therefore unsuitable for virtual reality and similar applications where fidelity of tactile sensations is critical.

Some prior art telepresence or telerobotic systems rely on applying vibrotactile stimulation to the user. U.S. Pat. No. 5,619,180, Massimino et al., dated Apr. 8, 1997 describes a system for generating a feedback signal corresponding to a force sensed by an effector in a remote environment. The feedback signal is delivered to the local site of the operator where it is transduced into a vibrotactile sensory substitution signal to which the operator is sensitive. Vibrotactile display elements can be located on the operator's limbs, such as the hands, fingers, arms or legs. The operator therefore “feels” the forces that the effector senses to some degree depending upon the fidelity of the force sensing and reproduction system. Previous artificial tactile displays of this sort have been limited primarily to homogenous arrays of relatively small vibrators that provide low-amplitude, high frequency stimulation of the tactile system. Such vibrotactile systems are useful for some applications, but they do not enable a highly sensitive localized response.

U.S. Pat. No. 4,655,673, Hawkes, dated Apr. 7, 1987, describes a telerobotic apparatus comprising a vibration sensitive transducer. The output signal from the transducer is converted to audible sounds which can be intuitively interpreted by the operator as indicators of texture, hardness and the like. Other tactile display developers have proposed acoustic feedback systems using surface acoustic waves to apply shear stresses to a finger surface of a user.

Prior art devices that rely on applying pneumatic stimulation to the user are also known in the prior art. Some prior art systems direct compressed air to a skin surface in the form of air jets, air cuffs or air bellows. For example, Robert Stone conceived a pneumatic bellows glove in 1989, referred to as Teletact, that employed a plurality of air pockets to provide tactile feedback to the fingers and palm of a user.

Electrotactile or electrocutaneous systems are also known for delivering electrical current to electrodes placed on the user's skin to induce a tactile response. U.S. Pat. No. 4,926,879, Sevrain et al., dated May 22, 1990 describes an electro-tactile stimulator comprising a flexible substrate on one surface of which is formed an electrically conductive pattern having a number of electrodes which are placed in contact with a skin surface of a user. The primary purpose of such electrotactile devices is to assist hearing or vision impaired persons in interpreting environmental stimuli. The specific object of the Sevrain et al. invention is to achieve this result while avoiding skin irritation caused by electrically induced changes in the pH of skin tissue.

U.S. Pat. No. 4,390,756, Hoffmann et al., dated Jun. 28, 1983, relates to an electrocutaneous stimulation apparatus which may be used as a hearing prosthesis by deaf individuals. According to this invention acoustical signals, in particular speech sounds, are encoded into electrocutaneous stimulation patterns. The tactile stimulations are applied via skin surface electrodes, for example on the user's forearm, and can be interpreted as speech information.

U.S. Pat. No. 5,957,812, Harrigan, dated Sep. 28, 1999, relates to an electrical muscle stimulation (EMS) device configured as a vending machine that allows a user to control the amount of electronic impulses required to stimulate the user's muscles to contract and exercise. Other similar electrocutaneous devices for electrically stimulating injured tissue are well-known in the medical and rehabilitation fields. For example, totally implanted pulse generator (IPG) and radio frequency (RF) electrocutaneous systems having been used as spinal cord stimulators and pain relief devices. Subdermal electrocutaneous stimulation has also been used to provide sensory feedback to users of upper extremity neuroprostheses.

Conventional EMS devices, electric massagers and electrocutaneous devices used for tissue rehabilitation purposes and the like employ comparatively few electrodes each applying a relatively large amount of current (e.g. greater than 15 milliamps). The result is the provision of blunt sensations to the user. Most devices are not designed to selectively stimulate all four types of mechanoreceptors in the human skin described above. Moreover, the current applied is pre-programmed and there is no provision for adjustment of the current parameters based on user interactivity or other variable data input.

Although there is a growing awareness of the advantages of electro-neuromodulation therapy for medical or rehabilitative purposes, thus far electrotactile or electrocutaneous devices have not been effectively applied in the information and entertainment industries, such as part of a virtual reality computer application. The need has therefore arisen for an improved electrotactile system useful for virtual reality simulations and the like which overcomes the various limitations of the prior art. Unlike conventional devices, applicant's invention is capable of inducing very sensitive tactile sensations in a user, has network data transfer capability and relatively low current and power requirements, and is operable bidirectionally as both a tactile sensation input and feedback device. Other features and advantages of the invention are described below.

SUMMARY OF THE INVENTION

In accordance with the invention, an electrotactile module for delivering tactile stimuli to a skin surface of a user is provided. The module comprises a housing having a first surface and a second surface, an array of electrodes arranged on the first surface for contacting the user's skin, and an integrated circuit electrically connected to the electrodes for independently providing electrical current to each of the electrodes in a predetermined control sequence.

Preferably the first surface contacting the user's skin is outwardly curved. A PC board, which is preferably flexible, is connected to the second surface. The device may further include at least one connector for connecting the module to the PC board. In one embodiment of the invention, the connector is spring-loaded. Multiple spring-loaded connectors for connecting the module to the PC board may also be provided.

The integrated circuit has data processing and current driving capability and is preferably located within the housing. The current delivered from the integrated circuit to each of the electrodes is preferably small, for example less than 4 milliamps. The predetermined control sequence may be received by the integrated circuit from a remote source, such as via a conventional data network.

A group of electrotactile modules may be operatively coupled together to form an electrotactile device. The device may include the PC board as well as power and data connectors. Multiple electrotactile devices may in turn be deployed in a wearable article to maintain the electrotactile devices in contact with the user's skin. For example, the wearable article may comprise a plurality of pockets for receiving the electrotactile devices. The wearable article may be constructed from flexible fabric capable of conforming to the contours of the user's body. In one embodiment the wearable article may be adjustable to adjustably position the electrotactile devices in contact with the user's skin. An electronic control device may also be provided for receiving and processing encoded tactile data transmitted from a remote site via a data network. The electronic control device preferably has the same shape and size as the electrotactile devices so that it may also removably fit into a pocket of the wearable article.

A method of delivering an electrical signal to a user wearing an electrotactile device as described above is also disclosed. The method includes the steps of transmitting input signals to the electrotactile device representative of predetermined desired tactile sensations; processing the input signals in the electrotactile device to produce output signals comprising tactile sensation and location data; and delivering current to selected electrodes of the electrode array in a sequence corresponding to the output signals.

The input signals may be generated, for example, by manipulation of a hand-held input device by the user while immersed in a virtual reality application. The input signals may also be generated by movement of the electrotactile device relative to a skin surface of the user. In some applications the input signals are transmitted to the electrotactile device from a remote site, for example in a scalable streaming data format suitable for real-time network transmission.

The method may further include the step of mapping tactile sensation and location data to user parameters stored in memory to adapt the output signals to the particular skin properties of the user.

An electrotactile system for delivering tactile information to a skin surface of a user is also disclosed. The system includes at least one electrotactile module as described above comprising an array of electrodes for contacting the user's skin surface and an integrated circuit electrically connected to the electrodes; a receiver connectable to a data network for receiving input signals comprising encoded tactile data; and a signal processor for decoding the encoded tactile data to produce output signals suitable for transfer to the integrated circuit, the output signals comprising tactile sensation and location data. According to the system, the integrated circuit provides current to the electrodes in accordance with the output signals received from the signal processor.

As indicated above, the tactile data may be encoded at a site remote from the electrotactile device, such as by a virtual reality simulator. The signal processor may further include a mapping algorithm for adjusting the tactile sensation and location data to the specific skin parameters of the user stored in memory.

A tactile feedback subsystem may also optionally be provided for measuring changes in the conductivity of the electrodes caused by changes in the position of the electrodes relative to the skin surface of the user over time.

DETAILED DESCRIPTION

This application relates a modular electrotactile system and method. As used in this patent application the term “electrotactile” refers to the use of electrical stimuli to induce a tactile sensation in a user, such as a sensation of touch or pressure. The terms “electrotactile device” or “electrocutaneous device” both refer to an apparatus that activates nerve axons in the skin by applying electric current directly to the skin surface.

FIGS.1(a)-1(j) illustrate an electrotactile module10developed by the applicant for delivering tactile stimuli to a user. A plurality of modules10may be assembled together to form an integrated electrotactile device12as shown in FIGS.3(a)-3(e). Further, a plurality of devices12may be deployed in a wearable article14for placement in contact with a skin surface of a user as illustrated in FIGS.4(a)-4(d).

As shown best in FIGS.1(a)-1(j), each electrotactile module10comprises a thin housing16having a first surface18and a second surface20. An integrated circuit22is mounted on or within each housing16(FIGS.1(i)-1(j). An array of electrodes24are arranged on first surface18and extend outwardly therefrom a short distance for contacting the user's skin (FIG.2(a)). As explained further below, each of the electrodes24is electrically connected to integrated circuit22such that electrical current may be independently provided to electrodes24in a pre-determined control sequence to induce tactile sensation.

Housing first surface18carrying electrodes24is preferably curved outwardly to conform to the natural contour a surface31of the user's skin as shown best in FIG.2(a). This arrangement enhances user comfort and reduces shear forces along the edges of module10contacting the skin. However, as will be apparent to a person skilled in the art, the curvature of surface18is not a critical feature of the invention and other geometric shapes could be substituted in alternative embodiments of the invention.

As shown best in FIGS.1(b),1(i) and2(a), each electrotactile module10is connected to a PC board26by one or more connectors28extending from housing second surface20. Preferably a plurality of connectors28, arranged for example in a circular array, connect module10and PC board26together. In one embodiment of the invention connectors28are spring-loaded to gently bias electrodes24in contact with surface31of the user's skin. Further, PC board26may be flexible to enable module10to better conform to the user's skin and enhance the comfort of wearable article14.

Integrated circuit22functions as a microprocessor for controlling the delivery of current to electrodes24. In the embodiment illustrated in FIGS.1(i) and1(j), circuit22is deployed on a PC board25. Wire conductors23are provided for connecting circuit22to contact pads21. Each contact pad21contacts a corresponding electrode24when module10is assembled. As shown in FIGS.1(i) and1(j), wire conductors23also electrically connect circuit22to contact pads27. Each contact pad27contacts a corresponding connector28when module10is assembled. As will be apparent to a person skilled in the art, other similar microelectronic circuits could be substituted for circuit22. For example, circuit22could be deployed on a conventional silicon chip. As used in this patent application “integrated circuit” refers to any microelectronic component having data processing and current driving capability.

As shown in FIG.2(a), a plurality of electrotactile modules10may be connected to a single PC board26in one embodiment of the invention. For example, a single flexible PC board26may be used to route data to all of the modules10comprising an electrotactile device12.

The structure of an electrotactile device12is illustrated in detail in FIGS.3(a)-3(e). Device12includes a PC board26and a plurality of modules10as explained above. Device12further includes data connectors29and a data transmitting/receiving unit30mounted on PC board26. As described below, external control data is received by device12and transmitted through data transmitting/receiving unit30to integrated circuits22of individual modules10. Sensory and position feedback data may also be transmitted from modules10through transmitting/receiving unit30to an external processor in a similar manner.

Device12fits within a socket32. In the illustrated embodiment socket32is annular in shape for receiving a device12formed in a circular shape. However, as will be appreciated by a person skilled in the art, other geometric shapes could be selected.

Wearable article14is preferably constructed from a flexible fabric material to conform to the contour of the user's body (FIGS.4(a)-4(d)). In the illustrated embodiment a vest is shown. Wearable article14includes a plurality of pockets or sleeves34each sized to receive a socket32and an accompanying electrotactile device12. Each device12is inserted into its corresponding sleeve34in an orientation wherein modules10extend inwardly in contact or potential contact with the user's skin (FIG.4(a)). In this deployed orientation data connectors29are accessible for connecting device12to an external data source, such as by a network communication link.

In one embodiment of the invention wearable article14may be adjustable to adjustably position electrotactile devices12in contact with a skin surface of a user. For example, article14may be inflatable to provide a comfortable contact fit between devices12and the user's skin surface.

In order to facilitate reception and transmittal of control and feedback data, an electronic control device40is also provided as shown schematically in FIG.5. In one embodiment of the invention control device40has a size and geometric shape similar to an electrotactile device12. Control device40can therefore fit removably into a sleeve34of wearable article14in the same manner as an electrotactile device12. As discussed further below, control device40includes controllers for receiving data in a streaming format and delivering it to the appropriate electrotactile device12.

The network data transfer interface of applicant's invention is shown in FIG.5. Data is transmitted between a data network42and individual electrotactile modules10via electronic control device40and data/address bus44. The means for transmission can be wired or wireless. Each electrotactile device12is essentially a network peripheral capable of receiving and transmitting data, for example, in a scalable streaming format.

As shown inFIG. 5, device40includes a network receiver46for receiving tactile data and control data from network42. Tactile data describes the tactile texture of an object which virtually contacts the skin of the user. Control data describes the sequence of operation of individual electrotactile modules10and electrodes24thereof, including the maximum current of each electrode24and other control parameters. The tactile data may be transmitted in a “frame by frame” or “packet” data structure similar to video data.

Tactile data and control data is passed from network receiver46to a location mapping controller48. Controller48accesses a DRAM type memory50to support its data processing operations and an EEPROM memory52which stores a location mapping table. The purpose of the location mapping is to filter incoming data and to process “tactile pixels” corresponding to discrete tactile locations. For example, in one embodiment of the invention only tactile pixels which have changed after the prior frame will be filtered to pass to the next step in the data processing process and excessive incoming data will be filtered out.

The processed tactile data and control data are passed from the location mapping controller48to a tactile scanning controller54which generates tactile sensation data. Similar to controller48, controller54accesses a DRAM type memory56to support its data processing operations and an EEPROM memory58which stores a tactile sensation mapping table. Tactile data may be provided according to different data protocols depending upon the data source. Tactile scanning controller54maps the input protocol to a data format compatible with the data receiving/transmitting30of each electrotactile device12.

Since skin properties vary for different parts of the body and also vary from one person to another, it is necessary to incorporate an “adjusting algorithm” in tactile scanning controller54. A distribution matrix may be used for distributing tactile current data over a region of neighbouring electrodes24. For example, different skin mechanoreceptors require a different distribution of tactile current to trigger tactile sensation.

As indicated above, control electronic device40may be capable of receiving data in a scalable streaming format. In this case the decoding algorithms implemented by controllers48,54preferably ensure that data is delivered to electrotactile devices12at a constant rate and any gaps in the data stream are effectively bridged. Thus electronic device40is preferably programmed to compensate for transmission latency and delays inherent in the communications link. As will be apparent to a person skilled in the art, the specific nature and bandwith of the communication link or other means for transmitting data to devices12from a remote site is not a critical feature of the present invention. As mentioned above, the means for transmission can be wired or wireless.

By way of example, specially adapted tactile hardware may be used to collect or generate time-variant tactile data at a remote site (in a manner similar to time-variant video data). The tactile data is encoded by streaming format encoding hardware. The encoded data is transmitted in a scalable streaming format via a conventional data network. The encoded data is then decoded and converted by an inverse tactile processor, such as by electronic control device40.

Processed data from electronic control device40is delivered to an electrotactile device12via data/address bus44. Since bus44is a common pathway, all the data/address information transmitted by any of the electrotactile devices12or control device40can be potentially received by any of the other devices12connected to bus44. However, only the device12(and modules10) with the designated address will accept and be able to process the information to generate tactile sensation.

Data from electronic control device40is received by each individual electrotactile device12by data receiving/transmitting30. Unit30distributes the data to integrated circuits22of the various electrotactile modules10which comprise each device12. Each integrated circuit22receives the processed data and transforms it into small currents, preferably less than 4 milliamps, which are applied to electrodes24. Accordingly, each module10has both data processing and current driving capability. Further, the current can be applied to each electrode24independently to provide very precise sensory control and to enable the delivery of multi-channel stimuli. Since the current delivered to each individual electrode24is small (i.e. typically less than 4 milliamps) each device12is relatively energy efficient. Accordingly, devices12are well-suited for use in association with portable electronic equipment having power management constraints, such as personal digital assistants or mobile phones. Also, since the current delivered to each individual electrode24is small, each device12is relatively safe to use and is unlikely require rigorous regulatory pre-approval as do conventional electrocutaneous medical devices.

In order to provide detailed sensory information, a dense array of electrodes24are desirable (on the order of 104). However, the total number of electrotactile devices (and electrodes24) may vary depending upon the application. For each electrode24, the stimuli from threshold to about 50 dB can be delivered throughout the frequency range from near DC to about above 200 dHz.

By way of example, an electrode array on each module10, or group of modules10, may consist of between approximately 8×8 electrodes24(i.e. 64 electrodes in total) to 24×24 electrodes24(i.e. 576 electrodes in total). Each electrotactile device12(comprising multiple modules10) may have on the order of 1,000 electrodes24in total, although the number of electrodes24may vary without departing from the invention as indicated above. The pitch (i.e. the distance between respective electrodes24) may be on the order of 2 mm. The maximum operating current through each electrode24may be on the order of 0.8 milliamps (mA). Thus the present invention features a relatively large number of electrodes24each delivering a relatively small current in comparison to prior art electrocutaneous devices. As will be appreciated by a person skilled in the art, the amount of current required to be delivered to each electrode24to generate the desired tactile sensation is dependent on the pitch and density of the electrode array as well as the contact area of the electrode24on the skin.

Electrotactile devices12may be used as part of both a network-enabled sensory input and sensory feedback system for interactive virtual reality applications and the like. FIG.6(a) illustrates schematically a traditional virtual reality system wherein an operator60manually manipulates a control device62such as a joystick or computer mouse. Data processing unit64interprets the user inputs and generates control commands for controlling user sensory feedback devices, such as audio or video devices66.

FIG.6(b) illustrates a virtual reality system modified to include a tactile input and feedback subsystems in accordance with the invention. In this case user inputs received by data processing unit64are interpreted to generate tactile parameters in the form of tactile data and control data in addition to audio or video control signals. As discussed above, the tactile and control data is transmitted to one or more electrotactile devices12, such as via a data network42(FIG.5), where it is processed and transformed into small currents applied to the user's skin to stimulate skin mechanoreceptors and the like. Wearable article14is an example of an application employing such an input subsystem where there is no relative movement between electrotactile device(s)12and the user's skin.

If there is relative movement between electrotactile device(s)12and the user's skin, then devices12may be employed as data feedback devices in addition to input devices. In particular, when the user's skin comes in contact with a device12, the contacted electrodes24form part of an electrically conductive loop while the non-contacted electrodes34behave like open-loop electrodes. A time-variant “current map” of the open-loop electrodes24, which corresponds to the input movements of the user, can be captured by a tactile feedback or subsystem. For example, providing that the current map's relation with time is known, the system can be used to determine the movement of the skin (for instance, the hands) relative to the electrotactile devices12. The captured input information may then be fed back to data processing unit64(FIG.6(b)) to generate modified tactile parameters as discussed above. Thus in this embodiment electrotactile devices12are bi-directional, that is they may function as both input and “man-in-the-loop” feedback devices.

In operation, a plurality of electrotactile modules10may be assembled together as described above to form an integrated electrotactile device12. Depending upon the application, a plurality of electrotactile devices12may in turn be electrically coupled together and placed in contact with a skin surface of the user, such as by means of a wearable article14. Each electrotactile device12is capable of receiving data from an electronic control device40removably positioned on or within wearable article14. To this end, each device12includes data and power connectors29as described above to enable passage of data and electrical power to respective modules10.

In one embodiment of invention wearable article14(including electrotactile devices12) may function as a network peripheral. As described above, tactile and control data generated at a remote site may be delivered to wearable article14via a network communication link, such as in a scalable streaming data format. The data may, for example, correspond to user input generated in a virtual reality application or to real world haptic data generated by telepresence or telerobotic applications.

The input data is received and decoded by electronic control device40as described above. The decoded, processed data is then delivered via data/address bus44to the data transmitting/receiving unit30of each individual device12which in turn distributes the data to the integrated circuit22of each individual electrotactile module10. Each integrated circuit22receives the decoded data and transforms it into small currents which are applied to electrodes24in the desired pattern and sequence to stimulate tactile sensations.

In one application of the invention electrotactile devices12may also form part of a system for encoding feedback data and transmitting such data to a data processor as part of an interactive application. As described above, if electrotactile devices12are capable of moving relative to the user's skin, the conductivity of individual electrodes24may be mapped during a data acquisition time period to generate an electrical representation of the movements of a user. This time-variant map may in turn be fed back to a processor to generate tactile data based on the user feedback. Depending upon the application, the haptic feedback could be continuous or restricted to discrete time periods.