Transmitting information to a user's body

A method for transmitting information to a body includes providing a set of standardized codes. Each code of the standardized codes is associated with a predefined activation pattern for at least one actuator designed to transmit the information to the body using skin as an interface. A code or a combination of a plurality of codes is selected and transmitted to a wearable accessory having integrated therein the at least one actuator. The at least one actuator is outside a user's body. The at least one actuator is controlled according to the predefined activation pattern associated with the code or the combination of the plurality of codes transmitted. Features about emotional states or states of mind are extracted from biometric and/or environmental signals of biometric and/or environmental sensors. The features are mapped to stored meta information. The meta information is wirelessly transmitted to a receiver.

FIELD AND BACKGROUND OF THE INVENTION

The present invention generally relates the transmission of information to a user's body using the skin as an interface.

BRIEF DESCRIPTION OF THE PRESENT STATE OF THE ART

WO 01/41636 A1 refers to a tactile communication system enabling remotely control of tactile output devices, e.g. motor actuators, which are used for a sensory/tactile stimulation of human muscles, limbs, and body joints by vibrations, warmth electromagnetic impulses, etc. The system enables e.g. parents to transmit touch feelings to their child by placing one hand onto a pressure-sensitive pad and sending control signals derived by evaluating these pressure signals via a computer network to a tactile output device being in physical contact with the child.

WO 98/14860 pertains to a method for transmitting the touch of a sensor array via a computer or computer network and a corresponding system for transmitting perceptible feelings, which comprises at least one computer, a sensor array for detecting touches from a person and providing sensor signals, an actuator array for generating physically perceptible pressure signals, and a control unit linking the sensor array and the actuator array to the computer.

EP 0 875 821 A2, a communication terminal device with a tactile feedback is described. The device comprises a pen-input tablet, a microphone, an acceleration sensor and a number of pressure sensors.

A computerized system including a touch-/pressure-sensitive transducer and a computer responsive thereto for producing a sentic cycle and recording touch expression is disclosed in U.S. Pat. No. 5,305,423. Thereby, different emotions are expressed and generated by applying appropriate finger pressure to specific actuators of said transducer.

PROBLEMS ENCOUNTERED WITH PRIOR-ART SOLUTIONS

Physical expressions of love and affection (a hug, a kiss, a caress, etc.) are only possible in face-to-face communication as they require close proximity of people. Conventional communication systems (e.g. email systems), which enable the expression of feelings with the aid of signs, are oblivious to affective expressions and not capable of transmitting this type of information. Of course, email users can describe emotions, sensations or states of a person's mind with the aid of written or spoken words, symbols (e.g. Internet or SMS emoticons such as “:-)”, “:-|”, or “:-(”), sound or images, but they are not able to physically share these feelings with other email users.

OBJECT OF THE PRESENT INVENTION

In view of the explanations mentioned above, it is the object of the present invention to propose a technique for transmitting information to a user' body.

This object is achieved by means of the features of the independent claims. Advantageous features are defined in the subordinate claims.

SUMMARY OF THE INVENTION

According to one aspect of the present invention a method for transmitting information to a user's body is proposed, said method comprising the following steps:providing a set of standardized codes, each code being associated with a predefined activation pattern for at least one actuator designed to transmit information to a user using the skin as an interface to the user,selecting and transmitting a code or a combination of a plurality of codes to a wearable accessory having integrated therein the at least one actuator, andcontrolling the at least actuator according to the activation pattern associated with the code transmitted.

The actuator can be designed to transmit tactile, virational, pressure, heat stimuli to the skin of a user.

A code can be selected based on a user's manual, visual or audio input or signals originating from biometric sensors.

According to another aspect of the present invention a computer program product is proposed supporting such a method when running on a computing device.

According to a still further aspect of the invention a wearable piece of garment or accessory comprises:an array of actuators designed to transmit stimuli to a user using the skin of the users's body as an interface, anda controller being functionally connected to the array of actuators in order to activate them according to a defined operational pattern.

Finally, a system for transmitting physical expressions of affection comprises:means for generating a coded signal out of a set of coded signals, each coded signal being associated with a predefined activation pattern for at least one actuator designed to transmit information to a user using the skin as an interface to the user,means for transmitting the at least one of the coded signals to a wearable accessory having integrated therein the at least one actuator, andmeans for controlling the at least acuator according to the activation pattern associated with the coded signal transmitted.

The system can comprise means for selecting a coded signal based on a user's manual, visual or audio input or signals originating from biometric or environmental sensors.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

In the following embodiments of the present invention as depicted inFIGS. 1ato6shall be explained in detail.

FIG. 1ashows a user wearing a “smart” jacket101, which smart jacket101has integrated therein at least one actuator106which is able to generate vibrational, heat, pressure or electric pulses to transmit information using the skin of the user as an interface. Note that the smart jacket101is just one example for a wearable computing device, preferentially integrated in a garment or accessory.

102designates a liquid crystal display (LCD) which is incorporated in the upper right front part of the smart jacket101and which is e.g. useable to display emotional states of the user wearing the jacket101e.g. via emoticons or other standardized symbols and signs.

104designates a smart bottom for bottoming up said jacket101, which bottom can also be used for activating or deactivating a function allowing the wearer of the jacket101e.g. to listen to music recorded on an integrated microchip, displaying moving images on the liquid crystal display102and to control the actuator106(f.e. switching on/off the actuator106).

105designates biometric sensors (e.g. body temperature, blood pressure, respiration rate and/or pulse rate) for monitoring these biometric signals originating from the user of the jacket101.

106′ designates processing means for processing e.g. the biometric signals taken from the user of the jacket101and/or measured environmental parameters of the user. As will be explained later on, these processing means106′ are also able to map information (taken from the biometric signals, typed in via an integrated keyboard107a, etc.) to the standardized vocabulary of a “feeling language”.

Furtheron, these processing means106′ control the function of the at least one actuator106by being functionally connected thereto.

106″ designates a battery pack which is integrated in the waste belt of the smart jacket101in order to provide an autonomous energy resource.

107is a control display integrated into the left sleeve of the smart jacket101, whereby107adesignates a manual input device (f.e. keyboard, touchscreen etc) for entering user and control information and107bdesignated a “Jog dial” integrated into the left sleeve of the smart jacket101.

The hardware shown inFIG. 1acan be used to carry out a method as schematically shown inFIG. 1b. As shown inFIG. 1bdifferent input signals, such as signs (emoticons)110a, spoken or written words10b(generally: audio, visual or manually entered information) and biometric signals110cfrom a user are mapped (step112) to a standardized vocabulary (codes) of a “feeling language”. This feeling language is a protocol to transmit any input information110a,110b,110cin a unified (standardized) way and to interpret this information on a receiver side via the operational control pattern of the actuators using the skin of the user as an interface (e.g. via vibrational, heat, pressure, or electrical stimulation).

FIG. 2shows a block diagram for an implementation of the schematic procedure shown inFIG. 1b. Sensors105detect f.e. environmental signals or biometric signals from the user's body in order to detect a current environmental, physical or physichological context of the user and to transmit the context information to a transformation module201, i.e. an expression interpreter202, a mapping unit204and a stimulator unit206.

The transformation module201comprises an expression interpreter202, a mapping unit204as well as a stimulator unit206for controlling actuators106a,106b,106caccording to defined operational pattern.

The expression interpreter202of the transformation module201recognizes both the signals from the sensors105or other visual, audio or manual input such as spoken utterances or written statements110a/110band generates an output signal to the mapping unit204. The expression interpreter202thereby comprises a character scanner202afor recognizing characters, numbers, signals and signs of a entered written information e.g. typed in on the integrated keyboard107of the smart jacket101.

Furthermore, a lexical scanner202bis provided for automatically recognizing feeling language tokens belonging to a “feeling language vocabulary”.

Furtheron, the expression interpreter can comprise a phonetic scanner for automatically recognizing the phonemes (vaults, diphthongs, semi-vaults and consonants) in a spoken utterance110aof the user.

A syntax scanner202canalyses the grammatical structure of a recognized “feeling language statement”, while a semantic scanner202dinterprets feelings expressed by the user by performing a semantic analysis of recognized and synthetically analyzed feeling language statements.

The output signal of the expression interpreter202is supplied with the mapping204which is operationally connected to a memory (e.g. a ramp)204′.

In the memory204′ the vocabulary, i.e. standardized codes is stored. Each standardized code is associated with a predefined activation pattern for at least one actuator designed to transmit information to a user using the skin as an interface to the user. The mapping unit204maps the output signals of expression interpreter202to one or a combination of the codes stored in the memory204′ and transmits205with standardized codes or the combination thereof to a stimulator unit206which can e.g. be the array of actuators in the smart jacket101according toFIG. 1a. The stimulating unit206comprises a sequencer206afor simultaneously rendering multi-channel patterns controlling different actuators106a-106don a single base. A synthesizer unit206bin the tactile stimulator unit206provides a number of different basic wave shapes and possibilities for adapting those wave shapes to achieve different operational patterns.

FIG. 3shows a flow chart of the process carried out by the expression interpreter202. In the step S301a character scan procedure for automatically recognizing characters, numbers and/or symbols e.g. typed in the integrated keyboard107aof the smart jacket101is carried out.

In a step S302a lexical scan procedure is carried out for automatically recognizing feeling language tokens belonging to a feeling language vocabulary.

In a step S303a syntax scan procedure for analyzing the grammatical structure of a “feeling language statement” is carried out. Finally, in a step S304a semantic scan procedure for interpreting the output signal of the syntax scan step S303is carried out by performing a semantic analysis of the recognized and synthetically analyzed feeling language statements.

FIG. 4shows a flow chart of the steps carried out by the mapping unit204.

In a step S401the memory104′ which serves as a repository for meta information (codes) representing feelings, technical sensations perceived and/or physical affection is searched. In a step S402the UPF pattern descriptors600(s.FIG. 6lateron) is enriched or updated with additional information derived from available context information provided by a number of different biometric (105) and/or environmental sensors (105′) which can e.g. be integrated in the clothing101worn by a person.

FIG. 5is a flow chart showing the processing bases carried out in the tactile stimulator unit206. The different tracks respectively carrying operational patterns for the actuators A, B, C, . . . N are multiplexed over channels1. . . n to the actuators by the sequencer206aof the technical stimulator206.

FIG. 6finally shows the format structure of an example for a pattern descriptor in a unified pattern format (UPF) wherein such a descriptor represents one example for a standardized code. Thereby602designates the head of the pattern descriptor600containing the descriptor identifier.604designates the parameter block of the pattern descriptor600containing a number of a required optional parameters needed for a parameterization of the respective pattern (amplitude, frequency, etc.).606designates the meta information contained in the pattern descriptor600needed for the selection of an appropriate pattern by the mapping unit204.

According to the invention a smart piece of clothing or wearable accessory includes at least one integrated electronic devices. Tactile sensations, emotional states and/or states of mind of a person wearing this piece of clothing (in contrast to conventional communication media such as written text messages, emoticons, voice and images) can be detected and analyzed in order to stimulate another person depending on said feelings and/or the respectively perceived sensations by combining vibrational, heat, pressure and/or electric impulses generated by actuators worn on and/or placed near said person's body. Since the proposed system supports the use of the tactile communication channel on the sender and receiver side, it enables users to “touch” each other despite being at distinct locations and combines the benefits of traditional long-distance communication (which includes writing, speaking, sending pictures, signs and symbols, etc.) and traditional communication systems (email, SMS, MMS, EMS, etc.) with a new method for both sending and receiving emotional information. The invention thereby describes the syntax of a “feeling language”, uses fixed or freely defined feeling messages and translates written emoticons (e.g. “:-)”) or colloquial spoken utterances (e.g. “Whaoou!!”) into understandable verbal phrases reflecting emotions of a person (e.g. “It's amazing! I feel good!”). Signals reflecting said information are sent to one or several recipients or posted on web sites, WAP or I-mode sites.

According to one embodiment of the invention, a single- or multi-location vibrational force is generated in a piece of clothing or any other accessory that can possibly be in contact with another piece of clothing (e.g. belts, buttons, tags, brooches, and strings) after having evaluated the wearer's emotional information, state of mind and/or tactile sensations. Also conceivable are accessories such as watches, wrist bracelets, arm bracelets, rings, etc. —so-called digital jewelry. The obtained force pattern may vary in length of time, in the strength of the signal or in frequency, accompanied by heat, pressure and/or electric impulses.

Reception of the aforementioned emotional and tactile information transmitted from a first user wearing a first wearable computing device to a second user wearing a second wearable computing device is enabled by vibration, heat, pressure and/or electric actuators which are integrated within a piece of clothing worn by said second user. Furthermore, body-contactless actuators in the form of small air nozzles directing an air flow to the skin of the person wearing this type of clothing can be provided. The arrangement of all these actuators within a piece of clothing gives said second user the possibility of being close to those parts of the skin of said first user which are sensitive to these types of perceptions.

It should be noted that the invention is also applicable to any device worn close to the body, carried in a pocket close to the body or in contact to the body while being used (e.g. mobile phones).

According to the invention, different ways of “entering” tactile and emotional information are conceivable. Said information can e.g. be entered via “direct input”, which means that each sensation is directly coded with letters and signs on a standard keyboard. For example, <vib r 1 h> means a vibration of short duration with high frequency at right location. More examples are given below. Moreover, a keypad having keys for the most frequently occurring feelings can be used for entering this information. A standard keyboard can also be used to send words or emoticons of a new type of communication language. Besides, biometric signals measured by the first user can be evaluated and transmitted to said second user. This implies that both users (the sender and the recipient) wear a smart piece of clothing equipped with a wearable computing device. In this case, the sender is equipped with a number of biometric sensors (body temperature, blood volume pressure, respiration rate and/or pulse rate sensors, etc.) which are used for monitoring a number of biometric signals indicating the physiological, physical and/or psychological condition, tactile sensations perceived and/or physical expressions of affection reflecting feelings, impressions, emotional states and states of mind of the sender. Emotions can also be described with text on a web, WAP or I-mode page. A hyperlink on this text triggers sensations if a user is equipped with a piece of clothing as described above. The sensations can also be triggered when the user is reading, but this may imply an eye-tracking system since different readers have different reading speeds. In the future, gestures can also be a possible input. For example, a hand gesture initiating a caress can be detected and translated into actuator impulses. Moreover, a voice recognition system can be used to translate spoken utterances of a user wearing said piece of clothing (e.g. “Whaoou!!”) into understandable verbal phrases reflecting his/her feelings and emotions (e.g. “This is exciting!”). Finally, tactile sensations can be triggered as soon as they are received or upon acknowledgment of the recipient only (e.g. by clicking on an attachment or by authorization).

If a tactile communication channel (sensor input and actuator output) is used on the sender as well as on the receiver side, information reflecting emotional states of a first user (the sender) wearing a piece of clothing equipped with sensing means for measuring e.g. biometric signals of the first user and actuating means for generating pressure, vibrational and/ or electric impulses based on perceived sensations of said first user can directly be transmitted to a second user (the receiver) wearing such a piece of clothing by mapping a set of sensor values to an actuator pattern which generates feelings that are equal or at least similar to the sender's feelings.

A typical Internet or SMS emoticon such as e.g. “:-)” is translated into a short impulse or sequence of impulses at different force actuators. A typical emotional expression such as “I like you” is e.g. represented as “a longer, low-frequency force pressure with maximum heat and a tender pressure” at the center position of the actuators:

The translation of the emoticon “:-)” could also be represented as follows:

In case both users (the sender and the receiver) are equipped with mobile phones capable of generating vibration alerts, the emoticon “:-)” in combination with a vibrational message is generated as an SMS (e.g. “:-) & <vib>”), which results in an SMS message display plus i.e. three short vibrations of the vibration alarm of the mobile phone.

A further embodiment of the present invention pertains to a new way of identifying people who are interested in meeting other people having a similar area of interest. Thereby, identification is based on a personal profile which can be downloaded into a processing unit of a mobile device, wearable device or electronic clothing equipped with an integrated radio transceiver and a vibrational force generator. As soon as two of these RF transceivers come into the range of joint operation, the personal profiles of their owners are compared and if interests are matching, a vibration alert indicates a person that another person with same or similar interests is close. The closer the distance, the stronger the activity of the vibrational force actuator. If those people are in visible range (especially if they are part of a crowd of people), they can recognize themselves via visual signals displayed on a display placed on the front and/or back of the top of a person's clothes. These signals can have specific characteristics such as color, brightness and contrast, which can vary by time. Recognition is possible as the visual signals are synchronized with the pattern mentioned above. For example, a T-shirt could flash with the rhythm of the patterns. If the visual communication channel is used, said patterns have to be locally unique.

In the following advantageous arrangements of actuators and their control by a user will be explained in detail.

As shown inFIG. 7a,7b, one or a series of the same or different type of actuators106a-pwhich are embedded in casings706a1-a4that can be located on a user's body or very close to the body to transmit tactile messages. The actuators, f.e their intensity can be controlled in several ways, e.g. by rotating the casings. These casings are wearable and can detachably be mounted to achieve “plug-and-play” usage.

According to a first embodiment of the invention, a number of different actuator elements106a-pgenerating vibro-tactile sensations are integrated into a piece of jewelry such as e.g. a necklace704, a bracelet, rings with a single or variable number of beads, gemstones, etc. These actuator elements are embedded in detachable casings which are pivotable about their suspension axes. Said actuators106a-pgenerate a tactile sensation which becomes the stronger the closer the actuators106a-pare placed to the clavicle of a person wearing said necklace. Thereby, said gemstones can have any shape. Each of said beads, gemstones, etc. or at least some of them includes at least one integrated actuator element106a-p. A spherical bead can have e.g. four actuators all around its inside surface. A cuboidal gemstone can have four actuators, one on each side that can be turned towards the skin of a user wearing said piece of jewelry.

Thereby, a wireless module is attached to the waist of the user or integrated into his/her clothes, bags, mobile phones, etc. The messages to be transformed into feelable sensations are wirelessly transmitted to the necklace, bracelet closure or ring. The closure has also a wireless system to receive the data. The closure directs the data to the actuators106a-pvia the necklace704, bracelet thread or via a wireless transmission channel in case the beads also comprise wireless modules. After having received such a message the actuators106a-pget into action to generate a vibro-tactile sensation associated with said message. The intensity of the message can be modulated by the system on the sender side (during input) or on the receiver side (e.g. by means of mode selection —dynamic mode, soft mode, etc.) but also real time by the recipient of the message. This is achieved by a rotation of the beads about at least one axis to modulate the sensations from strong to very attenuated. One bead thereby serves to switch the system on (closed circuit) and off (open circuit), respectively. By rotating each bead, it is also possible to switch at least one of the embedded actuators106a-pon and off.

In the following, for the sake of clarity, a cuboidal gemstone706acomprising a single or a variety of actuator elements106a-pshall be used as an example to further describe the invention.

FIG. 8shows the net of a cuboidal casing806, used as a digital bead of a necklace704that accommodates a variety of actuators106a-pfor generating tactile stimuli, wherein said casing being pivotable about its suspension axis. In case actuator106a(A), that is attached to side804of the cuboidal casing806, is e.g. a vibro-motor, then the vibrations are felt the strongest if side803is in physical contact with the clavicle702of a person wearing said necklace704. In case the opposite side of the cuboidal casing706a—side804—is in physical contact with said person's clavicle702, the vibrations are a bit attenuated. If anyone of the sides702or705touches the clavicle702, then said vibrations are much softer. This is one way to control the intensity (in addition to other methods such as intensity control by the electrical setup and control means) and/or switching on/off an actuator.

This gives the user the possibility to attenuate the vibrations according to his/her preferences. In some cases a user wants to feel the vibro-tactile sensations in a strong way, and in other cases in a soft way according to the tactile sensitiveness of his/her skin. Some vibrations may be perceived as aggressive to some users. The user thus has a possibility to tune parameters of the vibro-tactile signal, e.g. its frequency, duration and intensity, for his/her personal comfort.

FIGS. 9a-dand11show the intensity levels of different sequential or synchronous stimulation patterns which are used for controlling the actuators106a-pembedded in a single bead706a(cf.FIGS. 9a-d) or several beads706a1-a4of said necklace704(cf.FIG. 11).

The nets of the cuboidal beads are depicted inFIGS. 10a-d.The user has the possibility to choose between four different intensity levels (‘0’, ‘1’, ‘2’, or ‘3’), wherein ‘0’ denotes the weakest and ‘3’ the strongest intensity level.

Inside one bead the actuators106a-p,106e-h,106i-l, or106m-p, respectively, can be of the same type (e.g. vibromotors) or of different types (e.g. vibro-motors, audio speakers, piezo benders, heating elements, peltier elements, micro fans and/or electrostatic discharge modules which are used for generating heat, coolness, pressure, acoustic signals, vibrational and/or electric pulses, respectively). Said actuators can individually be activated, in a sequence, or simultaneously. The beads706a1-a4have to exert an optimal force to the clavicle702of a user wearing said necklace704such that vibro-tactile sensations (and thus vibro-tactile messages) can easily be understood by the recipient of the message. Thereby, said optimal force can be obtained by choosing an optimal weight factor for each bead, which is derived as a function of the casing shape, user size, and actuator type.

Since the vibrations or other tactile stimuli should be feelable together or one at a time on one or several spots of a user's body, each bead706a1-a4has to be isolated from the necklace thread and also from the other beads. (Otherwise, the vibrations, for example, can be felt along the clavicle702and even around the neck. If vibrations can be felt along the necklace704, the recipient can not understand the vibrotactile message.) This means that each bead706a1-a4which comprises at least one actuator element106a-phas to be isolated from other beads that do not comprise any actuator elements. Isolation materials can be any material known today to dampen vibrations and reduce heat transfer from one medium to another. In general, the material of the beads should be chosen such that vibrations or other tactile stimuli are perceived in an optimal way. This e.g. includes polymers, metals, wood, special alloys or composite materials.

According to another aspect of the present invention, the casings of the actuator elements106a-pare designed in such a way that their shapes are optimized for each actuator. For example, a cuboidal casing would have to be optimized e.g. for vibro-motors and/or micro fan. The micro fan e.g. requires that the air can circulate through the casing to allow the fan to produce cool air. Therefore, some openings can be provided. Openings are also necessary for peltier elements and to allow the cooling of the micro motors. Other actuator elements can be used without housings.

A further embodiment of the invention pertains to a piece of clothing, e.g. a T-shirt, which comprises at least one embedded actuator element106awhose casing is integrated into the collar or wrist cuffs of said T-shirt (cf.FIGS. 12a-d). Thereby, said actuator element106generates a tactile sensation that becomes the stronger the closer the respective actuator106ais placed to the clavicle702of a person wearing said piece of clothing. To vary the intensity of the tactile stimuli, the casings706a1-a4of said actuator elements106a-pare detachably mounted and can be pivoted about their suspension axes.

According to a further aspect of the invention as depicted inFIGS. 13a-c, a single gemstone (or bead)1300acan be mounted on a finger ring1300bor an earring. If the user wears two ear rings, he/she has a two-actuator array. Using finger rings, the user can have up to ten actuators. Thereby, said gemstones can be pivoted about at least one axis.

Wearable accessories that comprise embedded actuator elements106a-pas described above can also be worn on the arms (e.g. arm bands integrated into a piece of clothing) and/or in the pockets of a piece of clothing. Actuator casings706a1-a4with embedded actuator elements106a-pcan be located anywhere on a piece of clothing, e.g. in the form of buttons or as decorative stones (button design). The buttons are detachably mounted and can be pivoted about their suspension axes.

As shown inFIG. 14, the invention can also be implemented via a necklace type accessory in which the actuators106as well as the control unit1401can be integrated. Note that it is preferred to house the necessary micro controllers, wireless modules and batteries at the extremities of the open necklace1400, while the at least one actuator106is housed in the port of the necklace1400which rests on the side and the rear side of the neck of a user.

In the embodiment ofFIG. 15the actuators106as well as the electronics necessary to control the actuators is integrated in a belt type accessory device1500. Note that here it is preferred to house the control electronics RF interface, batteries etc. in the buckle portion1501of the belt1500.