- Deafblindness is the condition in which a person has little or no useful hearing and vision. The dysfunction in both senses can occur in different ways and degrees for each person. These differences impose specific barriers and strategies for the rehabilitation of this public , regarding protocols and assistive devices. Most of the technologies and procedures are geared towards a better adaptation of communication. Inspired by sign languages and Braille, we propose the construction of a device capable of transducing specific hand movements of an individual (emitter), into specific vibration patterns on the body of another individual (receiver). Through this device it is possible to adjust different stimulation parameters, based on the modulation of amplitude, frequency of vibration and temporal sequence of pulses, making possible enabling a sensory refinement of wrist and finger's movements, in order to have an improvement associated with the alphabet intended to be represented . This prototype allows communication through crossing -modalities (movement and touch) without the need of visual contact or auditory percepti on, enabling a future application for deaf -blind people . Keywords - deafblindness, indirect communication, control glove, movement, vibrotactile stimulation . I. INTRODUCTION Deafblindness is a rare condition that still affects a significant part of the world ’s population. In Brazil , there is still a lack of public data for this specific condition, and it is estimated that there are more than 40,000 people with this condition in the U.S. alone [1]. Deafblindness affects both hearing and vision with different degrees and modes [1]. The rehabilitation or habilitation of this group of people requires strategies that are completely different from those commonly adopted for "only" blind or "only" deaf individuals . The procedures vary mainly with the origin of the condition, whether congenital or acquired. The protocols and devices , therefore, are developed and directed to improve communication and facilitate greater autonomy and social inclusion of these people [2]. The main protocols and devices include reference objects (by association with daily facts), drawings, body movements, facial expressions, tactile sign language (conversation by signs through touch), manual tactile alphabet (drawing of each letter of the alphabet on the palm of the hand), ta doma (understanding of words by the perception of voice vibration through touch with the lips or vocal cords), lip reading (when there is visual residue), Braille system , and guide -interpreter [2,3]. Thus, the creation of new assistive technologies (ATs) t hat allow the development and rescue of an abstract and freer language, from tools that enable communication at a distance and with reduced attention , will lead to better interaction with the physical and social world. The concept of language can be defin ed as the ability to use a consensual system of symbolic patterns to establish communication between a sender and receiver. These symbolic patterns are represented by physical system s such as sound, image s, or movement. In this context, sign language uses the movement of hands and body to transmit information [4]. The Braille system is another language that allows the recognition of specific spatial patterns from touching a rough surface [5]. Both sign language and Braille are considered forms of communication because they have grammar with structured semantics and syntax [4,5]. Based on these concepts, a device capable of allowing communication at a distance through crossed sensory modalities was created ; arm and hand movements transduced into vibro -tactile patterns. The arm and hand movements of a subject (sender) are recorded by motion sensors and transduced into signals (electrical potentials) sent to a computer . These si gnals are processed for identification and classification of specific patterns from a predefined coding/decoding. Subsequently , this information is transferred to a set of micromotors arranged in a second subject (receiver). For each mechanical movement pa ttern of the transmitter's hand -arm assembly, there is a vibration pattern of several micromotors arranged on the receiver's body. Therefore, each vibration pattern corresponds to a movement pattern of the hand -arm set, performed by the Congresso Brasileiro de Engenharia Biomédica (CBEB 2020) sender, which the r eceiver perceives as vibro -tactile stimuli and interprets according to the pre -established symbolic mapping . This communication protocol is based on the premise that external stimuli, promoted by the environment, and internal stimuli, promoted by differen t neuromuscular and proprioceptive relations, induce activations of specific neural pathways, central and peripheral, and are a neurocognitive representation of the body ( body schema ) [6,8]. These pathways and neural activations are capable of representing different parts of the body as well as promoting an association between the various patterns of vibro -tactile stimulation, perceived in specific points of the body and in specific modes of intensity, frequency , and phase . Therefore, this trans duction can represent a pre -defined code of a sign of body movement language, such as the Brazilian sign language, allowing communication between two (or more) people . Using the device proposed here, it is then possible to build a new communication protoc ol from the integration of different senses (mechanoreceptors of touch, pressure, vibration, and movement [6 -8]) using a cross neurocognitive association with vibro -tactile stimuli of the receiver and arm - hand movements of the sender . This new protocol c reates a mix of codes between sign language and Braille code. The receiver s perceive the movement and position of the emitter's arm through vibro - tactile stimuli promoted by a set of micromotors arranged along the body at specific points. Based on a standa rdized training protocol, the hypothesis is that there will be a long - term associative neurocognitive induction in the receptor agent on this map that links the movements of the emitter's arm, producing a new language [9]. This new device can be extremely important in the rehabilitation of people with both congenital and acquired deafblindness because it allows gradual learning without necessarily referring to an already established language. II. MATERIALS AND METHODS For the development of the project, we built a prototype capable of registering, interpreting , and controlling external effectors that, from the movement records, generate s patterns of vibro -tactile stimuli. The prototype can be divided in to two parts: a recognition module (RM) that identifies gestures; and a tactile feedback module (TFM), capable of producing vibration patterns according to the symbol recognized by the RM. The two modules communicate through radio frequency , mediated by a computer . All materials are listed below : (a) Flexometers; (b) Inertial Measurement Unit MPU6050 (accelerometer and gyroscope ); (c) 433 MHz Radio Frequency ( RF) Transmitter Module; (d) 433 MHz RF Receiver Module; (e) Arduino Pro Mini, 16 MHz and 5V; ( f) LEDs and resistors; ( g) Vibration micromotors; ( h) Darlington Transistors TIP122; ( i) Arduino IDE 1.6.6; ( j) 3D modeling software; ( l) 3D printer and Polylactic acid ( PLA) filament; ( m) Island type printed circuit board ( PCB), solder tin; ( n) Pin bar and coppe r wires; (o) Heat shrink tubing. Fig. 1Schematic of the integrative parts of registration and control of the sensor -glove and effector vest. (A) Illustration of the glove prototype with the microcontroller that registers the finger and arm movements, using an arduino pro-mini, inertial sensors , and flexometer. (B) Detail of the movement recognition module and signal sending system via radio signals. (C) Detail of the circuit made to integrate the arduino pro -mini with the receiver unit, which performs the decoding of gesture signals and transduc es them into vibro -tactile patterns. (D) Detail of the box of the inertial signals reception module and transformation into tactile feedback (TFM ). (E) Panel showing the main equipment associated with and used in the implementation of each of the two hand/arm movement encoding and vibro -tactile pattern encoding devices. Congresso Brasileiro de Engenharia Biomédica (CBEB 2020) Fig. 2 Block diagram of the recognition module (RM) and tactile feedback module (TFM ). (A) Sensors register motion data and send it to the microcontroller where, an algorithm identifies the specific gestures through a class of predefined binary codes. The result of this classification is transmitted by both the radio frequency module and serial port so that there is optimization in data transmission, which can be used by any device that also uses this type of information transfer protocol. (B) TFM block diagram . On receiving the identified symbol, the microcontroller generates the control outputs for each of the six micromotors. (C) Coding -decoding scheme from the finger movements , creating a binary code which is mapped, from “if -else” instances to predefined vibration patterns. A. Recognition Module ( RM) Rubber ducts were fixed on the upper part of each of the fingers of the glove. Each flexometer was properly allocated and fixed in its respective duct, ensuring that the sensors had a relative malle ability and protection from the external environment. The MPU6050 was positioned on the upper part of the glove so that it could register the angles of the user's hand in a three -dimensional space. A circuit was then made using PCB that contained all the necessary connections for the communication of the sensors and the RF transmission module with the Arduino board. This circuit was properly placed inside a box modeled and printed using PLA in a 3D printer. The box was fixed to a forearm protection and conn ected to the glove through cables and connectors to enable the separation between these two parts. The control program, made in Arduino IDE, was downloaded soon after the circuit was completed as shown in Fig. 2-A. B. Tactile Feedback Module ( TFM ) A PCB circuit was built that contained the necessary vias for the connection of the RF receiver module and all the structure s for the independent connection of the six micromotors to the PWM ports of the Arduino Pro -Mini. A box was modeled and printed in a 3D pr inter to store all these components. This box, in turn, was properly allocated in a bag so that the user could attach it to the waist. After completing the circuit tests, the programming of the board was conducted using the Arduino IDE. All micromotors wer e allocated in small supports modeled and printed in a 3D printer. These supports were attached to Velcro belts so that they could be easily attached and removed by the user as shown in Fig. 2-B. C. Module Operation The sensor calibration procedure was performed based on the basal activity associated with the voltage variation corresponding to each sensor of each finger . From this evaluation, a confidence interval was established with 95% significance in relation to the mean. Thus, when the position of one of the fingers changes , it generates a voltage variation associated with each sensor. When this voltage variation exceeds the threshold determined by the confidence interval (dotted line in Fig. 2C1), a binary code is recorded. Different combinations we re previously established from this binary code that mapped specific vibration patterns of the six micro motors arranged on the receiver's body . Thus, the interpretation algorithm was implemented with "if-else" instances in the form: if BINARY CODE 1 then VIBRATION PATTERN A . To extend the proposed codification, the symbols are also conditioned to the position of the arm , doubling the possible message patterns . Thus , the RM evaluates the position of each one of the fingers and the hand movement (with arm mo vement) by Congresso Brasileiro de Engenharia Biomédica (CBEB 2020) reading the data provided by the flexometers and inertial unit. The set of data obtained by the sensors is used as input for an evaluation algorithm . This algorithm translates the gestures and movements of the transmitter into symbols, which are sent through the transmission module of the TFM . The TFM in turn executes the control outputs for each of the vibrating micromotors, producing a vibration pattern corresponding to the pre -defined symbol. Thus, when performing a gesture or movement previously coded in the RM, the TFM user feels the corresponding vibration pattern along the body. It is expected that the subject will be able to make a cognitive association between the gesture/movement and vibration pattern, appropriating the tactile code. III. RESULTS AND DISCUSSION In the current prototype, the RM consists of a glove capable of recognizing the movement of the fingers independently (flexometers) and the wrist (inertial unit). The TFM was built to control up t o six vibratory micromotors, distributed along the body as follows: one on the belly, one on the chest, two on the back (upper and lower) , and one on each arm. Fig . 3 shows illustrations of how the user interacts with the device, exemplifying how deafblind people are able to communicate. Fig. 3 Illustration of use of the prototype for people with deafblindness . (A) Two deafblind subjects wear the device , each perceiving the other's hand movement patterns through vibro -tactile clothing . The tr ansmitter produces movements with one hand and arm, representing a predefined code of signals. These movements are captured by motion sensors and sent to a computer that filters , processes , and classifies the recorded signals according to the pre -defined alphabet (movement -alphabet). After this step , there is a second representation (alphabet -vibration) that encodes the recognized motion as specific vibration patterns generated on the receiv er's body . (B) A single person using the complete glove -sensor system and vibro -tactile clothing . (C) Some of the gestures encoded in the system for testing. In this first prototype, eleven different symbols were coded, which indicated directions, some control states , and some letters of the Portuguese language alphabet coded in letters of the alphabet in Libras . Fig. 3-C shows an illustration of some o f the coded gestures and Table 1 lists all the implemented codes. Table 1. Description of the implemented gestures and codes with their respective vibration patterns. Gesture Meaning Vibration Pattern W (pounds) W Chest motor vibrates A (pounds) A Left arm motor vibrates S (pounds) S Upper back engine vibrates D (pounds) D Right arm motor vibrates Aiming upwards Up Belly and lower back motors vibrate, followed by chest and upper back motors Pointing down Down Chest and upper back motors vibrate, followed by belly and lower back motors Aim left To Left Chest and upper back motor vibrate, followed by the left arm motor Aim right To Right Chest and upper back motor vibrate, followed by the right arm motor Aiming forward Forward Arm motors vibrate, followed by belly and chest motors Aiming back Back Arm motors vibrate, followed by back motors Show palm Stop All engines vibrate With the use of the prototype there will be a lower demand for attention and physical contact for deafblind people, allowing greater freedom and integration in leisure and daily tasks. The prototype brings a new approach in AT for people with deafblindness. This condition , while being extremely limiting , still impos es many barriers scientifically , in the understanding of neurocognitive limits involved in the process of learning, perception , and interaction with the world as well as technological ly in the development of devices that offer a simple, cheap , and effectiv e response to introduce these people to everyday activities. Even today, the great world reference is the activist and writer Helen Keller [1] who gave voice and showed the world the full capacity that deafblind people have. She showed that any device an d protocol that advances towards greater inclusion for this group is of utmost importance and deserves Congresso Brasileiro de Engenharia Biomédica (CBEB 2020) our attention. Using this system, in preliminary tests, it was possible to guide a user to follow a specific trajectory, based only on the tactile feedback provided by the TFM . However, further improvements and tests are still necessary for this prototype . Additionally , we understand that this is a great step towards a group rarely mentioned in the scope of AT development. One limitation is that the current method of decoding the sender's hand/arm movements works well for cases where the number of predefined symbol s is small . However, for a larger set of symbols , the gesture identification process may produce ambiguities, with many false negatives and false positives. To improve the interpretation capability, a robust machine learning based classification method is being implemented [10]. This new method will integrate different signal sources, both signals from current RM sensors and signals from new sensors, such as myoelectric electrodes (EMG), and other inertial units. With these improvements it will be possible to increase the precision, efficiency and accuracy of gesture interpretation. Additionally, the number of vibrating micromotors arranged on the body of the subject receiver is being increased . Their positioning is also being oriented according to the posit ioning of neuromuscular junctions, associated with the sensorial points of afferent pathways [6 -8]. Another implementation is the vibration gradient for intensity and modular frequency in each set of micromotors, according to the semantics associated with the desired information. Currently , the prototype still requires the use of a computer to process and decode the information from hand movements and se nd it to the vibro -tactile device. An embedded version is already being discussed using a raspberry -pi to perform the processing and classification of signals from hand/arm movements , allowing the prototype to be fully portable. An experimental protocol , to test the functional effectiveness of the device, with a number of participants sufficient to achieve statistical robustness, is being planned with an adequate experimental design. Finally, parallel to these modifications, a module capable of interpret ing some vocal patterns and transducing them into vibratory -tactile patterns is also being proposed. The latter is still in the incipient stage but could raise the applicability of the proposal considerably. These modifications will result in a significan t change not only in the quality and efficiency of recording and encoding , but also in the ability to identify and interpret the transmitted symbols . IV. CONCLUSION In this project a functional prototype was developed , capable of establishing communication across two sensorial modalities (movement and touch) , enabling an interaction interface at a distance , inspired in sign language and Braille code, aimed at deafblind people . This device creates conditions for the eme rgence of a new paradigm of communication at a distance, which does not require visual contact or emission of sounds for the transfer of meaningful information between two interlocutors . This device is, therefore , ideal for communication and social integra tion of people with deafblindness. THANKS Project funded by CNPq: 84/2013 MCTIC -SECIS/CNPq - Assistive Technology. Process 458855/2013 -8. CONFLICT OF INTEREST The authors declare that there is no conflict of interest .