a smart device that translates sign language while being worn on the wrist could bridge the communications gap between the deaf and those who dont know sign language says a texas a&m university biomedical engineering researcher who is developing the technology the wearable technology combines motion sensors and the measurement of electrical activity generated by muscles to interpret hand gestures says roozbeh jafari associate professor in the universitys department of biomedical engineering and researcher at the texas a&m engineering experiment stations (tees) center for remote health technologies and systems although the device is still in its prototype stage it can already recognize 40 american sign language words with nearly 96 percent accuracy notes jafari who presented his research at the institute of electrical and electronics engineers (ieee) 12th annual body sensor networks conference this past june the technology was among the top award winners in the texas instruments innovation challenge this past summer the technology developed in collaboration with texas instruments represents a growing interest in the development of high-tech sign language recognition systems (slrs) but unlike other recent initiatives jafaris system foregoes the use of a camera to capture gestures video-based recognition he says can suffer performance issues in poor lighting conditions and the videos or images captured may be considered invasive to the users privacy whats more because these systems require a user to gesture in front of a camera they have limited wearability – and wearability for jafari is key "wearables provide a very interesting opportunity in the sense of their tight coupling with the human body jafari says because they are attached to our body they know quite a bit about us throughout the day and they can provide us with valuable feedback at the right times with this in mind we wanted to develop a technology in the form factor of a watch in order to capture the intricacies of american sign language jafaris system makes use of two distinct sensors the first is an inertial sensor that responds to motion consisting of an accelerometer and gyroscope the sensor measures the accelerations and angular velocities of the hand and arm jafari notes this sensor plays a major role in discriminating different signs by capturing the users hand orientations and hand and arm movements during a gesture however a motion sensor alone wasnt enough jafari explains certain signs in american sign language are similar in terms of the gestures required to convey the word with these gestures the overall movement of the hand may be the same for two different signs but the movement of individual fingers may be different for example the respective gestures for please and sorry and for name and work are similar in hand motion to discriminate between these types of hand gestures jafaris system makes use of another type of sensor that measures muscle activity known as an electromyographic sensor (semg) this sensor non-invasively measures the electrical potential of muscle activities jafari explains it is used to distinguish various hand and finger movements based on different muscle activities essentially its good at measuring finger movements and the muscle activity patterns for the hand and arm working in tandem with the motion sensor to provide a more accurate interpretation of the gesture being signed he says these two technologies are complementary to each other and the fusion of these two systems will enhance the recognition accuracy for different signs making it easier to recognize a large vocabulary of signs jafari says in jafaris system both inertial sensors and electromyographic sensors are placed on the right wrist of the user where they detect gestures and send information via bluetooth to an external laptop that performs complex algorithms to interpret the sign and display the correct english word for the gesture as jafari continues to develop the technology he says his team will look to incorporate all of these functions into one wearable device by combining the hardware and reducing the overall size of the required electronics he envisions the device collecting the data produced from a gesture interpreting it and then sending the corresponding english word to another persons smart device so that he or she can understand what is being signed simply by reading the screen of their own device in addition he is working to increase the number of signs recognized by the system and expanding the system to both hands the combination of muscle activation detection with motion sensors is a new and exciting way of understanding human intent with other applications in addition to enhanced slr systems such as home device activations using context-aware wearables jafari says jafari is associate professor in texas a&ms department of biomedical engineering associate professor in the department of computer science and engineering and the department of electrical and computer engineering and researcher at tees center for remote health technologies and systems his research focuses on wearable computer design and signal processing he is director of the embedded signal processing laboratory photo: a prototype of jafaris sign language recognition technology that he aims to scale down to the size of a watch about the center for remote health technologies and systems (crhts) the center for remote health technologies and systems is designing and developing advanced health technologies and systems to enable healthy living through health monitoring and disease diagnosis management and prevention the centers mission is to identify and overcome the unmet needs of patients and health care providers through the development of breakthrough remote health care devices biosignal mapping algorithms remote health analytics and information systems that will improve access enhance quality and reduce the cost of health care