Query: Evaluation of man-machine interface

title: Heuristic evaluation of user interfaces
abstract: Heuristic evaluation is an informal method of usability analysis where a number of evaluators are presented with an interface design and asked to comment on it. Four experiments showed that individual evaluators were mostly quite bad at doing such heuristic evaluations and that they only found between 20 and 51% of the usability problems in the interfaces they evaluated. On the other hand, we could aggregate the evaluations from several evaluators to a single evaluation and such aggregates do rather well, even when they consist of only three to five people.

title: Bleu: a Method for Automatic Evaluation of Machine Translation
abstract: Human evaluations of machine translation are extensive but expensive. Human evaluations can take months to finish and involve human labor that can not be reused. We propose a method of automatic machine translation evaluation that is quick, inexpensive, and language independent, that correlates highly with human evaluation, and that has little marginal cost per run. We present this method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations.

title: Somatosensory computation for man machine interface from motion capture data and musculoskeletal human model
abstract: We discuss the computation of somatosensory information from motion capture data. The efficient computational algorithms previously developed by the authors for multibody systems, such as humanoid robots, are applied to a musculoskeletal model of the human body. The somatosensory information includes tension, length, and velocity of the muscles, tension of the tendons and ligaments, pressure of the cartilages, and stress of the bones. The inverse dynamics of the musculoskeletal human model is formulated as an optimization problem subject to equality and inequality conditions. We analyzed the solutions obtained by linear and quadratic programming methods, and showed that linear programming has better performance. The technological development aims to define a higher dimensional man machine interface and to open the door to the cognitive level communication of humans and machines.

title: Development of man machine interface utilizing myoelectricity
abstract: This paper aims to develop a man machine interface utilizing myoelectricity as input signals. The accurate relationship between an operator's intention and myoelectricity arising as the result of the intension is required to realize such an interface. In our study, paying attention to the wrist motion, an NARX model from myoelectricity as input to the wrist angle as output is derived, and is utilized to estimate the wrist angle from myoelectricity measured by EMG sensors. The Lebesgue type identification method for NARX models is used to obtain the model. Control of unstable systems is a difficult task in general, hence, the concept of safe manual control is combined with our interface. The effectiveness of the proposed man machine interface is verified by experiments in which the interface is used for control of a Furuta pendulum.

title: ELIZA a computer program for the study of natural language communication between man and machine
abstract: ELIZA is a program operating within the MAC time sharing system of MIT which makes certain kinds of natural language conversation between man and computer possible. Input sentences are analyzed on the basis of decomposition rules which are triggered by key words appearing in the input text. Responses are generated by reassembly rules associated with selected decomposition rules. The fundamental technical problems with which ELIZA is concerned are: (1) the identification of key words, (2) the discovery of minimal context, (3) the choice of appropriate transformations, (4) generation of responses in the absence of key words, and (5) the provision of an editing capability for ELIZA "scripts" A discussion of some psychological issues relevant to the ELIZA approach as well as of future developments concludes the paper.

title: Learning to Control a Brain Machine Interface for Reaching and Grasping by Primates
abstract: Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed loop brain machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e. hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach and grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

title: The state of the art in automating usability evaluation of user interfaces
abstract: Usability evaluation is an increasingly important part of the user interface design process. However, usability evaluation can be expensive in terms of time and human resources, and automation is therefore a promising way to augment existing approaches. This article presents an extensive survey of usability evaluation methods, organized according to a new taxonomy that emphasizes the role of automation. The survey analyzes existing techniques, identifies which aspects of usability evaluation automation are likely to be of use in future research, and suggests new ways to expand existing approaches to better support usability evaluation.

title: Design of man machine interfaces in process control
abstract: Computerized CRT displays are rapidly being introduced into man machine interfaces for process control and supervision. The flexibility of these devices can promote the incorporation of human factors in interface design. The paper gives a survey of design criteria and guidelines for: Allocating system functions between "man" and "machine" coping with operator skills; evaluating CRT displays and choosing controls and dialogue structures for these type of displays. Also, attention is paid to experimental comparison of design alternatives, and to the influences of the human organization, high mental stress and job satisfaction on interface design.

title: Modular Design of Man Machine Interfaces with Larissa
abstract: The man machine interface of a small electronic device like a wristwatch is a crucial component, as more and more functions have to be controlled using a small set of buttons. We propose to use Argos, an automaton based language for reactive systems, and Larissa, its aspect oriented extension, to show that several interfaces can be obtained from the same set of basic components, assembled in various ways. This is the basis of a quite general component based development method for man machine interfaces.

title: Automatic evaluation of machine translation quality using n gram co occurrence statistics
abstract: Evaluation is recognized as an extremely helpful forcing function in Human Language Technology R&D. Unfortunately, evaluation has not been a very powerful tool in machine translation (MT) research because it requires human judgments and is thus expensive and time consuming and not easily factored into the MT research agenda. However, at the July 2001 TIDES PI meeting in Philadelphia, IBM described an automatic MT evaluation technique that can provide immediate feedback and guidance in MT research. Their idea, which they call an "evaluation understudy" compares MT output with expert reference translations in terms of the statistics of short sequences of words (word N grams) The more of these N grams that a translation shares with the reference translations, the better the translation is judged to be. The idea is elegant in its simplicity. But far more important, IBM showed a strong correlation between these automatically generated scores and human judgments of translation quality. As a result, DARPA commissioned NIST to develop an MT evaluation facility based on the IBM work. This utility is now available from NIST and serves as the primary evaluation measure for TIDES MT research.