The prototype multimodal user station has been implemented. A gimbaled camera mounted on the workstation follows the face, illuminates the eye with infrared, and computes the angle between corneal reflection and pupil centroid to produce a screen cursor at the point of attention. The user simply looks at the position where the cursor is desired. Positioning accuracy is approximately 1 degree and the visual feedback allows vernier adjustment. A microphone array, mounted on the workstation, is fix-focused at the user position and captures a good quality speech signal even in the presence of ambient acoustic noise.
Pneumatic thrusters apply computed force to the finger tips or sense muscle force.
LED and photodetector pairs are located coaxially within the thrusters and determine grasp position. Absolute hand position is provided from a Polhemus coil on the wrist. Using this tactile interface one can grasp visual objects in the workspace and move and position them. The interface is serves as a front-end for the following application beans. We have developed a powerfull multi-document editor that can be extended to build customized document editors.