A compact representation of the continuous video of the scene is generated in the form of a mosaic, within which the system detects text and signs. We will develop an object detection and recognition system that uses multi-scale features that are invariant to changes in viewing angle and illumination. Text is detected using the same techniques and is recognized using a commercial OCR system. We will also examine the OCR's intermediate representations, and a word dictionary for tolerance to character recognition errors. In both text and signs information from the gyroscopically referenced head mounted camera is used to filter the potential set of match candidates for improved efficiency.
Research is conducted in three phases roughly corresponding to a year each.
Both design and evaluation are conducted in conjunction with Lighthouse International, which specializes in rehabilitation training for the visually impaired. In the first phase individual components, which include camera stabilization, mosaics, text and sign recognition are developed. In the second phase, a head mounted unit, attitude inference, a wearable computer, a camera, speech synthesizer and user interface model are developed. The last phase consists of a series of user and system evaluations and refinements. Co-author of the grant proposal, Allen is Chandu's advisor whose interests in the areas of compact image representations and object recognition form key elements of this proposal.