Motion stereo for mobile robot applications based on novel geometric measures and image features

3D computer vision is still an incompletely solved problem. Despite of many emerging problems are theoretically solved and described, practical applications are far beyond the current state of science. One reason is the fact that combining well known algorithms often lead to unexpected problems. The optimal cooperation of emerging subtasks still has to be developed further. At the beginning of Ph.D. studies, my primary goal was to understand the whole 2D→3D reconstruction process, starting with 2D image preprocessing and ending with 3D data handling. Due to this aim, I was working with many emerging subtasks. A few of them incorporated undiscovered possibilities to be enhanced. This thesis describes the novel modifications discovered. Computer vision requires geometric approach regarding the problem, but needs algebraic approach regarding the solutions. Algebraic approach results faster operation on computers, but is sensitive to certain geometric situations called singular cases. Most of the novel methods described in this thesis use geometric approach in the solution. It was preferred to handle singular cases by the same formulation used during normal operation. The investigation of the affine Lucas-Kanade (LK) method, which is used to detect small misalignments of similar images, focused my attention to its singular cases. Further investigation of these led to the development of the Self-Affine Feature Transform (SAFT), which describes the sensitivity of images to small affine transformations. This topic turned to be very fruitful, the achieved results are described in two thesis groups. In the first thesis group, information extraction by SAFT is analyzed. The usage of SAFT for image registration is shown in the second thesis group. The theory and relations of SAFT show similarities of problems, which were handled separately before, including: Harris Corner detector, LK detector, detection of conics and even segmentation of 3D volumetric data. The third thesis group presents results which were required to the reliable real-time operation of the visual navigation of the indoor autonomous flying quadrotor helicopter developed in our laboratory. Enhancements of often emerging problems using the geometric approach are given: A geometric solution is presented to the inverse Rodrigues problem which is used to describe 3D rotations. I introduced a novel method to factorize the essential matrix used in stereo image processing. The development of computer vision is strongly connected to the development of the computational capabilities of the computers used. The runtime of developed application is crucial during debug, tests and parameter adjustment. The prerequisite of up to date research in computer vision is to use proper development and computational tools. The third thesis group also contains results relating to real time computer vision utilizing the power of programmable Graphical Processing Units (GPUs) and efficient implementation of high-speed linear algebra operations. Results presented in this thesis enable smooth and easier implementation of real time 3D vision systems when compared to other solutions. Corresponding results were utilized in the control system of an aerial indoor mobile robot developed at the Department of Control Engineering and Information Technology of the Budapest University of Technology and Economics (BME), Hungary.