Doğadan esinlenen koku ve ikili görüye dayalı gerçek zamanlı bir gömülü sistem tasarımı
A bio-inspired real time embedded system design based on olfaction and stereo vision
Yazar: SELMAN ERGÜNAY
Danışman: DOÇ. DR. MÜŞTAK ERHAN YALÇIN
İstanbul Teknik Üniversitesi / Fen Bilimleri Enstitüsü / Elektronik ve Haberleşme Mühendisliği Ana Bilim Dalı / Elektronik Mühendisliği Bilim Dalı
Derece: Yüksek Lisans, 2013
Konu: Elektrik ve Elektronik Mühendisliği = Electrical and Electronics Engineering
Dizin: Sayısal elektronik = Digital electronic

Özet:
Günümüzde mühendislik problemlerine yönelik çalışmalarda doğadan esinlenen çözümlere eğilim sıklıkla karşımıza çıkmaktadır. Bu eğilim doğadaki canlıların merkezi ve çevresel sinir sistemini yaşamlarındaki karmaşık olaylara karşı yüksek başarımlı olarak kullanabilmelerinden kaynaklanır. Canlıların karar mekanizmalarındaki bu başarısı günümüz teknolojilerinin hem hız hem de doğruluk bazında yakalayamadığı boyutlarda olabilir. Doğadaki bu başarılı örnekler sebebiyle canlıları incelemek ve taklit etmek daha iyi mühendislik sistemlerinin geliştirilmesine katkı sağlayabilir. 

Bu fikirden yola çıkarak yapılan tez çalışmasında ilk olarak koku bilgisini hızlı algılama ve sınıflandırma özelliğine sahip canlıların veriyi değerlendirmede kullandıkları biyolojik sinir ağı yapısından esinlenen Hücresel Yapay Sinir Ağı tabanlı bir sistem tasarımı yapılması, bunun yanı sıra canlıların koku kaynağını bulmada gösterdiği davranışların incelenerek doğadan esinlenen yönlendirme algoritmaları üzerine tartışılması amaçlanmıştır.
Bunun için ilk etapta Hücresel Yapay Sinir Ağı ve Hibrit İşlemci Topluluğu konularına ilişkin teorik bilgi verilmiş, daha sonra canlıların koku karşısında gösterdikleri davranışlar incelenerek mühendislik uygulamalarında bu davranışlardan esinlenen yöntemlerden bahsedilmiştir. Ardından bu kapsamda gerçeklenen algoritmalar ele alınmıştır.

Tez kapsamında incelenen diğer konu insandaki \textit{stereopsis} olayından esinlen ikili görü yöntemidir. Stereopsis iki gözden gelen görüntülerin değerlendirilerek görülen sahneye ait uzaklık ve derinlik bilgisinin algılanması olayıdır. Buna göre iki gözden alınan dış dünyaya dair görüntüler arasındaki farklılıktan derinlik bilgisi çıkarılabilir. 
Farklı iki kamera görüntüsü ya da hareket halindeki tek kamera görütüsü ile görüntülenen sahnenin üç boyut bilgisinin çıkarılması bu kapsamda mümkündür. Bu projede ikili kamera ile derinlik bilgisinin gerçek zamanlı olarak elde edilmesi ele alınmıştır.

Son olarak doğadan esinlenen koku algoritmaları ile ikili görü algoritmalarının birlikte gerçeklenebileceği, farklı donanım ve yazılım uygulamalarına elverişli esnek bir gömülü sistem geliştirme ortamı sunmak bu tezin amacı dahilindedir. Bu kapsamda FPGA tabanlı bir sistem önerilmiş ve sistemi oluşturan algoritmaların yazılım-donanım gerçeklemeleri incelenmiştir.

Abstract:
The tendency to bio-inspired solutions to current engineering problems is frequently encountered in the literature. This is basically issued from the fact that the living beings are able to use with high performance their central and peripheral neural system in case of complex events they face to. This performance can be far from the capabilities of today's technology in terms of the achieved speed and accuracy.

It can be said that there are two basic factors providing this capability to the living beings. The first factor is that the living beings strengthen their information baseline related to the environment by fusing the data coming from multiple body sensors. An example of this is the target assessment of animals by using their visual sense and olfaction together. In fact, the real-time evaluation of two or more senses in living beings is noteworthy and is crucial to be applied in engineering problems. The second factor is the parallel processing of the data coming to sensors thanks to the neural network structure. This property leads to the ability of giving reaction/decision very quickly even though the neurons are working at very low frequency. The current technology can reach quite high performance if this parallel processing procedure in the living beings can be properly implemented.

The development  of algorithms integrating such properties of living beings is important in terms of providing real-time and low-cost solutions to robotic , bio-medical, etc. applications. Accordingly, this project focus on a system design which produces decisions by fusing and processing in parallel visual senses and scents. The proposed system presents a solution to the problem of target assessment of robotic applications by processing in parallel the data coming from stereo or binocular vision and olfaction. Before the design, the odor-detection based on the behavior of olfactory neurons to odors and the depth-detection 3D-grasp of the environment based on stereo vision are examined in detail and the algorithm is developed accordingly. The implementation of the algorithm both in software and hardware platforms is also a part of the project scope.

Although new methods and technologies enables more accurate and faster sensors in odor perception, artificial olfaction systems are far beyond to biological systems. 
Accomplishment behind the biological systems lie not only performance of biological sensors, but also pre-processing the sensed signal with unique neural networks and behaviour caused by instinct of the living being. For example it is seen that insects use antennal lobe, and mammals use olfactive bulb as pre-processor for odor data.
These biological neural network structures enables feature extraction, additionally  simplifies and speeds up identification which will be performed by central processing unit.

Speed and accuracy performance of biological odor systems is notable and many engineering systems are inspired from them. In the odor part of this thesis, firstly biological neural structures for odor processing is given. Cellular neural network is one of the techniques that is inspired from biological neural structures, and which combines cellular automata and neural networks. But its network structure connects identical cells unlike in nature. For instance, it is known that neural network structure in retina contains two different kind of neurons which acts inhibitor and excitator. In this thesis Wilson-Cowan neural network model which models retina-like networks examined and a new network model that benefits superior features of CNN and Wilson-Cowan model is introduced. Additionally, an application of this network for odor processing is realized and hardware-software design of implementation on FPGA is given.

Another consideration about bio-inspired odor processing is behavior of animals which interact with odor. In nature, because odor data is very important for surviving for most animals, their ways to find the odor source is interesting and can include some inspiring methods for robot navigation. In this thesis, olfaction based behaviours of some living beings are examined, and their algorithmic expressions are given. Software development of these algorithms are considered within embedded system design methodology.

In addition to the examination and the evaluation of the olfaction system, the evaluation of the information coming from the visual sense is also another issue that this thesis focused on. The 3D-grasp of the environment can be provided by the stereo vision, which is inspired by the event of stereopsis in the human beings . In fact, when two views from different positions of the same scene is available, the third dimension, i.e., the depth can be estimated by the epipolar geometry. Accordingly, one can reconstruct the 3D correspondence of a 2D point by considering its coordinates in each of the scene images. 
There are, however,  two basic issues of the stereo vision systems that need to be solved in order to use epipolar geometry in finding the depth of the scene. The first problem is the correspondence problem and the other one is the reconstruction problem. The correspondence problem is defined as the problem of the finding the matched points in the stereo images. This problem is treated in two different aspects: what points should be matched to each other and which similarity measure should be used to be able to match the points accurately. In the literature, the first aspect leads to different algorithms as they either handle all the pixels or some subset of the all pixels on to find their match. 
The methods trying to match all the pixels in the image is called as dense methods whereas the others that consider some "interest points" to match is named as sparse methods.
The choice of the methods is dependent of the application where the number of the matched points may influence the performance. The second aspect of the correspondence problem is to determine a robust similarity measure that would find the best matching pixels in the dual images. The determination of the similarity measure is critical since the precision in the matched points in the stereo images has a significant influence on the accuracy of 3D reconstruction of the scene. As for the second problem of the stereo vision, i.e. reconstruction problem, one seeks for a solution based on the epipolar geometry which is determined by the intrinsic and extrinsic camera parameters. Accordingly, there appear three different cases for each of which the reconstruction problem is considered differently: the case where the intrinsic and extrinsic parameters are known, the case where only intrinsic parameters are known, the case where none of the parameters are known. In the first case, a simple triangulation method properly solves the problem whereas the two latter cases needs to more complex approaches in order to find the correct solution. In this thesis, the dense approach is preferred for the correspondence problem because all 3D reconstruction is sought for all the pixels of the scene. Accordingly, the block template matching is used. The sum of absolute differences (SAD) is chosen as a similarity measure. Assuming all the extrinsic and extrinsic camera parameters are known, the third dimension of the scene objects are exracted via the triangulation method. Those steps are implemented in MATLAB and the algorithm is tested via simulations.


To sum up, the main goal of this thesis to build up a bio-inspired real-time embedded system which is based on olfaction and stereo vision. To reach this goal, some bio-inspired structures and algorithms are examined, and a new network model for odor processing is introduced. At the implementation side of this work it is aimed to obtain a flexible hardware-software development platform which enables to design FPGA based embedded systems which use the Hybrid Processor Population for odor processing and depth map estimation hardware for visual perception of environment.