List of Abbreviations - Ultra fast cnn based Hardware Computing Platform Concepts for adas visual Sensors and Evolutionary...

List of Abbreviations
ACC
………………………..
Adaptive Cruise Control
ADAS
………………………..
Advanced Driver Assistance System
ANN
………………………..
Artificial Neural Network
CNN
………………………..
Cellular Neural Networks
CPU
………………………..
Central Processing Unit
DDA
………………………..
Digital Differential Analyzer
DDR
………………………..
Double Data Rate
–
memory
DOF
………………………..
Degree of Freedom
DSP
………………………..
Digital Signal Processing
DT
………………………..
Discrete Time
FPGA
………………………..
Field-programmable Gate Array
GA
………………………..
Genetic Algorithm
GPU
………………………..
Graphics Processing Unit
HDL
………………………..
Hardware Description Language
ISE
………………………..
Integrated Software Environment
LDW
………………………..
Lane Departure Warning
MLP
………………………..
Multi-layer Perceptron Neural Network
ODE
………………………..
Ordinary Differential Equations
OpenCL
………………………
Open Computing Language
OpenCV …………………….…
Open Source Computer Vision
PDE
………………………..
Partial Differential Equation
PLB
………………………..
Processor Local Bus
UM
………………………..
Universal Machine
2
ABSTRACT
Vehicle driving and traveling with car are part of our daily life. More than 80% of all
personal travels are done by car and only 20% are done by public transportation. Alone in
Europe every year we have more than 3 million additional private cars on roads and
highways. Due to the congestion and capacity limitations of both highways and roads every
year we have more than 7500 kilometers of blocked roads in Europe. This congestion and
traffic has a negative direct impact on the economy and on total social costs. The
estimation of traffic congestion and traffic safety on social costs for European people is
more than 130 billion euro per year; see Ref. [1-3] .
Monitoring and controlling traffic and
improving road safety can reduce this cost; but still every year 40,000 people die because
of car accidents in Europe [1], [4-6] . There are two solutions for overcoming the critical
issue of road safety: improving the driver safety education programs and improving the
vehicle safety using advance technology like ADAS (Advanced Driver Assistance System).
One of the main factors in car accidents and traffic safety is the human factor. If the driver
is tired or asleep the probability of accident will dramatically increase. A convenient way to
avoid these types of accidents is using an assistance system.
This thesis answers the following eight research questions which are related to the
potential performance improvement of ADAS technology with respect to the involved real-
time image processing:
Research question 1: What are the hard requirements of ADAS concerning real-
time image processing and design flexibility? How far do traditional approaches fail
to satisfy these requirements?
Research question 2: What are the major limitations of traditional high
performance computing ap
proaches if used to ensure “real
-
time” image
processing?
Research question 3: What is the huge potential of neurocomputing involving
either traditional neural networks (NN) or cellular neural networks (CNN) for high-
speed and flexible image processing for ADAS? Are there any limitations and how
can these eventually be addressed?
Research question 4: What are the major template calculation schemes of relevance
for CNN based image processing? How can these calculations be performed in a
real-time high performance computing context?
3
Research question 5: How far can the advantages of "analog computing" be
used/gained through an emulation of analog computing on digital hardware
platforms like FPGA (for the benefit of an ultrafast image processing)?
Research question 6: How far can an efficient implementation of CNN on FPGA and
GPU be designed and implemented?
Research question 7: How far can CNN be used/involved in an evolutionary
computing/control context example (for illustration)?
To cover the research question-1, we have conducted a survey concerning different ADAS
concepts. High definition cameras are playing a very important role in ADAS concept and
almost every ADAS concept includes one camera or some form of visual radar. We could
formulate the overall requirements that ADAS systems set to the image processing based
sensor functionality. And to finish we have shown as an example that there are many
common modules for different ADAS systems.
Concerning research question-2, we have shown the limitations of traditional/sequential
computing concepts for processing high quality images in the ADAS context and did
propose a parallel processing model based on CNN.
Concerning research question-3 we have shown the advantages of neuro-computing
especially of CNN as a high performance computing system in terms of flexibility in design
and robustness.
Research question-4 is considering different methods for CNN template calculation. We did
pass a review of the related state-of-the-art before proposing a genetic algorithm based
approach for the calculation of complex templates. This concept can be implemented in
hardware, for example of FPGA along with the CNN processor system. This should ensure a
performance in real-time.
Concerning research question-5 we have demonstrated that the advantage of analog
computing can be used in a real time solution for solving complex dynamic systems.
Further we have shown the implementation an “analog computing” emulation on a digital
platform (FPGA); the implementation was based on the
Digital Differential Analyzer
(DDA)
method which has shown to be thousand time faster than a CPU.
4
The research question 6 does concern the implementation of CNN in a discrete-time
version on both FPGA and GPU. We have shown the advantages and drawbacks of each of
the implementations. Overall we could realize a CNN implementation on both platforms
and the performance was very good.
Concerning the last research question 7, we have shown the efficient use of CNN and a
genetic algorithm for controlling a legged-robot in the form of a neuro-evolutionary
technique. The results did clearly demonstrate the effectiveness of the approach.
The evaluation of image sequences with the purpose of extracting useful information
(about the environment, vehicle situation and traffic) is the main and challenging issue in
ADAS visual sensors. This process does however involve a huge computational effort.
Therefore, to have a real time processing platform we need appropriate hardware and
software architectures. Today, there are many platform options for machine vision: DSP,
FPGA and GPU. The requirements related to image quality, image size and frame rate per
second for a processing in hardware are increasing. Overall we do face a tradeoff between
processing time, power consumption on one hand, and video quality and precision on the
other hand. This challenge does motivate research related to new architectures and
software algorithms for video and signal processing. Designing and implementing a
“single
task” image processing functionality in hardware in not a big issue. However, the “multi
tasking” case is much challenging. Having a good model for image processing will reduce
hardware resources. A CNN model has this potential as by dynamically changing the
related templates values we can change the functionality of the system and thereby
without any further hardware or software explicit reconfiguration. Taking advantage of the
inherent parallel processing nature of CNN processors a strong integration of both
hardware and software can be ensured. This thesis does address two main challenging
issues in ADAS technology: a) a universal model and architecture for a real time visual
processing; and b) the implementation of a prototype system on both GPU and FPGA.
|