Apply Modern Image Recognition Techniques with CUDA Implementation on Autonomous Systems

iv ACKNOWLEDGEMENTS . v TABLE OF CONTENTS . vi LIST OF FIGURES . viii LIST OF TABLES . xi LIST OF ABBREVIATIONS . xii CHAPTER 1: INTRODUCTION . 1 1.1. Computer Vision . 1 1.2. Computer Vision on Autonomous Vehicles . 4 1.3. History of Autonomous Vehicles . 4 1.4. Censors on Autonomous Vehicles . 7 1.5. Other Applications . 13 1.6. GPU Programing . 14 1.7. Novelty and Contribution . 19 1.8. Thesis Outline.20 CHAPTER 2: FEATURE DETECTION . 21 2.1. Feature Detection Overview . 21 2.2. Scale Space Theory . 22 2.3. Integral Image . 24 2.4. SURF Detector . 25 2.5. SURF Feature Detection Result . 28 CHAPTER 3: FEATURE DESCRIPTION.29 3.1. Feature Description Overview . 29 3.2. FREAK Descriptor . 31 3.3. Saccadic Search . 34 M.Sc. Thesis Y. Liu; McMaster University – Mechanical Engineering vii 3.4. FREAK Feature Description Result . 35 CHAPTER 4: GPU PROGRAMMING . 37 4.1. Nvidia GPU Introduction . 37 4.2. CUDA Introduction . 38 4.3. CUDA Scalability Programming Model . 39 4.4. Streaming Multiprocessor . 41 4.5. GPU Performance Optimization . 43 4.6. Parallel Implementation for the Integral Image.47 4.7. Parallel Implementation . 53 CHAPTER 5: EXPERIMENT SETUP AND RESULTS . 56 5.1. Experiment Setup.56 5.2. Experiment Results . 58 CHAPTER 6: CONCLUSION AND FUTURE WORK . 64 REFERENCES . 67 M.Sc. Thesis Y. Liu; McMaster University – Mechanical Engineering viii LIST OF FIGURES FIGURE 1.3.1 THE LEVELS OF AUTOMATION [27] . 6 FIGURE 1.4.1 EVALUATIONS OF LADAR . 8 FIGURE 1.4.2 EVALUATIONS OF ELECTROMAGNET WAVE RADAR . 9 FIGURE 1.4.3 EVALUATIONS OF ULTRASONIC RADAR . 9 FIGURE 1.4.4 THE EVALUATIONS OF PASSIVE VISION SENSOR . 11 FIGURE 1.4.5 COMBINATION OF THE OF FOUR SENSORS . 12 FIGURE 1.4.6 GOOGLE AUTONOMOUS VEHICLE . 12 FIGURE 1.6. 1 FEATURE SIZE OF PROCESSORS OVER TIME [38] . 16 FIGURE 1.6. 2 CLOCK FREQUENCY OF PROCESSORS OVER TIME [38] . 17 FIGURE 1.6. 3 STRUCTURES OF CPU AND GPU [39] . 17 FIGURE 2.2.1 SCALE SPACE PYRAMID REPRESENTATION . 23 FIGURE 2.3.1 INTEGRAL IMAGE CALCULATION EXAMPLE . 25 FIGURE 2.4.1 SURF APPROXIMATE HESSIAN MATRIX [49] . 26 M.Sc. Thesis Y. Liu; McMaster University – Mechanical Engineering ix FIGURE 2.4.2 SURF SCALE SPACE PYRAMID [49]. 27 FIGURE 2.4.3 GRAPHICAL REPRESENTATION OF THE SCALES FOR THREE DIFFERENT OCTAVES [49] . 27 FIGURE 2.5.1 SURF FEATURE DETECTION RESULT . 28 FIGURE 3.2.1 THE HUMAN RETINA [68] . 33 FIGURE 3.2.2 FREAK SAMPLING PATTERN [68] . 33 FIGURE 3.2.3 FREAK SAMPLING PAIRS [68] . 34 FIGURE 3.3.1 ILLUSTRATION OF CASCADE SEARCH [68] . 35 FIGURE 3.4.1 FREAK FEATURE DESCRIPTION AND MATCHING RESULT . 36 FIGURE 4.2.1 TYPICAL CUDA THREAD CONFIGURATION FOR IMAGE PROCESSING . 39 FIGURE 4.3.1 CUDA AUTOMATIC SCALABILITY [39] . 40 FIGURE 4.4.1 CUDA THREAD BLOCK CONFIGURATION [39] . 42 FIGURE 4.5.1 GPU MEMORY ARCHITECTURE [39] . 44 FIGURE 4.5.2 SPATIAL LOCALITY MEMORY ACCESS PATTERN . 45 M.Sc. Thesis Y. Liu; McMaster University – Mechanical Engineering x FIGURE 4.6. 1 PARALLEL UP-SWEEP SCAN IMPLEMENTATION [72] . 49 FIGURE 4.6. 2 PARALLEL DOWN-SWEEP SCAN IMPLEMENTATION [72] . 49 FIGURE 4.6. 3 PARALLEL SCAN INVOLVING MULTIPLE THREAD BLOCKS [72] . 51 FIGURE 4.6. 4 CPU VS. GPU INTEGRAL IMAGE PROCESSING TIME . 52 FIGURE 4.7. 1 ARCHITECTURE OF THE PARALLELIZATION OF SURF DETECTION . 53 FIGURE 4.7. 2 SCHEMATIC OF THE SYSTEM . 55 FIGURE 5.2.1 TRAINING IMAGE AND TARGET IMAGES . 59 FIGURE 5.2.2 FEATURE MATCHING RESULTS . 61 FIGURE 5.2.3 INTEGRAL IMAGE GENERATION TIME COMPARISON BETWEEN CPU AND GPU . 62 FIGURE 5.2.4 SURF FEATURE POINT DETECTION TIME COMPARISON BETWEEN CPU AND CUDA . 62 FIGURE 5.2.5 SURF AND FREAK EXECUTION TIMES COMPARISON BETWEEN CPU AND CUDA .