Practical image and video processing using MATLAB 4.2.5 Exploring the Contents of an Image 69 4.2.6 Writing the Resulting Image onto a File 70 4.3 Tutorial 4.1: MATLAB Image Processing Toolbox a Guided Tour 72 4.4 Tutorial 4.2: Basic Image Manipulation 74 4.5 Problems 80 5 IMAGE SENSING AND ACQUISITION 83 5.1 Introduction 83 5.2 Light Color and Electromagnetic Spectrum 84 5.2.1 Light and Electromagnetic Spectrum 84 5.2.2 Types of Images 85 5.2.3 Light and Color Perception 86 5.2.4 Color Encoding and Representation 87 5.3 Image Acquisition 89 5.3.1 Image Sensors 89 5.3.2 Camera Optics 92 5.4 Image Digitization 93 5.4.1 Sampling 95 5.4.2 Quantization 96 5.4.3 Spatial and Gray Level Resolution 97 5.5 Problems 101 6 ARITHMETIC AND LOGIC OPERATIONS 103 6.1 Arithmetic Operations: Fundamentals and Applications 103 6.1.1 Addition 104 6.1.2 Subtraction 106 6.1.3 Multiplication and Division 109 6.1.4 Combining Several Arithmetic Operations 110 6.2 Logic Operations: Fundamentals and Applications 111 6.3 Tutorial 6.1: Arithmetic Operations 113 6.4 Tutorial 6.2: Logic Operations and Region of Interest Processing 118 6.5 Problems 122 7 GEOMETRIC OPERATIONS 125 7.1 Introduction 125 7.2 Mapping and Affine Transformations 127 7.3 Interpolation Methods 130 7.3.1 The Need for Interpolation 130 7.3.2 A Simple Approach to Interpolation 131 7.3.3 Zero Order (Nearest Neighbor) Interpolation 132 7.3.4 First Order (Bilinear) Interpolation 132 7.3.5 Higher Order Interpolations 132 7.4 Geometric Operations Using MATLAB 132 7.4.1 Zooming Shrinking and Resizing 133 7.4.2 Translation 134 7.4.3 Rotation 134 7.4.4 Cropping 134 7.4.5 Flipping 134 7.5 Other Geometric Operations and Applications 134 7.5.1 Warping 134 7.5.2 Nonlinear Image Transformations 135 7.5.3 Morphing 137 7.5.4 Seam Carving 137 7.5.5 Image Registration 137 16.2.3 The HSV Color Model 398 16.2.4 The YIQ (NTSC) Color Model 401 16.2.5 The YCbCr Color Model 401 16.3 Representation of Color Images in MATLAB 401 16.3.1 RGB Images 402 16.3.2 Indexed Images 403 16.4 Pseudocolor Image Processing 406 16.4.1 Intensity Slicing 406 16.4.2 Gray Level to Color Transformations 407 16.4.3 Pseudocoloring in the Frequency Domain 408 16.5 Full color Image Processing 409 16.5.1 Color Transformations 410 16.5.2 Histogram Processing 412 16.5.3 Color Image Smoothing and Sharpening 412 16.5.4 Color Noise Reduction 414 16.5.5 Color Based Image Segmentation 414 16.5.6 Color Edge Detection 417 16.6 Tutorial 16.1: Pseudocolor Image Processing 419 16.7 Tutorial 16.2: Full color Image Processing 420 16.8 Problems 425 17 IMAGE COMPRESSION AND CODING 427 17.1 Introduction 427 17.2 Basic Concepts 428 17.2.1 Redundancy 428 17.2.2 Image Encoding and Decoding Model 431 17.3 Lossless and Lossy Compression Techniques 432 17.3.1 Lossless Compression Techniques 432 17.3.2 Lossy Compression Techniques 433 17.4 Image Compression Standards 435 17.4.1 Binary Image Compression Standards 435 17.4.2 Continuous Tone Still Image Compression Standards 435 17.4.3 JPEG 436 17.4.4 JPEG 2000 437 17.4.5 JPEG LS 437 17.5 Image Quality Measures 438 17.5.1 Subjective Quality Measurement 438 17.5.2 Objective Quality Measurement 439 17.6 Tutorial 17.1: Image Compression 440 18 FEATURE EXTRACTION AND REPRESENTATION 447 18.1 Introduction 447 18.2 Feature Vectors and Vector Spaces 448 18.2.1 Invariance and Robustness 449 18.3 Binary Object Features 450 18.3.1 Area 450 18.3.2 Centroid 450 18.3.3 Axis of Least Second Moment 451 18.3.4 Projections 451 18.3.5 Euler Number 452 18.3.6 Perimeter 453 18.3.7 Thinness Ratio 453 18.3.8 Eccentricity 454 10.6 Combining Spatial Enhancement Methods 223 10.7 Tutorial 10.1: Convolution and Correlation 223 10.8 Tutorial 10.2: Smoothing Filters in the Spatial Domain 225 10.9 Tutorial 10.3: Sharpening Filters in the Spatial Domain 228 10.10 Problems 234 11 FREQUENCY DOMAIN FILTERING 235 11.1 Introduction 235 11.2 Fourier Transform: the Mathematical Foundation 237 11.2.1 Basic Concepts 237 11.2.2 The 2D Discrete Fourier Transform: Mathematical Formulation 239 11.2.3 Summary of Properties of the Fourier Transform 241 11.2.4 Other Mathematical Transforms 242 11.3 Low pass Filtering 243 11.3.1 Ideal LPF 244 11.3.2 Gaussian LPF 246 11.3.3 Butterworth LPF 246 11.4 High pass Filtering 248 11.4.1 Ideal HPF 248 11.4.2 Gaussian HPF 250 11.4.3 Butterworth HPF 250 11.4.4 High Frequency Emphasis 251 11.5 Tutorial 11.1: 2D Fourier Transform 252 11.6 Tutorial 11.2: Low pass Filters in the Frequency Domain 254 11.7 Tutorial 11.3: High pass Filters in the Frequency Domain 258 11.8 Problems 264 12 IMAGE RESTORATION 265 12.1 Modeling of the Image Degradation and Restoration Problem 265 12.2 Noise and Noise Models 266 12.2.1 Selected Noise Probability Density Functions 267 12.2.2 Noise Estimation 269 12.3 Noise Reduction Using Spatial domain Techniques 269 12.3.1 Mean Filters 273 12.3.2 Order Statistic Filters 275 12.3.3 Adaptive Filters 278 12.4 Noise Reduction Using Frequency domain Techniques 278 12.4.1 Periodic Noise 279 12.4.2 Bandreject Filter 280 12.4.3 Bandpass Filter 281 12.4.4 Notch Filter 282 12.5 Image Deblurring Techniques 283 12.5.1 Wiener Filtering 286 12.6 Tutorial 12.1: Noise Reduction Using Spatial domain Techniques 289 12.7 Problems 296 13 MORPHOLOGICAL IMAGE PROCESSING 299 13.1 Introduction 299 13.2 Fundamental Concepts and Operations 300 13.2.1 The Structuring Element 301 Appendix A: HUMAN VISUAL PERCEPTION 591 A.1 Introduction 591 A.2 The Human Eye 592 A.3 Characteristics of Human Vision 596 A.3.1 Resolution Viewing Distance and Viewing Angle 596 A.3.2 Detail and Sharpness Perception 598 A.3.3 Optical Transfer Function and Modulation Transfer Function 599 A.3.4 Brightness Perception 600 A.3.5 Contrast Ratio and Contrast Sensitivity Function 603 A.3.6 Perception of Motion 605 A.3.7 Spatiotemporal Resolution and Frequency Response 606 A.3.8 Masking 608 A.4 Implications and Applications of Knowledge about the Human Visual System 609 Appendix B: GUI DEVELOPMENT 611 B.1 Introduction 611 B.2 GUI File Structure 611 B.3 Passing System Control 613 B.4 The UserData Object 615 B.5 A Working GUI Demo 616 B.6 Concluding Remarks 618 REFERENCES 619 INDEX 627 LIST OF FIGURES xxi LIST OF TABLES xxxix FOREWORD xli PREFACE xliii ACKNOWLEDGMENTS xlix PART I IMAGE PROCESSING 1 INTRODUCTION AND OVERVIEW 3 1.1 Motivation 3 1.2 Basic Concepts and Terminology 5 1.3 Examples of Typical Image Processing Operations 6 1.4 Components of a Digital Image Processing System 10 1.5 Machine Vision Systems 12 1.6 Resources 14 1.7 Problems 18 2 IMAGE PROCESSING BASICS 21 2.1 Digital Image Representation 21 2.1.1 Binary (1 Bit) Images 23 2.1.2 Gray Level (8 Bit) Images 24 2.1.3 Color Images 25 2.1.4 Compression 26 2.2 Image File Formats 27 2.3 Basic Terminology 28 2.4 Overview of Image Processing Operations 30 2.4.1 Global (Point) Operations 31 2.4.2 Neighborhood Oriented Operations 31 2.4.3 Operations Combining Multiple Images 32 2.4.4 Operations in a Transform Domain 32 3 MATLAB BASICS 35 3.1 Introduction to MATLAB 35 3.2 Basic Elements of MATLAB 36 3.2.1 Working Environment 36 3.2.2 Data Types 37 3.2.3 Array and Matrix Indexing in MATLAB 37 3.2.4 Standard Arrays 37 3.2.5 Command Line Operations 38 3.3 Programming Tools: Scripts and Functions 38 3.3.1 M Files 39 3.3.2 Operators 40 3.3.3 Important Variables and Constants 42 3.3.4 Number Representation 42 3.3.5 Flow Control 43 3.3.6 Code Optimization 43 3.3.7 Input and Output 43 3.4 Graphics and Visualization 43 3.5 Tutorial 3.1: MATLAB a Guided Tour 44 3.6 Tutorial 3.2: MATLAB Data Structures 46 3.7 Tutorial 3.3: Programming in MATLAB 53 3.8 Problems 59 4 THE IMAGE PROCESSING TOOLBOX AT A GLANCE 61 4.1 The Image Processing Toolbox: an Overview 61 4.2 Essential Functions and Features 62 4.2.1 Displaying Information About an Image File 62 4.2.2 Reading an Image File 64 4.2.3 Data Classes and Data Conversions 65 4.2.4 Displaying the Contents of an Image 68 7.6 Tutorial 7.1: Image Cropping Resizing Flipping and Rotation 138 7.7 Tutorial 7.2: Spatial Transformations and Image Registration 142 7.8 Problems 149 8 GRAY LEVEL TRANSFORMATIONS 151 8.1 Introduction 151 8.2 Overview of Gray level (Point) Transformations 152 8.3 Examples of Point Transformations 155 8.3.1 Contrast Manipulation 155 8.3.2 Negative 157 8.3.3 Power Law (Gamma) Transformations 157 8.3.4 Log Transformations 159 8.3.5 Piecewise Linear Transformations 160 8.4 Specifying the Transformation Function 161 8.5 Tutorial 8.1: Gray level Transformations 163 8.6 Problems 169 9 HISTOGRAM PROCESSING 171 9.1 Image Histogram: Definition and Example 171 9.2 Computing Image Histograms 173 9.3 Interpreting Image Histograms 174 9.4 Histogram Equalization 176 9.5 Direct Histogram Specification 181 9.6 Other Histogram Modification Techniques 184 9.6.1 Histogram Sliding 185 9.6.2 Histogram Stretching 185 9.6.3 Histogram Shrinking 186 9.7 Tutorial 9.1: Image Histograms 188 9.8 Tutorial 9.2: Histogram Equalization and Specification 191 9.9 Tutorial 9.3: Other Histogram Modification Techniques 195 9.10 Problems 200 10 NEIGHBORHOOD PROCESSING 203 10.1 Neighborhood Processing 203 10.2 Convolution and Correlation 204 10.2.1 Convolution in the One Dimensional Domain 204 10.2.2 Convolution in the Two Dimensional Domain 206 10.2.3 Correlation 208 10.2.4 Dealing with Image Borders 210 10.3 Image Smoothing (Low pass Filters) 211 10.3.1 Mean Filter 213 10.3.2 Variations 213 10.3.3 Gaussian Blur Filter 215 10.3.4 Median and Other Nonlinear Filters 216 10.4 Image Sharpening (High pass Filters) 218 10.4.1 The Laplacian 219 10.4.2 Composite Laplacian Mask 220 10.4.3 Directional Difference Filters 220 10.4.4 Unsharp Masking 221 10.4.5 High Boost Filtering 221 10.5 Region of Interest Processing 222 18.3.9 Aspect Ratio 454 18.3.10 Moments 455 18.4 Boundary Descriptors 456 18.4.1 Chain Code Freeman Code and Shape Number 459 18.4.2 Signatures 461 18.4.3 Fourier Descriptors 462 18.5 Histogram based (Statistical) Features 464 18.6 Texture Features 466 18.7 Tutorial 18.1: Feature Extraction and Representation 470 18.8 Problems 474 19 VISUAL PATTERN RECOGNITION 475 19.1 Introduction 475 19.2 Fundamentals 476 19.2.1 Design and Implementation of a Visual Pattern Classifier 476 19.2.2 Patterns and Pattern Classes 478 19.2.3 Data Preprocessing 479 19.2.4 Training and Test Sets 480 19.2.5 Confusion Matrix 480 19.2.6 System Errors 481 19.2.7 Hit Rates False Alarm Rates and ROC Curves 481 19.2.8 Precision and Recall 482 19.2.9 Distance and Similarity Measures 485 19.3 Statistical Pattern Classification Techniques 487 19.3.1 Minimum Distance Classifier 488 19.3.2 k Nearest Neighbors Classifier 490 19.3.3 Bayesian Classifier 490 19.4 Tutorial 19.1: Pattern Classification 491 19.5 Problems 497 PART II VIDEO PROCESSING 20 VIDEO FUNDAMENTALS 501 20.1 Basic Concepts and Terminology 501 20.2 Monochrome Analog Video 507 20.2.1 Analog Video Raster 507 20.2.2 Blanking Intervals 508 20.2.3 Synchronization Signals 509 20.2.4 Spectral Content of Composite Monochrome Analog Video 509 20.3 Color in Video 510 20.4 Analog Video Standards 512 20.4.1 NTSC 513 20.4.2 PAL 513 20.4.3 SECAM 514 20.4.4 HDTV 514 20.5 Digital Video Basics 514 20.5.1 Advantages of Digital Video 515 20.5.2 Parameters of a Digital Video Sequence 516 20.5.3 The Audio Component 517 20.6 Analog to Digital Conversion 517 20.7 Color Representation and Chroma Subsampling 520 20.8 Digital Video Formats and Standards 521 20.8.1 The Rec. 601 Digital Video Format 522 20.8.2 The Common Intermediate Format 523 13.3 Dilation and Erosion 304 13.3.1 Dilation 305 13.3.2 Erosion 307 13.4 Compound Operations 310 13.4.1 Opening 310 13.4.2 Closing 311 13.4.3 Hit or Miss Transform 313 13.5 Morphological Filtering 314 13.6 Basic Morphological Algorithms 315 13.6.1 Boundary Extraction 317 13.6.2 Region Filling 319 13.6.3 Extraction and Labeling of Connected Components 321 13.7 Grayscale Morphology 322 13.7.1 Dilation and Erosion 323 13.7.2 Opening and Closing 323 13.7.3 Top Hat and Bottom Hat Transformations 325 13.8 Tutorial 13.1: Binary Morphological Image Processing 325 13.9 Tutorial 13.2: Basic Morphological Algorithms 330 13.10 Problems 334 14 EDGE DETECTION 335 14.1 Formulation of the Problem 335 14.2 Basic Concepts 336 14.3 First order Derivative Edge Detection 338 14.4 Second order Derivative Edge Detection 343 14.4.1 Laplacian of Gaussian 345 14.5 The Canny Edge Detector 347 14.6 Edge Linking and Boundary Detection 348 14.6.1 The Hough Transform 349 14.7 Tutorial 14.1: Edge Detection 354 14.8 Problems 363 15 IMAGE SEGMENTATION 365 15.1 Introduction 365 15.2 Intensity based Segmentation 367 15.2.1 Image Thresholding 368 15.2.2 Global Thresholding 369 15.2.3 The Impact of Illumination and Noise on Thresholding 370 15.2.4 Local Thresholding 371 15.3 Region based Segmentation 373 15.3.1 Region Growing 374 15.3.2 Region Splitting and Merging 377 15.4 Watershed Segmentation 377 15.4.1 The Distance Transform 378 15.5 Tutorial 15.1: Image Thresholding 379 15.6 Problems 386 16 COLOR IMAGE PROCESSING 387 16.1 The Psychophysics of Color 387 16.1.1 Basic Concepts 388 16.1.2 The CIE XYZ Chromaticity Diagram 390 16.1.3 Perceptually Uniform Color Spaces 393 16.1.4 ICC Profiles 395 16.2 Color Models 396 16.2.1 The RGB Color Model 396 16.2.2 The CMY and CMYK Color Models 398 20.8.3 The Source Intermediate Format 524 20.9 Video Compression Techniques and Standards 524 20.9.1 Video Compression Standards Codecs and Containers 525 20.10 Video Processing in MATLAB 526 20.10.1 Reading Video Files 527 20.10.2 Processing Video Files 527 20.10.3 Playing Video Files 527 20.10.4 Writing Video Files 528 20.11 Tutorial 20.1: Basic Digital Video Manipulation in MATLAB 528 20.12 Tutorial 20.2: Working with YUV Video Data 534 20.13 Problems 539 21 VIDEO SAMPLING RATE AND STANDARDS CONVERSION 541 21.1 Video Sampling 541 21.2 Sampling Rate Conversion 542 21.3 Standards Conversion 543 21.3.1 Deinterlacing 543 21.3.2 Conversion between PAL and NTSC Signals 545 21.3.3 Color Space Conversion 545 21.3.4 Aspect Ratio Conversion 546 21.3.5 3:2 Pull Down 547 21.4 Tutorial 21.1: Line Down Conversion 548 21.5 Tutorial 21.2: Deinterlacing 550 21.6 Tutorial 21.3: NTSC to PAL Conversion 556 21.7 Tutorial 21.4: 3:2 Pull Down 557 21.8 Problems 559 22 DIGITAL VIDEO PROCESSING TECHNIQUES AND APPLICATIONS 561 22.1 Fundamentals of Motion Estimation and Motion Compensation 561 22.2 General Methodologies in Motion Estimation 564 22.2.1 Motion Representation 566 22.2.2 Motion Estimation Criteria 567 22.2.3 Optimization Methods 567 22.3 Motion Estimation Algorithms 568 22.3.1 Exhaustive Search Block Matching Algorithm 568 22.3.2 Fast Algorithms 570 22.3.3 Hierarchical Block Matching Algorithm 571 22.3.4 Phase Correlation Method 573 22.4 Video Enhancement and Noise Reduction 573 22.4.1 Noise Reduction in Video 574 22.4.2 Interframe Filtering Techniques 575 22.5 Case Study: Object Segmentation and Tracking in the Presence of Complex Background 576 22.6 Tutorial 22.1: Block based Motion Estimation 579 22.7 Tutorial 22.2: Intraframe and Interframe Filtering Techniques 585 22.8 Problems 589