Using GLCM features in Haar wavelet transformed space for moving object classification

IET Intelligent Transport SystemsVolume 13, Issue 7 p. 1148-1153 Research ArticleFree Access Using GLCM features in Haar wavelet transformed space for moving object classification Nadia Kiaee, Nadia Kiaee Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, IranSearch for more papers by this authorElham Hashemizadeh, Corresponding Author Elham Hashemizadeh hashemizadeh@kiau.ac.ir orcid.org/0000-0001-5902-4830 Department of Mathematics, Karaj Branch, Islamic Azad University, Karaj, IranSearch for more papers by this authorNima Zarrinpanjeh, Nima Zarrinpanjeh Department of Civil Engineering, Qazvin Branch, Islamic Azad University, Qazvin, IranSearch for more papers by this author Nadia Kiaee, Nadia Kiaee Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, IranSearch for more papers by this authorElham Hashemizadeh, Corresponding Author Elham Hashemizadeh hashemizadeh@kiau.ac.ir orcid.org/0000-0001-5902-4830 Department of Mathematics, Karaj Branch, Islamic Azad University, Karaj, IranSearch for more papers by this authorNima Zarrinpanjeh, Nima Zarrinpanjeh Department of Civil Engineering, Qazvin Branch, Islamic Azad University, Qazvin, IranSearch for more papers by this author First published: 05 April 2019 https://doi.org/10.1049/iet-its.2018.5192Citations: 6AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This article proposes an integrated system for segmentation and classification of two moving objects, including car and pedestrian from their side-view in a video sequence. Based on the use of grey-level co-occurrence matrix (GLCM) in Haar wavelet transformed space, the authors calculated features of texture data from different sub-bands separately. Haar wavelet transform is chosen because the resulting wavelet sub-bands are strongly affecting on the orientation elements in the GLCM computation. To evaluate the proposed method, the results of different sub-bands are compared with each other. Extracted features of objects are classified by using a support vector machine (SVM). Finally, the experimental results showed that use of three sub-bands of wavelets instead of two sub-bands is more effective and has good precision. 1 Introduction Automatic image analysis systems for detection, tracking, and classification of moving objects have been developed within the different fields of computer vision such as computer conceptual relations with humans, robotics and medical imaging, smart video compression, and automatic video analysis [1, 2]. This issue is one of the most important problems in image processing and machine vision. Displaying the location of an object, tracking it in the sequence of video images and classifying objects with high accuracy has its own specific purpose and can help in the precise analysis and monitoring of traffic conditions or the retrieval of a specific object from a video sequence [3-5]. In recent years, many studies have been done for detection and classification of cars and pedestrians. They often used the features like Haar wavelet, histograms of oriented gradients, local binary patterns, and grey-level co-occurrence matrix (GLCM). There are different methods for video surveillance. In [6], Juan and et al. have classified four moving objects, including pedestrians, cars, motorcycles, and bicycles by using local shape features and the histogram of oriented gradients in Haar wavelet transform. These features improve classification performance and provide smaller data dimensionality. In [7], Moreno and et al. present a method based on Histogram to extract features of a specific range of the moving objects and uses support vector machine (SVM) for classification in the crowded traffic scene. Kumar and et al. have classified the flying birds in the sky and the airplane on the runway by using three methods of feature extraction like Haar wavelet, co-occurrence matrix based on Haar wavelet and co-occurrence matrix in [8]. They concluded that co-occurrence matrix based on Haar wavelet leads to get better results in a short time and increase the classification accuracy. Papageorgiou and et al. have extracted modified Haar wavelet features from Haar wavelet transformed space and provided the results for the vehicle and face detection in [9]. Triggs and et al. in [10] introduced the histogram features in Haar wavelet transformed space. In recent years, the histogram feature is used for detection and classification issues such as vehicle detection and facial recognition. Various features have their own power to describe an object. Histogram stored local shape features of considered objects based on the image direction and derivatives size. [Local Binary Pattern.] LBP is a texture descriptor that is computationally efficient, constant in the grey-level variation and can filter out the noise in multi-scale mode by means of the same pattern [11]. Wen and et al. in [12] presented a comprehensive system for the extraction of parked vehicle information, which include vehicle recognition, localisation, classification, and change detection to monitor the parking. This system presents a variable vehicle model to fit the vehicle hypothesis for accurate localisation, and each parameter is used as target visualisation. In [13], Chandra Mouli and et al. presented a pedestrian method to overcome complex background, which is based on SVM and human visual mechanism. According to human visual attention, first mean and Laplacian of Gaussian filter are applied to reduce the background noise but it increases the contrast relation between the foreground and background. In addition, to remove the remaining noise, morphological process on filtered image is used. Then, pedestrian and non-pedestrian are obtained by local threshold segmentation on processed morphological image. In order to detect pedestrian truly and reduce the false detection rate, SVM is used. This paper provides a two-class object detection and classification system for using in a video sequence with a fixed camera. The aim of this paper is to improve classification rate for these two types of moving objects by proposing a new set of features in this field. The proposed method classified objects by using GLCM features in Haar wavelet transformed space with SVM classifier. The remainder of this paper is organised as follows: Section 2 introduces the segmentation method that is used to detect moving objects, Section 3 explains the feature extraction process. Section 4 presents classification process. While Section 5 presents the experimental results of the proposed classification approach. Section 6 includes the conclusions of this paper. 2 Proposed method 2.1 Background subtraction Video data which recorded by a surveillance camera are converted into frames or images. The conventional techniques for extracting moving objects are background subtraction. In this method, the moving object is obtained by subtracting the input frame from the background image. In order to guarantee the motion detection and achieve better results, we can overcome and reduce the changes problems of brightness by updating the background image. In this case, the exact background image will be created. Several background estimation algorithms have been proposed. The background model of our proposed method is implemented by the Gaussian mixture model [12]. Actually, we have used The ForegroundDetector function in our Matlab code, which compares a colour video frame to a background model to determine whether individual pixels are part of the background or the foreground. It computes a foreground mask. Then by using background subtraction, we detect foreground objects in each frame. ForegroundDetector computes and returns a foreground mask using the Gaussian mixture model. The parameters that we set to this function are the NumGaussians to 3 and NumTrainingFrames to 50. NumGaussians is the number of Gaussian mixture model and the NumTrainingFrames is the number of initial video frames for training background model. 2.2 Feature extraction Feature extraction is one of the most important and challenging issues in image processing. In this process, we extract the features that can describe the image and the objects in the image well. Fig. 1 shows the flowchart of the grey-level co-occurrence matrix–Wavelet (GLCM-W) feature extraction process. Pre-processing usually prepares the image for segmentation and handles the changes in size and brightness of objects in an image. The Haar wavelet transformation is proposed to improve the distinguishability of extracted features in two classes of objects. So first, the first-level Haar wavelet transformation is carried out. Then, the GLCM features are gained from the area of each object in each sub-band separately (see Fig. 2) [13, 14]. Fig. 1Open in figure viewerPowerPoint Block diagram of sub-band's feature extraction by applying wavelet Fig. 2Open in figure viewerPowerPoint Haar wavelet transform is applied on an image in the three spectrums (a) Red, (b) Blue and, (c) Green 2.2.1 Pre-processing To remove the undesirable background noise and fill the empty space of the foreground mask, opening and closing of morphological operations are used (Fig. 3). Fig. 3Open in figure viewerPowerPoint Effect of morphological operation in removing background noise output (a), (b) Images which involve noise, (c), (d) Images with noise reduction As the main objective of this paper is moving objects classification by using the extracted features. Hence, we extract the features with the information, which obtained from the object. Although the two-dimensional image after morphological operations has no appropriate information, but we use it to find the coordinates of the moving object in the colour frames. Therefore, the background and foreground labels refer to the pixels with the value of zero and one, respectively. After the labelling step, the maximum and minimum coordinates of the rows and columns pixel with one value is applied to colour frames. In this case, we find the position of moving objects in the colour frame by using the coordinates, which obtained from the two-dimensional frame. A window rectangle around any detected object is created as shown in Fig. 4. Fig. 4Open in figure viewerPowerPoint Rectangular window created around the (a) Car and, (b) Pedestrian 2.2.2 Haar wavelet transformed space Alfred Haar, Hungarian mathematician in 1909, first proposed the wavelet idea of its simplest form. The only problem with this wavelet is its discontinuity and uncertain nature [16, 17]. Haar mother's wavelet is defined as follows: (1) The scaling function of the Haar wavelet is: The scaling function can decompose an image into four sub-images. (Fig. 5). Fig. 5Open in figure viewerPowerPoint Decomposition of four sub bands Applying wavelet to an image: In discrete wavelet transform, the signal of an image is decomposed into two sub-sections through low- and high-pass filters at each step. The low-pass filter corresponds to the averaging operator, which summarises the useless information of the signal. The high-pass filter, which also corresponds to the differential operator, summaries the detailed signal information. A two-dimensional conversion is the result of two separate one-dimensional transformations. According to Fig. 6, the low-pass filter coefficients are stored in the left-hand side of the matrix and the high-pass filter coefficients are stored at the right-hand of the matrix. Due to the reduction, the size of the entire transformed image is the same size as the original image. Then, we filter the image along the x and y axes and reduce it with two factors at the end. After applying the filter along x and y axes, the image is divided into four sub-bands [18-20]. Fig. 6Open in figure viewerPowerPoint Original image and apply a two-level wavelet transform 2.2.3 GLCM features in the Haar-wavelet transformed space In order to achieve further details in the input image, we apply Haar wavelet transform in three-colour range of red, green, and blue spectrums, separately. Each object will pass from two banks of high-pass and low-pass filters. Thus, each image in each spectrum decomposed to four sub-bands as shown in Fig. 5. A total of 12 images is achieved. While other sub-bands contain more detailed information in a different orientation, LL sub-band provides an approximate image. Therefore, we ignored LL sub-band and just used the other sub-bands for feature extraction. In fact, the low-pass filter corresponds to an averaging operator that summarises the useless signal information and high-pass filters corresponds to the difference operator that provides a summary of the detailed signal information. Since applying wavelet is affected on the image texture, we use the GLCM to extract features. In this matrix, as shown in Fig. 7, the events of pixels are checked in a (2, 0) row. After applying wavelet on each spectrum, nine GLCMs are extracted from three sub-bands separately [21]. Fig. 7Open in figure viewerPowerPoint Calculation of the co-occurrence matrix in row (2,0) The feature vector, which consists of four components, energy, correlation, contrast, and homogeneity, is computed in each matrix by Table 1. Then, 36 features of each object stored in the feature matrix. According to the inappropriate information of LL sub-band, we do not consider it. Therefore, we extract six GLCMs from two sub-bands separately, as shown in Fig. 2. In this case, 24 features of each object stored in the feature matrix as shown in Fig. 1. Table 1. GLCM texture statistics defined Row Features Formula 1 energy 2 entropy 3 contrast 4 homogeneity 5 correlation 6 average 2.3 Classification The purpose of using SVM is to separate feature vectors of a p -dimension space into a higher p -1 dimensional feature space and then creating a hyper-plane with maximum margin. SVM decides based on the hyper-plane with maximum margin. Equation (1) is defined hyper-plane with maximum margin. X is the input variables, W is the coefficients vector, and b is the constant, which determine the hyper-plane. (2) Here, features are energy, homogeneity, conflict, cohesion, and classes include pedestrian and cars. First, we create the labels matrix by assigning a label to each row of the features object. Thus, using cross-validation method, 70% of features matrix are considered for training and the rest of them as test data. In the test phase, with the arrival of each new object, SVM predicts by using the features of input object and their similarity. If the value of is belonged to , it classified as pedestrian and otherwise as car (Fig. 8). In other words, if there is similarity with pedestrian features class, the desired object is classified as a pedestrian and otherwise as cars. There are a total of N training samples for each class of the pedestrians and cars. Suppose that all the relevant data points belong to one class. We want to assign the X input vector to one of −1, +1 classes. Therefore, if the sign of the function in (1) is positive, we assign +1 to x and otherwise −1 will assign to X. For training the SVM, the favourite output to train N samples is '1', which belongs to the pedestrian class and the favourite output is '−1' for cars. It should be mentioned that the main subject here is to select features that can help us to improve our problem solving and we use the Rbf kernel function of second poly-order. To conclude, the reason for choosing this classifier is that we had two classes and fortunately, SVM was well-suited to our problem. Fig. 8Open in figure viewerPowerPoint Flowchart of pedestrian and car classification using Support Vector Machine 3 Results and discussion To evaluate the proposed method, the collected data in the field of urban transport are provided with a fixed camera from their side view (Fig. 9). Then, a human labels the data so the standard data are available. The images of car and pedestrian are captured by Cannon A1400 camera, with size of each frame and 30 frames in each second of imaging rate. MATLAB programming environment is introduced for the implementation and testing the proposed system. This programming language has the ability and flexibility in working with video and images in various fields such as image processing and computer vision. The program is run on a personal computer with Core-i7 processor, 2.2 GHz frequency, and 8 GB Memory. This hardware is not much stronger than what is used in real applications in intelligent systems. So solving the problem on this hardware is promising good performance in real issues. Fig. 9Open in figure viewerPowerPoint Collected training samples 3.1 Performance of the proposed method in different sub-bands Table 2 shows the accuracy, sensitivity, specificity for each of the three and two sub-bands. Fig. 10 shows comparison of the number of features and accuracy for each set of extracted features. It illustrates that by increasing the number of features, which extracted from different sub-bands, the accuracy of the classification will increase. Table 2. Evaluation of two and three sub-bands Proposed method Features number Accuracy Precision Sensitivity Specificity in three sub-bands 36 0.9875 0.9867 1 0.8333 on two sub-bands 24 0.8125 0.9360 0.8559 0.2778 Fig. 10Open in figure viewerPowerPoint Comparison of two and three sub-bands in red, green and blue spectrums. Features number = 12 in three sub-bands and features number = 8 in two sub-bands Since the sensitivity of surveillance cameras in different colour spectrums are different, we have examined the results of extracted features from each frame by applying a wavelet in red, green, and blue separately, as shown in Table 3 [22, 23]. Fig. 10 shows the results of the extracted features in three colour spectrums in a bar graph that the blue colour range is leading to decrease the classification accuracy [24]. Table 3. Evaluation of two and three sub-bands in three red, green and blue spectrums separately Proposed method Features numbers Spectrums Accuracy Precision Sensitivity Specificity Recall three- sub bands 12 red 0.9858 0.9955 1 0.9444 1 12 green 0.9958 0.9955 1 0.9444 1 12 blue 0.9875 0.9867 1 0.8333 1 two-sub bands 8 red 0.9942 0.9942 1 0.9333 1 8 green 0.9917 1 0.9910 1 0.9910 8 blue 0.9917 0.9955 0.9955 0.9944 0.9955 4 Conclusion This paper classifies two types of moving objects using GLCM features in Haar wavelet transformed space with SVM in intelligent monitoring systems with fixed cameras. First, according to the characteristics of wavelet such as de-noising and precise image retrieval, wavelet has been applied to the area of each object. As LL sub-band contains noise and other useless information, the results have been examined by removing this sub-band and using the two and three sub-bands of the wavelet. Information of texture image has been computed from GLCM. The GLCM-W feature, which extracts the GLCM features from the Haar wavelet-transformed space, shows the advantages of desired classification performance. The location information of the area of each object is investigated in three red, green, and blue colour spectrums separately. It can be concluded that for three sub-bands of wavelet decomposition in red and green spectrums, the features were more outstanding instead of determining them in two sub-bands of blue one. In addition to these, the introduced algorithm is optimised for applications that the histogram colour variation of each object is low. It is also recommended to compare the results to other similar methods as performed in pioneering studies presented in Varma et al. [24] and Fu et al. [25], which we did not practice in this research due to our focus on SVM and wavelet. This could be an interesting field of research for future work. 5 Acknowledgments The authors would like to thank the anonymous reviewers of this paper for their careful reading, constructive comments and suggestions which have improved the paper very much. 6 References 1Hu W. Tan T. Wang L. et al.: 'A survey on visual surveillance of object motion and behaviors', IEEE Trans. Syst. Man Cybern. C, Appl. Rev., 2004, 34, (3), pp. 334– 352 2Zhang R. Liu X. Hu J. et al.: 'A fast method for moving object detection in video surveillance image', Signal. Image. Video. Process., 2017, 11, (5), pp. 841– 848 3Kavitha C. Ashok S.D.: 'A new approach to spindle radial error evaluation using a machine vision system', Metrol. Meas. Syst., 2017, 24, (1), pp. 201– 219 4Wen X. Shao L. Fang W. et al.: 'Efficient feature selection and classification for vehicle detection', IEEE Trans. Circuits Syst. Video Technol., 2015, 25, (3), pp. 508– 517 5Shah M. Deng J.D. Woodford B.J.: 'Video background modeling: recent approaches, issues and our proposed techniques', Mach. Vis. Appl., 2014, 25, (5), pp. 1105– 1119 6Liang C.-W. Juang C.-F.: 'Moving object classification using local shape and HOG features in wavelet-transformed space with hierarchical SVM classifiers', Appl. Soft Comput., 2015, 28, pp. 483– 497 7Zangenehpour S. Miranda-Moreno L.F. Saunier N.: 'Automated classification based on video data at intersections with heavy pedestrian and bicycle traffic: methodology and application', Transp. Res. C, Emerg. Technol., 2015, 56, pp. 161– 176 8Mohanaiah P. Sathyanarayana P. GuruKumar L.: 'Image texture feature extraction using GLCM approach', Int. J. Sci. Res. Publ., 2013, 3, (5), p. 1 9Papageorgiou C.P. Oren M. Poggio T.: ' A general framework for object detection'. Sixth Int. Conf. on Computer Vision, Bombay, India, 1998 10Dalal N. Triggs B.: ' Histograms of oriented gradients for human detection'. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005. CVPR 2005, Kolkata, India, 2005 11Nanni L. Lumini A. Brahnam S.: 'Survey on LBP based texture descriptors for image classification', Expert Syst. Appl., 2012, 39, (3), pp. 3634– 3641 12Zivkovic Z.: ' Improved adaptive Gaussian mixture model for background subtraction'. Proc. of the 17th Int. Conf. on Pattern Recognition, 2004. ICPR 2004, Cambridge, UK, 2004 13Mokji M. Bakar S.A.: ' Gray level co-occurrence matrix computation based on haar wavelet' ( IEEE, London, UK, 2007) 14Guo L. Cui H. Li P. et al.: 'Urban road congestion recognition using multi-feature fusion of traffic images', J. Artif. Intell., 2016, 1, pp. 20– 24 15Dheekonda R.S.R. Panda S.K. Khan N. et al.: ' Object detection from a vehicle using deep learning network and future integration with multi-sensor fusion algorithm'. SAE Technical Paper, 2017 16Kaur H. Mittal R. Mishra V.: 'Haar wavelet approximate solutions for the generalized Lane–Emden equations arising in astrophysics', Comput. Phys. Commun., 2013, 184, (9), pp. 2169– 2177 17Fard M.R. Mohaymany A.S. Shahri M.: 'A new methodology for vehicle trajectory reconstruction based on wavelet analysis', Transp. Res. C, Emerg. Technol., 2017, 74, pp. 150– 167 18Agarwal S. Verma A. Singh P.: ' Content based image retrieval using discrete wavelet transform and edge histogram descriptor'. 2013 Int. Conf. on Information Systems and Computer Networks (ISCON), Bali, Australia, 2013 19Tang Y. Zhang C. Gu R. et al.: 'Vehicle detection and recognition for intelligent traffic surveillance system', Multimedia Tools Appl., 2017, 76, (4), pp. 5817– 5832 20Hashemizadeh E. Rahbar S.: 'The application of legendre multiwavelet functions in image compression', J. Mod. Appl. Stat. Methods., 2016, 15, (2), pp. 510– 525 21Kamkar S. Safabakhsh R.: 'Vehicle detection, counting and classification in various conditions', IET Intell. Transp. Syst., 2016, 10, (6), pp. 406– 413 22Huang X. Zhang L.: 'An SVM ensemble approach combining spectral, structural, and semantic features for the classification of high-resolution remotely sensed imagery', IEEE Trans. Geosci. Remote Sens., 2013, 51, (1), pp. 257– 272 23Kiaee N. Hashemizadeh E. Zarrinpanjeh N.: 'Evaluation of moving object detection based on various input noise using fixed camera', ISPRS-Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., 2017, pp. 151– 154 24Varma S. Sreeraj M.: ' Object detection and classification in surveillance system'. 2013 IEEE Recent Advances in Intelligent Computational Systems (RAICS), Trivandrum, 2013, pp. 299– 303 25Fu Y. Ma D. Zhang H. et al.: ' Moving object recognition based on SVM and binary decision tree'. 2017 6th Data Driven Control and Learning Systems (DDCLS), Chongqing, 2017, pp. 495– 500 Citing Literature Volume13, Issue7July 2019Pages 1148-1153 FiguresReferencesRelatedInformation