# Selection of Outline Descriptors Based on LightGBM with Application to Infrared Image Target Recognition

Authors: Xiaohong Hu; Ziyang Yao
Journal: Scientific Programming
 (2021)

---

## Abstract

Infrared sensing technology can be well used for night observation, which is becoming an important measure for battlefield reconnaissance. It is a powerful way to implement precision strikes and situational awareness by improving the ability of target recognition based on infrared images. For the problem of infrared image recognition, the Light Gradient Boosting Machine (LightGBM) is employed to select the outline descriptors extracted based on the elliptic Fourier series (EFS), which is combined with sparse representation-based classification (SRC) to achieve target recognition. First, based on the target outlines in the infrared image, the multi-order outline descriptors are extracted to characterize the essential characteristics of the target to be recognized. Then, the LightGBM feature selection algorithm is used to screen the multi-order outline descriptors to reduce redundancy and improve the pertinence of features. Finally, the selected outline descriptors are classified based on SRC. The method effectively improves the effectiveness of the final features through the feature selection of LightGBM and reduces the computational complexity of classification at the same time, which is beneficial to improve the overall recognition performance. The mid-wave infrared (MWIR) dataset of various targets is employed to carry out verification experiments for the proposed method under three different conditions of original samples, noisy samples, and partially occluded samples. By comparing the proposed method with several types of existing infrared target recognition methods, the results show that the proposed method can achieve better performance.


---

## Body

## 1. Introduction

The development and maturity of infrared imaging technologies provides an important tool for night observation [1–4]. In the civil field, traffic monitoring can be performed based on infrared images or videos, which effectively avoids accidents. In the military field, infrared imaging can be used for battlefield monitoring and precision strikes. The target recognition method based on infrared image aims to confirm the target category in the measured samples, so as to obtain valuable information for technical support. As a supervised classification problem, infrared image target recognition usually adopts a two-stage process of feature extraction and classification. Feature extraction acquires features such as intensity distribution, local texture, key points, and regional distributions of the target through the analysis of infrared images. In [5], the boundary features were extracted for target recognition. In [6], the image moments were employed to analyze the regional characteristics for discriminating different categories of targets. Local textual descriptors, i.e., histogram of oriented gradient (HOG) features, were applied to infrared image target recognition in [7]. The multi-scale HOG features were developed in [8] to describe the intensity distribution properties of infrared images for target recognition. The classification process employs mature classifiers to train the extracted features and obtain the corresponding categories of the unknown samples. Typical classifiers used in infrared image target recognition include support vector machine (SVM) [8], sparse representation-based classification (SRC) [9], neural networks [10], etc. Since the emergence of deep learning theory and algorithms, the classification models represented by convolutional neural network (CNN) have been widely used in image processing and have become a powerful tool for infrared target recognition [10–13].The results of feature extraction are used as the input of subsequent decision making. In this sense, the effectiveness of the extracted features directly determines the validity of the final decision. According to the existing literatures, most of the features used in infrared target recognition were directly applied based on mature algorithms in the field of pattern recognition, lacking targeted analysis and screening. In fact, some of the extracted features may have little contribution to the final decision or even cause negative influences on the classification process. For example, the redundant features would definitely increase the computational load and possibly cause some wrong decisions. In [14, 15], the genetic algorithm and particle swarm optimization algorithm were used to select the extracted Zernike moment features for synthetic aperture radar (SAR) target recognition. A basic comparison between the results of using all the features and the selected features verified the necessity and validity of feature selection. Inspired by this work, this paper employs the Light Gradient Boosting Machine (LightGBM) [16–20] for the selection of outline descriptors extracted by elliptic Fourier series (EFS) [21–23] and applies it in infrared image target recognition. As a new boosting algorithm, LightGBM is operated on the gradient boosting decision tree (GBDT) developed in the early stage and optimizes learning through additive models and forward step-by-step algorithms. According to relevant research studies, LightGBM can effectively improve the learning speed and model accuracy. Based on LightGBM, the extracted multi-order outline descriptors can be analyzed and screened. Only the effective components are maintained, and redundant ones are eliminated, so as to ensure the effectiveness of the final features. In the classification stage, SRC is used as the basic classifier to characterize and recognize the final selected outline descriptors. As a classifier based on the compressive sensing theory, SRC could adapt to different kinds of features and achieve robust performance [24–31]. In the experiments, the public mid-wave infrared (MWIR) target image dataset is employed to test and verify the proposed method, and the results reflect its effectiveness and robustness.


## 2. Outline Descriptors

The ellipse Fourier descriptor uses ellipse superposition to approach the boundary of a specific object. It can approach the boundary infinitely in the form of multiple harmonics and has the advantages of scale transformation, rotation transformation, and invariance of starting point transformation [21–23]. This paper first uses the method in [32–35] to obtain the outline point set of the target. For a closed boundary C, it can be expressed as a vector form vt=xtytT, where t∈0,2π. Therefore, vt is the periodic function of the parameter t, which can be expressed in the form of Fourier series as shown in the following equation:(1)xtyt=∑k=0∞Fkcosktsinkt,where Fk∈R2×2,k=0,1,2,…, represents the EFS outlier descriptors, which are calculated as follows:(2)Fk=akbkckdk=12π∫02πxtytTcosktsinktdt.In equation (2), F0 is the 0-order elliptic Fourier descriptor, which represents the DC component of the harmonic. It denotes the center of the closed boundary, which contains two parameters a0c0T. For k≥1 subharmonics, there are four coefficients akbkckdkT. This paper adopts the first 6 orders of the EFS outline descriptors for the following classification.


## 3. Proposed Target Recognition Method

### 3.1. Feature Selection Based on LightGBM

The multi-layer features from the same images are usually not completely uncorrelated, and the contribution of different features to the target classification is also distinguished. As the number of extracted features increases, the amount of repetitive redundancy will increase, and the noise and error will also increase. The increase will not only increase the calculation amount of the algorithm but also reduce the recognition rate of the target. Therefore, it is necessary to select features that are beneficial to target classification. In this paper, LightGBM is used for feature selection [16–20], the training samples are put into LightGBM for training, the importance of each feature is calculated, and the key features are selected according to the importance to reduce the number of features required for classification. The classification performance reduces the recognition time at the same time.The LightGBM algorithm measures the importance of feature attributes according to the total number of times the feature is used to split in all decision trees, sorts the feature elements in descending order, starts from the full set of sample features, and judges whether to exclude according to the accuracy of the result. The current feature with the lowest degree of importance is looped in this way to achieve feature selection. The specific process is described as follows (Algorithm1).Algorithm 1: Feature selection using LightGBM.
Input: dataset D, feature set F=Ti∣i=1,2,…,d;Output: optimal feature subset Fbest;Step 1: LightGBM is used to calculate the importanceIi of sample feature elements Ti, respectively.Step 2: the feature elementsTi are arranged in descending order according to the results Ii obtained in the first step.Step 3: LightGBM is used to evaluate and calculate the accuracyai of the current feature subset F.Step 4: the sorted feature subset is selected in step 2 by deleting the feature elementTi with the least importance in F and calculating the recognition accuracy abest.Step 5: ifai<abest, then go back to step 3 for execution; otherwise, recycle the feature elements Ti deleted in the previous step and then go back to step 3 for execution.Step 6: when all the feature elementsTi have been traversed, the optimal feature subset Fbest is output.It can be seen that the proposed LightGBM feature selection algorithm sorts according to the importance of features and preliminary evaluation of the model results, which greatly reduces the volatility of features. It is beneficial to the improvement of the accuracy of target recognition in the later stage. At the same time, because the features of relatively low importance are deleted, the redundancy of the features is reduced, so the time consumption required for target recognition can also be reduced.


### 3.2. SRC for Classification

SRC is a non-parametric classifier developed from the compressive sensing theory, which was first applied to face recognition with excellent performance. In that work, the effectiveness and robustness of SRC were validated. Afterwards, SRC was widely used in pattern recognition like SAR target recognition [25–27]. For a sample from the ith class, SRC assumes that it can be linearly represented by the samples from the same class as follows:(3)yi,test=Ai∗xi+εi,where yi,test is a test sample assumed from the ith class; Ai comprises the training samples of the ith class; xi denotes the coefficient vector with only a few non-zero elements; and εi is the reconstruction error.When the test sampley is from an unknown class, SRC is performed on the global dictionary A=A1,A2,…,AC as follows:(4)x^=argminx0,s.t.y−Ax22≤ε,where x^ denotes the coefficient vector over the global dictionary.The optimization task in (4) can be robustly solved by the ℓ1-norm relaxation or greedy algorithm like orthogonal matching pursuit (OMP). With an optimal solution of x^, the reconstruction errors related to different classes can be computed as equation (5) for decision making.(5)ri=y−A∗δiα^2,Classy=argminiri,where δiα^ corresponds to the coefficients of ith training class.


### 3.3. Implementation Details

Figure1 shows the basic process of the proposed method, which can be defined as two stages of feature extraction and classification decision according to traditional idea. In the feature extraction stage, the first step is to use EFS outline descriptors to analyze the target contour in the infrared image to obtain multi-order features. On this basis, LightGBM is used for training and learning, the best feature subset is selected, and the redundant components are eliminated. For the training samples, the same operation is followed to construct the corresponding global dictionary, which contains the outline descriptors corresponding to all the samples. Finally, SRC is used to classify the outline descriptors of the test sample and obtain its corresponding target category.Figure 1
Procedure of infrared target recognition based on selected outline descriptors by LightGBM.


## 3.1. Feature Selection Based on LightGBM

The multi-layer features from the same images are usually not completely uncorrelated, and the contribution of different features to the target classification is also distinguished. As the number of extracted features increases, the amount of repetitive redundancy will increase, and the noise and error will also increase. The increase will not only increase the calculation amount of the algorithm but also reduce the recognition rate of the target. Therefore, it is necessary to select features that are beneficial to target classification. In this paper, LightGBM is used for feature selection [16–20], the training samples are put into LightGBM for training, the importance of each feature is calculated, and the key features are selected according to the importance to reduce the number of features required for classification. The classification performance reduces the recognition time at the same time.The LightGBM algorithm measures the importance of feature attributes according to the total number of times the feature is used to split in all decision trees, sorts the feature elements in descending order, starts from the full set of sample features, and judges whether to exclude according to the accuracy of the result. The current feature with the lowest degree of importance is looped in this way to achieve feature selection. The specific process is described as follows (Algorithm1).Algorithm 1: Feature selection using LightGBM.
Input: dataset D, feature set F=Ti∣i=1,2,…,d;Output: optimal feature subset Fbest;Step 1: LightGBM is used to calculate the importanceIi of sample feature elements Ti, respectively.Step 2: the feature elementsTi are arranged in descending order according to the results Ii obtained in the first step.Step 3: LightGBM is used to evaluate and calculate the accuracyai of the current feature subset F.Step 4: the sorted feature subset is selected in step 2 by deleting the feature elementTi with the least importance in F and calculating the recognition accuracy abest.Step 5: ifai<abest, then go back to step 3 for execution; otherwise, recycle the feature elements Ti deleted in the previous step and then go back to step 3 for execution.Step 6: when all the feature elementsTi have been traversed, the optimal feature subset Fbest is output.It can be seen that the proposed LightGBM feature selection algorithm sorts according to the importance of features and preliminary evaluation of the model results, which greatly reduces the volatility of features. It is beneficial to the improvement of the accuracy of target recognition in the later stage. At the same time, because the features of relatively low importance are deleted, the redundancy of the features is reduced, so the time consumption required for target recognition can also be reduced.


## 3.2. SRC for Classification

SRC is a non-parametric classifier developed from the compressive sensing theory, which was first applied to face recognition with excellent performance. In that work, the effectiveness and robustness of SRC were validated. Afterwards, SRC was widely used in pattern recognition like SAR target recognition [25–27]. For a sample from the ith class, SRC assumes that it can be linearly represented by the samples from the same class as follows:(3)yi,test=Ai∗xi+εi,where yi,test is a test sample assumed from the ith class; Ai comprises the training samples of the ith class; xi denotes the coefficient vector with only a few non-zero elements; and εi is the reconstruction error.When the test sampley is from an unknown class, SRC is performed on the global dictionary A=A1,A2,…,AC as follows:(4)x^=argminx0,s.t.y−Ax22≤ε,where x^ denotes the coefficient vector over the global dictionary.The optimization task in (4) can be robustly solved by the ℓ1-norm relaxation or greedy algorithm like orthogonal matching pursuit (OMP). With an optimal solution of x^, the reconstruction errors related to different classes can be computed as equation (5) for decision making.(5)ri=y−A∗δiα^2,Classy=argminiri,where δiα^ corresponds to the coefficients of ith training class.


## 3.3. Implementation Details

Figure1 shows the basic process of the proposed method, which can be defined as two stages of feature extraction and classification decision according to traditional idea. In the feature extraction stage, the first step is to use EFS outline descriptors to analyze the target contour in the infrared image to obtain multi-order features. On this basis, LightGBM is used for training and learning, the best feature subset is selected, and the redundant components are eliminated. For the training samples, the same operation is followed to construct the corresponding global dictionary, which contains the outline descriptors corresponding to all the samples. Finally, SRC is used to classify the outline descriptors of the test sample and obtain its corresponding target category.Figure 1
Procedure of infrared target recognition based on selected outline descriptors by LightGBM.


## 4. Experiments and Discussion

### 4.1. Dataset

The experiments are carried out on the MWIR dataset, which contains images of 10 typical military targets acquired by the mid-wave infrared sensors. Figure2 shows the exemplar images of the 10 targets. According to the relevant experimental settings in [21], 120 images are randomly selected as the training set and 100 images are used as the test sample. During the experiment, four types of methods are selected for comparison, which are denoted as SVM, SRC, EFS, and CNN methods. Among them, the EFS method does not use the feature selection by LightGBM in the proposed method and directly classifies the extracted multi-order outline descriptors based on SRC.Figure 2
Images of targets in MWIR dataset. (a) Pickup. (b) SUV. (c) BTR70. (d) BRDM2. (e) BMP2. (f) T72. (g) ZSU23-4. (h) 2S3. (i) MTLB. (j) D20.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)


### 4.2. Original Test Samples

First, experiments are conducted on the basis of the original training and test samples. At this time, the test sample and the training sample come from similar conditions and their difference is small. So, the recognition problem is relatively simple. Table1 shows the detailed recognition results of the ten types of targets in this paper. The recognition rates of all types of targets are above 97%, and the final average recognition rate is 97.9%, reflecting the effectiveness of the proposed method. Table 2 further compares the average recognition rates of different methods on the original samples. The method in this paper has the highest performance and shows its advantages. The performance of the CNN method under the current conditions is second only to the method in this paper, indicating the effectiveness of the deep learning model. Compared with the method that directly uses outline descriptors, this paper introduces LightGBM to analyze and select them, and the retained features further improve the recognition performance, which shows the effectiveness of the proposed method.Table 1
Recognition results of the proposed method on the original test samples.
ClassPickupSUVBTR70BRDM2BMP2T72ZSU23/42S3MTLBD20Recognition rate (%)Pickup9702010000097SUV0980100000198BTR70001000000000100BRDM21109800000098BMP20010970010197T720002196010096ZSU23/420100097000972S3000000010000100MTLB0000000099199D200101010009797Average (%)97.9Table 2
Performance of different methods on original samples.
MethodAverage recognition rate (%)Proposed97.9SVM94.5SRC95.1EFS96.9CNN97.2


### 4.3. Noisy Test Samples

Natural noise, perceived interference, and sensor thermal noise may all cause the signal-to-noise ratio (SNR) of the final infrared image to be reduced. Therefore, it is necessary to improve the performance of the identification method under noise interference conditions. This paper firstly generates test sets with different signal-to-noise ratios based on the original test samples by adding white noise and then tests various methods to obtain the results shown in Figure3. It can be seen that noise corruption has caused the performance of various methods to decrease, but the proposed method maintains the strongest robustness. Compared with SVM, the SRC method has performance advantages and reflects the adaptability of the sparse representation mechanism to the influence of noise. Despite the lack of secondary feature selection, the EFS method can still maintain good noise robustness, reflecting the stability of outline descriptors under noise interference. In detail, EFS outline descriptors are mainly related to the target contour, and the overall regional characteristics of the target can be kept robust under noise corruption conditions, so this type of method can achieve stable performance under noise conditions. The proposed method combines the advantages of outline descriptors, LightGBM-based feature selection, and sparse representation, so the final recognition result has advantages.Figure 3
Performance of different methods under noise corruption.


### 4.4. Partially Occluded Test Samples

There may be two reasons to cause partially occluded infrared images. One is that the target itself is partially absence, and the other is that it is partially occluded during the imaging process. In both cases, there are some missing target characteristics in the final image, which makes the recognition problem more complicated. In this paper, based on the original test samples, the partial occlusion algorithm is used to simulate the construction of test sets under different missing levels (measured by the ratio of the occluded area to the target area). Figure4 shows the average recognition rate curve of various methods under partial missing conditions. It can be seen that the performance of all methods has been significantly reduced under the partial missing condition, and the method in this paper can maintain the highest recognition rate at all levels, indicating its robustness. The SRC method has a performance advantage over the SVM method, which shows the effectiveness of the sparse representation mechanism for partial deletions. The EFS method is at a disadvantage compared to the proposed method due to the lack of secondary feature screening, but it has certain advantages over other methods. This shows that the outline descriptors have certain adaptability to the damage of the target area caused by partial missing. In summary, this method combines outline descriptors, LightGBM-based feature screening, and SRC-based decision making to jointly improve the recognition performance under the current test conditions.Figure 4
Performance of different methods under partial occlusions.


## 4.1. Dataset

The experiments are carried out on the MWIR dataset, which contains images of 10 typical military targets acquired by the mid-wave infrared sensors. Figure2 shows the exemplar images of the 10 targets. According to the relevant experimental settings in [21], 120 images are randomly selected as the training set and 100 images are used as the test sample. During the experiment, four types of methods are selected for comparison, which are denoted as SVM, SRC, EFS, and CNN methods. Among them, the EFS method does not use the feature selection by LightGBM in the proposed method and directly classifies the extracted multi-order outline descriptors based on SRC.Figure 2
Images of targets in MWIR dataset. (a) Pickup. (b) SUV. (c) BTR70. (d) BRDM2. (e) BMP2. (f) T72. (g) ZSU23-4. (h) 2S3. (i) MTLB. (j) D20.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)


## 4.2. Original Test Samples

First, experiments are conducted on the basis of the original training and test samples. At this time, the test sample and the training sample come from similar conditions and their difference is small. So, the recognition problem is relatively simple. Table1 shows the detailed recognition results of the ten types of targets in this paper. The recognition rates of all types of targets are above 97%, and the final average recognition rate is 97.9%, reflecting the effectiveness of the proposed method. Table 2 further compares the average recognition rates of different methods on the original samples. The method in this paper has the highest performance and shows its advantages. The performance of the CNN method under the current conditions is second only to the method in this paper, indicating the effectiveness of the deep learning model. Compared with the method that directly uses outline descriptors, this paper introduces LightGBM to analyze and select them, and the retained features further improve the recognition performance, which shows the effectiveness of the proposed method.Table 1
Recognition results of the proposed method on the original test samples.
ClassPickupSUVBTR70BRDM2BMP2T72ZSU23/42S3MTLBD20Recognition rate (%)Pickup9702010000097SUV0980100000198BTR70001000000000100BRDM21109800000098BMP20010970010197T720002196010096ZSU23/420100097000972S3000000010000100MTLB0000000099199D200101010009797Average (%)97.9Table 2
Performance of different methods on original samples.
MethodAverage recognition rate (%)Proposed97.9SVM94.5SRC95.1EFS96.9CNN97.2


## 4.3. Noisy Test Samples

Natural noise, perceived interference, and sensor thermal noise may all cause the signal-to-noise ratio (SNR) of the final infrared image to be reduced. Therefore, it is necessary to improve the performance of the identification method under noise interference conditions. This paper firstly generates test sets with different signal-to-noise ratios based on the original test samples by adding white noise and then tests various methods to obtain the results shown in Figure3. It can be seen that noise corruption has caused the performance of various methods to decrease, but the proposed method maintains the strongest robustness. Compared with SVM, the SRC method has performance advantages and reflects the adaptability of the sparse representation mechanism to the influence of noise. Despite the lack of secondary feature selection, the EFS method can still maintain good noise robustness, reflecting the stability of outline descriptors under noise interference. In detail, EFS outline descriptors are mainly related to the target contour, and the overall regional characteristics of the target can be kept robust under noise corruption conditions, so this type of method can achieve stable performance under noise conditions. The proposed method combines the advantages of outline descriptors, LightGBM-based feature selection, and sparse representation, so the final recognition result has advantages.Figure 3
Performance of different methods under noise corruption.


## 4.4. Partially Occluded Test Samples

There may be two reasons to cause partially occluded infrared images. One is that the target itself is partially absence, and the other is that it is partially occluded during the imaging process. In both cases, there are some missing target characteristics in the final image, which makes the recognition problem more complicated. In this paper, based on the original test samples, the partial occlusion algorithm is used to simulate the construction of test sets under different missing levels (measured by the ratio of the occluded area to the target area). Figure4 shows the average recognition rate curve of various methods under partial missing conditions. It can be seen that the performance of all methods has been significantly reduced under the partial missing condition, and the method in this paper can maintain the highest recognition rate at all levels, indicating its robustness. The SRC method has a performance advantage over the SVM method, which shows the effectiveness of the sparse representation mechanism for partial deletions. The EFS method is at a disadvantage compared to the proposed method due to the lack of secondary feature screening, but it has certain advantages over other methods. This shows that the outline descriptors have certain adaptability to the damage of the target area caused by partial missing. In summary, this method combines outline descriptors, LightGBM-based feature screening, and SRC-based decision making to jointly improve the recognition performance under the current test conditions.Figure 4
Performance of different methods under partial occlusions.


## 5. Conclusion

This paper proposes an infrared image target recognition method based on feature selection of EFS outline descriptors. LightGBM is used to screen the multi-order outline descriptors of infrared targets to eliminate redundant components. On this basis, SRC is used to classify the selected outline descriptors to determine the target category of the test sample. The proposed method improves the final recognition performance through effective extraction, secondary feature selection, and SRC-based decision making. Experiments are carried out on the MWIR dataset to classify 10 typical military targets. According to the experimental results, the following conclusions can be drawn. (1) The proposed method achieves better recognition performance on the original test samples, noisy samples, and partially occluded samples than the four types of comparison methods, showing its performance advantages. (2) The introduction of LightGBM for the selection of the original multi-order outline descriptors effectively improved the overall recognition performance in the proposed method. (3) The LightGBM-based feature selection can be regarded as a general framework for other types of features for different kinds of pattern recognition problems.



---

*Source: 4940338-2021-11-10.xml*