# Research on Both the Classification and Quality Control Methods of the Car Seat Backrest Based on Machine Vision

Authors: Shengnan Sun; Jipeng Huang; Juan Zhu; Yan Yu; Luyao Zheng
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

In order to solve the problems of slow manual inspection speed and low fault detection accuracy of car seat back parts, this article using Q company’s car seat back parts researches and designs a car seat back classification and quality inspection screening system. Firstly, SURF (speeded up robust features) is combined with the CNN (convolutional neural network) to classify three types of car seat backrests: A, B, and C. Then, to establish the spring hook angle detection model of the car seat back to detect the misfitting and omission of the Class A car seat back springs, experimental results showed that the neural network-based car seat back detection method proposed in this paper had a feature point mismatch rate, which is less than 1.5% in the classification and recognition of car seat backs. The recognition rate of the training sample was 100% and that of the test sample was 99.56%. The accuracy rate of detection when inspecting 50 car seat backrests reached 98%, and the test results showed that the system can effectively reduce labor costs and improve the detection efficiency of auto parts.


---

## Body

## 1. Introduction

Car seat backrest is one of the main components of a car [1]. With the progress of science and technology, the discussion on the safety of car parts is becoming more and more a subject of debate. The structure of car seat backrest is one of key factors affecting the safety and comfort of the car [2, 3]. Before leaving the factory, it is very important to inspect parts of the car seats, such as back curtains and springs of car seat backrests. The traditional manual detection operation takes a long time and consumes a lot of manpower, which leads to human eye fatigue and low detection accuracy [4]. In industrial production, machine vision technology can replace human eyes to detect whether car seat backrests are adequate [5, 6]. Machine vision technology can not only deal with some situations that cannot be handled by manual operations but also greatly improves both the efficiency and accuracy of detection. In 2011, Xu et al. [7] proposed a real-time 3D shape detection system for car parts based on structured light mode. In order to meet the needs of online measurement, a single-lens mode using monochromatic light was proposed to avoid the influence of ambient light and the reflection characteristics of the measured part. In 2016, Chen et al. [8] studied the detection system for automobile oil support rods of using machine vision, which has been put into the market. In 2017, Huang and Chen [9] adopted a pulse coupled neural network (PCNN) for edge detection and recognition, which preliminarily proposed the automatic recognition and detection system of machine vision, and carried out detection and recognition experiments on small parts such as relay covers. In 2018, Praveenkumar et al. [10] proposed an online monitoring system for car transmission failure based on pattern recognition. In 2019, Du et al. [11] carried out an improved study on the detection method of the X-ray imaging of automobile casted aluminum parts; they proposed a defect detection system based on X-ray deep learning.To date, detection technology based on deep learning has been widely used in many fields. There is also a lot of research in the automotive industry, especially in the inspection of automotive parts [12]. But there have been few reports on the quality inspection of the car seat backrest. This paper studies the detection algorithm of car seat backrest parts, within the practical environment of Q Company. An automatic detection system for the car seat backrest based on neural network was proposed. It combines the SURF (speeded up robust features) and neural network algorithm to classify A, B, and C types of car seat backrests. The angle detection model of spring hooks for the car seat backrest was established. The classification of the car seat backrest and the function of qualified inspection for the problems of mismatching, misfitting, and missing of Q Company’s Class A car seat backrest were categorized.


## 2. Car Seat Backrest Modeling and Testing System Composition

### 2.1. System Composition and Workflow

Based on CCD imaging technology, Intelligent light source was selected for the classification and quality detection system of the car seat backrest to minimize the image pollution caused by the influence of illumination. At the same time, appropriate visual processing methods and an executive device were adopted to realize the system. Under certain light conditions, the system collects the image information of the surface of the car seat backrest through a CCD industrial camera, which converts the acquired analog image information into digital image information and saves it onto a computer. In the computer, the digital image information is processed and analyzed by visual processing software, so as to judge whether the shape and structure of the product meet the requirements, and whether the product is misinstalled or missing. If the test target is qualified, the green indicator light will be on and the qualified product will be passed to the next station. Otherwise, the red indicator light will be on, and the unqualified products will be rejected by the sorting process. The diagram of the car seat backrest detection system is showed in Figure1.Figure 1
The car seat backrest detection system.The technical indexes of the light source system in this paper meet the supply voltage AC220V and the light source power 100 W. In order to ensure that the lighting area of the car seat backrest can reach the optimal effect possible, this study mainly opted for the backlight lighting method in the lighting mode and uses the high-power integrated 100 W LED projection light source to test three kinds of car seat backrest products with different shapes and types. Haikang mv-ca060-10gc was selected to collect images; the camera resolution is 5 megapixels. In order to optimize the system design and obtain good image effect, the Tenglong 22HA industrial lens with a 50-degree field angle was selected for this study.


### 2.2. Establishment and Analysis of the Mathematical Model of Key Components

#### 2.2.1. The Establishment of the Mathematical Model of Car Seat Backrest

In this paper, the600mm×460mm car seat backrest of Q Company was used as the main research object. The schematic diagram of a Class A car seat backrest is shown in Figure 2. The length of the midline that plays a major supporting role is 370 mm, and there are four springs on each of the left and right sides. The corresponding superposition length and number of spring wire loops are as follows: 37mm×2, 47mm×2, 58mm×2, and 59mm×2. The outer diameter and number of springs are as follows: 10mm×2, 15mm×2, 10mm×2, and 20mm×2.Figure 2
Class A car seat backrest.In order to facilitate the specific mathematical analysis, this study established the corresponding mathematical model of the car seat backrest. The two-dimensional mathematical model of the Class A car seat backrest is shown in Figure3. Whether the backrest of Class A is to standard or not is judged by spring hook whether the hook is on the side of the slot. Therefore, an important step is to accurately locate the side slot which is hooked by the spring of the car seat backrest. First of all, the actual measured value of the car seat backrest midline is 370 mm. Then, according to the distance between each pair of side slot positions and the midline positions, the 8 side slot positions are located and their locations were marked. Finally, the measured values were input into the computer accordingly, and the image information was converted into digital information, so as to achieve the purpose of detection.Figure 3
Model of the Class A car seat backrest.


#### 2.2.2. The Establishment of Angle Model of Spring Hooks

It is the qualified standard of the car seat back spring hook, that the spring hook is connected to the side card slot. Whether the spring is to standard can be judged by detecting the angle between the center line (or parallel line) where the spring is located and the line where the side slot is located (it is parallel to the midline in Figure3). As shown in Figure 4, the line segment AB in the figure represents the spring, and the qualified angle range is between α1 and α2. Actually α1 is 15° and α2 is 75°. Figure 4(b) shows that the spring hook is within the qualified range; Figures 4(c) and 4(d) indicate that the spring hook is not qualified.Figure 4
Detection model of the angle.
(a)(b)(c)(d)In the experiment, Equation (1) is used to calculate α. If α is between α1 and α2, it means that the spring hook has been hooked correctly to the side slot. In the seat transmission process, the position of each spring hook on the seat is successively detected. After the angle detection, if the 8 spring hooks are all hooked in the side card slots, then the angles are between 15° and 75°, and then, the seat backrest is a to standard product.
(1)tanα=k2−k11−k1k2,wherek1 and k2 are the slopes of the two detection lines AB1 and AB2, respectively.


## 2.1. System Composition and Workflow

Based on CCD imaging technology, Intelligent light source was selected for the classification and quality detection system of the car seat backrest to minimize the image pollution caused by the influence of illumination. At the same time, appropriate visual processing methods and an executive device were adopted to realize the system. Under certain light conditions, the system collects the image information of the surface of the car seat backrest through a CCD industrial camera, which converts the acquired analog image information into digital image information and saves it onto a computer. In the computer, the digital image information is processed and analyzed by visual processing software, so as to judge whether the shape and structure of the product meet the requirements, and whether the product is misinstalled or missing. If the test target is qualified, the green indicator light will be on and the qualified product will be passed to the next station. Otherwise, the red indicator light will be on, and the unqualified products will be rejected by the sorting process. The diagram of the car seat backrest detection system is showed in Figure1.Figure 1
The car seat backrest detection system.The technical indexes of the light source system in this paper meet the supply voltage AC220V and the light source power 100 W. In order to ensure that the lighting area of the car seat backrest can reach the optimal effect possible, this study mainly opted for the backlight lighting method in the lighting mode and uses the high-power integrated 100 W LED projection light source to test three kinds of car seat backrest products with different shapes and types. Haikang mv-ca060-10gc was selected to collect images; the camera resolution is 5 megapixels. In order to optimize the system design and obtain good image effect, the Tenglong 22HA industrial lens with a 50-degree field angle was selected for this study.


## 2.2. Establishment and Analysis of the Mathematical Model of Key Components

### 2.2.1. The Establishment of the Mathematical Model of Car Seat Backrest

In this paper, the600mm×460mm car seat backrest of Q Company was used as the main research object. The schematic diagram of a Class A car seat backrest is shown in Figure 2. The length of the midline that plays a major supporting role is 370 mm, and there are four springs on each of the left and right sides. The corresponding superposition length and number of spring wire loops are as follows: 37mm×2, 47mm×2, 58mm×2, and 59mm×2. The outer diameter and number of springs are as follows: 10mm×2, 15mm×2, 10mm×2, and 20mm×2.Figure 2
Class A car seat backrest.In order to facilitate the specific mathematical analysis, this study established the corresponding mathematical model of the car seat backrest. The two-dimensional mathematical model of the Class A car seat backrest is shown in Figure3. Whether the backrest of Class A is to standard or not is judged by spring hook whether the hook is on the side of the slot. Therefore, an important step is to accurately locate the side slot which is hooked by the spring of the car seat backrest. First of all, the actual measured value of the car seat backrest midline is 370 mm. Then, according to the distance between each pair of side slot positions and the midline positions, the 8 side slot positions are located and their locations were marked. Finally, the measured values were input into the computer accordingly, and the image information was converted into digital information, so as to achieve the purpose of detection.Figure 3
Model of the Class A car seat backrest.


### 2.2.2. The Establishment of Angle Model of Spring Hooks

It is the qualified standard of the car seat back spring hook, that the spring hook is connected to the side card slot. Whether the spring is to standard can be judged by detecting the angle between the center line (or parallel line) where the spring is located and the line where the side slot is located (it is parallel to the midline in Figure3). As shown in Figure 4, the line segment AB in the figure represents the spring, and the qualified angle range is between α1 and α2. Actually α1 is 15° and α2 is 75°. Figure 4(b) shows that the spring hook is within the qualified range; Figures 4(c) and 4(d) indicate that the spring hook is not qualified.Figure 4
Detection model of the angle.
(a)(b)(c)(d)In the experiment, Equation (1) is used to calculate α. If α is between α1 and α2, it means that the spring hook has been hooked correctly to the side slot. In the seat transmission process, the position of each spring hook on the seat is successively detected. After the angle detection, if the 8 spring hooks are all hooked in the side card slots, then the angles are between 15° and 75°, and then, the seat backrest is a to standard product.
(1)tanα=k2−k11−k1k2,wherek1 and k2 are the slopes of the two detection lines AB1 and AB2, respectively.


## 2.2.1. The Establishment of the Mathematical Model of Car Seat Backrest

In this paper, the600mm×460mm car seat backrest of Q Company was used as the main research object. The schematic diagram of a Class A car seat backrest is shown in Figure 2. The length of the midline that plays a major supporting role is 370 mm, and there are four springs on each of the left and right sides. The corresponding superposition length and number of spring wire loops are as follows: 37mm×2, 47mm×2, 58mm×2, and 59mm×2. The outer diameter and number of springs are as follows: 10mm×2, 15mm×2, 10mm×2, and 20mm×2.Figure 2
Class A car seat backrest.In order to facilitate the specific mathematical analysis, this study established the corresponding mathematical model of the car seat backrest. The two-dimensional mathematical model of the Class A car seat backrest is shown in Figure3. Whether the backrest of Class A is to standard or not is judged by spring hook whether the hook is on the side of the slot. Therefore, an important step is to accurately locate the side slot which is hooked by the spring of the car seat backrest. First of all, the actual measured value of the car seat backrest midline is 370 mm. Then, according to the distance between each pair of side slot positions and the midline positions, the 8 side slot positions are located and their locations were marked. Finally, the measured values were input into the computer accordingly, and the image information was converted into digital information, so as to achieve the purpose of detection.Figure 3
Model of the Class A car seat backrest.


## 2.2.2. The Establishment of Angle Model of Spring Hooks

It is the qualified standard of the car seat back spring hook, that the spring hook is connected to the side card slot. Whether the spring is to standard can be judged by detecting the angle between the center line (or parallel line) where the spring is located and the line where the side slot is located (it is parallel to the midline in Figure3). As shown in Figure 4, the line segment AB in the figure represents the spring, and the qualified angle range is between α1 and α2. Actually α1 is 15° and α2 is 75°. Figure 4(b) shows that the spring hook is within the qualified range; Figures 4(c) and 4(d) indicate that the spring hook is not qualified.Figure 4
Detection model of the angle.
(a)(b)(c)(d)In the experiment, Equation (1) is used to calculate α. If α is between α1 and α2, it means that the spring hook has been hooked correctly to the side slot. In the seat transmission process, the position of each spring hook on the seat is successively detected. After the angle detection, if the 8 spring hooks are all hooked in the side card slots, then the angles are between 15° and 75°, and then, the seat backrest is a to standard product.
(1)tanα=k2−k11−k1k2,wherek1 and k2 are the slopes of the two detection lines AB1 and AB2, respectively.


## 3. Test of the Car Seat Backrest

The core algorithms of car seat backrest detection system are the classification algorithm and the quality detection algorithm of the Class A car seat backrest. Four aspects are described in this paper: the image preprocessing, the car seat backrest classification algorithm, the edge feature detection algorithm of the Class A car seat backrest curtain, and the spring hook detection algorithm. The specific algorithm flow is shown in Figure5.Figure 5
Flow chart of the algorithm.


### 3.1. Image Preprocessing

#### 3.1.1. Gray Transformation

In order to improve the computing speed, the color images are converted into grayscale. The image graying mode is determined according to(2)Vgray=ωrR+ωgG+ωbB,whereωr, ωg, and ωb are the weights of R, G, and B, respectively.A large number of experimental research data have proved that the obtained grayscale images are the most reasonable whenωr=−1.0, ωg=−1.0, and ωb=2.0. In other words, the 2B-G-R index is better than other indexes (R-2B-G and R+G-2B), and the contrast effect between the contour and the background of the conveyor belt is the most significant. The grayscale conversion results under different indexes are shown in Figure 6.Figure 6
Grayscale conversion graph under different indexes.
(a)
2B-G-R(b)
R-2B-G(c)
R+G-2B


#### 3.1.2. Filtering

In order to reduce the ambient noise and the noise interference caused by the camera’s dark current, gray images of the car seat backrest must be filtered to reduce the noise. Combined with the actual image acquisition situation in this study, Gaussian filtering in the7×7 neighborhood region was selected to denoise the image of the car seat backrests.


### 3.2. Image Classification Detection

Regarding the classification of the car seat backrests, this belongs to the application of computer vision [13, 14]. The purpose was to classify the three kinds of car seat backrests and prepare for the following inspection to determine whether the car seat backrest of Class A was to standard. In this study, the steps of car seat backrest classification were as follows.The first step was to obtain the image of the car seat backrest. In this paper, a large number of images were obtained by CCD camera, including the images of car seat backrests under different deflection angles, so as to establish a database of car seat backrests.The second step was to preprocess the image of car seat backrests and establish the corresponding data set for the processed image of the car seat backrests. The image data set was further amplified, by obtaining new image data through physical rotation and scaling.The third step was to extract the feature values of the processed images and to identify and classify the images by combining the SURF algorithm with the convolutional neural network.Three types of car seat backrest of Q Company were tested. As shown in Figure7, there are three types of car seat backrests.Figure 7
Three types of car seat backrests.
(a)
Class A(b)
Class B(c)
Class CAs shown in Figure8, SIFT and SURF algorithms are used to detect feature points. Figures 8(a)–8(c) show the effect diagrams of the SIFT algorithm matching and detecting the feature points of the Class A vehicle seat backrest, where the left part of both Figures 8(b) and 8(c) are rotated by 90 degrees and 180 degrees of left part of Figure 8(a) in the training set. The average number of feature matching points is 64, the mismatching rate is ≤7.2%, and the detection time is 102 ms. Figures 8(d)–8(f) are the effect diagrams of the SURF algorithm matching the feature points of the Class A vehicle seat backrest, in which the left part of both Figures 8(e) and 8(f) are rotated by 90 degrees and 180 degrees of the left part of Figure 8(d) in the training set. The average number of feature matching points detected is 1372, the mismatching rate≤1.46%, and the detection time is 1 ms. By comparison, it was found that the SURF algorithm had about 21.44 times the detection feature points of the SIFT algorithm, and the algorithm had a short running time and good antirotation performance. Considering the actual requirements of the industry, the car seat backrests need to be tested on a conveyor belt; therefore, the SURF algorithm was selected for image matching.Figure 8
The matching result of feature point detection.
(a)(b)


### 3.3. Combination of SURF Algorithm and Convolutional Neural Network

In the traditional convolutional neural network [15, 16] classifier, the first column is the input layer, the second and third columns are the hidden layers, and the last column is the output layer. In this study, the classification of convolutional neural networks was carried out based on the previous convolutional neural networks. Firstly, the tested images and template images were matched by the SURF algorithm to optimize their antirotation and translation performance. The output images were trained in the convolutional neural network. Secondly, the number of neurons at the end of the output layer was changed and the Softmax operation was added to meet the classification requirements. The number of neurons at the end of the output layer was consistent with the number of car seat backrest images to be classified in this study. The structural schematic diagram adopted in this study is shown in Figure 9.Figure 9
The structural schematic diagram.In order to choose the best network structure, the SURF algorithm with VGG, the AlexNet and MobileNet network structures are, respectively, combined to compare the recognition rate on the basis of the self-built data set. The results of the network identification are shown in Table1. Experimental data shows that the combination of the SURF and VGG networks has the best recognition effect. Therefore, the VGG network was selected as the network structure for this study.Table 1
Recognition rate results.
No.Identification methodRecognition rate (%)1SURF+VGG99.562SURF+AlexNet98.263SURF+MobileNet97.27In the VGG network structure, the core structure is the convolution layer and pooling layer. In view of the actual requirements, in the convolution layer, the convolution kernel with a size of3×3 is used for the convolution operation to extract the complete features effectively with the minimum window. In the pooling layer, the pooling window with a step size of 2 and size of 2×2 is selected for pooling operation. Figure 10 is a schematic diagram of the pooling operation. Since the texture and contour characteristics of the target object were both obvious in this study, the maximum pooling was selected in this study.Figure 10
Schematic of pooling operation.In this study, the final output of the network is the classification result. However, in the convolutional layer and pooling layer, the image features are represented in the form of two-dimensional data, so the two-dimensional data should be converted into one-dimensional data in this process. In the last layer, the Softmax operation was added to make the output the probability of classification. According to the actual requirements of this study, the final output was set asS=S1,S2,S3. The Softmax formula is given by:
(3)Si=eai∑k=13eaki=1,2,3.In the three classification tasks, the sample label was set to the vector classification form,y=a,b,c, where a, b, and c are set to 0 or 1. In vector y, only the index position of the corresponding category is 1, and the rest is 0. For example, y=0,1,0 represents a sample label for category 2. At this point, the cost function of Softmax is expressed as
(4)L=−∑i=13yilogsi.By establishing the cost function of Softmax, it can be concluded that the loss of prediction when it is correct is smaller than that when it is wrong. For example, when the number of categories is 3, category 2 is the correct classification result. When the result output isP=0.1,0.7,0.2, the loss value is −log0.7. When the result output is P=0.7,0.1,0.2, the corresponding loss value is −log0.1. When the output of the result is P=0.1,0.2,0.7, the corresponding loss value is −log0.2. Therefore, the above conclusion was verified-the loss of prediction on the right time is smaller than the loss of prediction on the wrong time. By constructing this type of Softmax cost function, the model can predict the optimization goal more accurately.


## 3.1. Image Preprocessing

### 3.1.1. Gray Transformation

In order to improve the computing speed, the color images are converted into grayscale. The image graying mode is determined according to(2)Vgray=ωrR+ωgG+ωbB,whereωr, ωg, and ωb are the weights of R, G, and B, respectively.A large number of experimental research data have proved that the obtained grayscale images are the most reasonable whenωr=−1.0, ωg=−1.0, and ωb=2.0. In other words, the 2B-G-R index is better than other indexes (R-2B-G and R+G-2B), and the contrast effect between the contour and the background of the conveyor belt is the most significant. The grayscale conversion results under different indexes are shown in Figure 6.Figure 6
Grayscale conversion graph under different indexes.
(a)
2B-G-R(b)
R-2B-G(c)
R+G-2B


### 3.1.2. Filtering

In order to reduce the ambient noise and the noise interference caused by the camera’s dark current, gray images of the car seat backrest must be filtered to reduce the noise. Combined with the actual image acquisition situation in this study, Gaussian filtering in the7×7 neighborhood region was selected to denoise the image of the car seat backrests.


## 3.1.1. Gray Transformation

In order to improve the computing speed, the color images are converted into grayscale. The image graying mode is determined according to(2)Vgray=ωrR+ωgG+ωbB,whereωr, ωg, and ωb are the weights of R, G, and B, respectively.A large number of experimental research data have proved that the obtained grayscale images are the most reasonable whenωr=−1.0, ωg=−1.0, and ωb=2.0. In other words, the 2B-G-R index is better than other indexes (R-2B-G and R+G-2B), and the contrast effect between the contour and the background of the conveyor belt is the most significant. The grayscale conversion results under different indexes are shown in Figure 6.Figure 6
Grayscale conversion graph under different indexes.
(a)
2B-G-R(b)
R-2B-G(c)
R+G-2B


## 3.1.2. Filtering

In order to reduce the ambient noise and the noise interference caused by the camera’s dark current, gray images of the car seat backrest must be filtered to reduce the noise. Combined with the actual image acquisition situation in this study, Gaussian filtering in the7×7 neighborhood region was selected to denoise the image of the car seat backrests.


## 3.2. Image Classification Detection

Regarding the classification of the car seat backrests, this belongs to the application of computer vision [13, 14]. The purpose was to classify the three kinds of car seat backrests and prepare for the following inspection to determine whether the car seat backrest of Class A was to standard. In this study, the steps of car seat backrest classification were as follows.The first step was to obtain the image of the car seat backrest. In this paper, a large number of images were obtained by CCD camera, including the images of car seat backrests under different deflection angles, so as to establish a database of car seat backrests.The second step was to preprocess the image of car seat backrests and establish the corresponding data set for the processed image of the car seat backrests. The image data set was further amplified, by obtaining new image data through physical rotation and scaling.The third step was to extract the feature values of the processed images and to identify and classify the images by combining the SURF algorithm with the convolutional neural network.Three types of car seat backrest of Q Company were tested. As shown in Figure7, there are three types of car seat backrests.Figure 7
Three types of car seat backrests.
(a)
Class A(b)
Class B(c)
Class CAs shown in Figure8, SIFT and SURF algorithms are used to detect feature points. Figures 8(a)–8(c) show the effect diagrams of the SIFT algorithm matching and detecting the feature points of the Class A vehicle seat backrest, where the left part of both Figures 8(b) and 8(c) are rotated by 90 degrees and 180 degrees of left part of Figure 8(a) in the training set. The average number of feature matching points is 64, the mismatching rate is ≤7.2%, and the detection time is 102 ms. Figures 8(d)–8(f) are the effect diagrams of the SURF algorithm matching the feature points of the Class A vehicle seat backrest, in which the left part of both Figures 8(e) and 8(f) are rotated by 90 degrees and 180 degrees of the left part of Figure 8(d) in the training set. The average number of feature matching points detected is 1372, the mismatching rate≤1.46%, and the detection time is 1 ms. By comparison, it was found that the SURF algorithm had about 21.44 times the detection feature points of the SIFT algorithm, and the algorithm had a short running time and good antirotation performance. Considering the actual requirements of the industry, the car seat backrests need to be tested on a conveyor belt; therefore, the SURF algorithm was selected for image matching.Figure 8
The matching result of feature point detection.
(a)(b)


## 3.3. Combination of SURF Algorithm and Convolutional Neural Network

In the traditional convolutional neural network [15, 16] classifier, the first column is the input layer, the second and third columns are the hidden layers, and the last column is the output layer. In this study, the classification of convolutional neural networks was carried out based on the previous convolutional neural networks. Firstly, the tested images and template images were matched by the SURF algorithm to optimize their antirotation and translation performance. The output images were trained in the convolutional neural network. Secondly, the number of neurons at the end of the output layer was changed and the Softmax operation was added to meet the classification requirements. The number of neurons at the end of the output layer was consistent with the number of car seat backrest images to be classified in this study. The structural schematic diagram adopted in this study is shown in Figure 9.Figure 9
The structural schematic diagram.In order to choose the best network structure, the SURF algorithm with VGG, the AlexNet and MobileNet network structures are, respectively, combined to compare the recognition rate on the basis of the self-built data set. The results of the network identification are shown in Table1. Experimental data shows that the combination of the SURF and VGG networks has the best recognition effect. Therefore, the VGG network was selected as the network structure for this study.Table 1
Recognition rate results.
No.Identification methodRecognition rate (%)1SURF+VGG99.562SURF+AlexNet98.263SURF+MobileNet97.27In the VGG network structure, the core structure is the convolution layer and pooling layer. In view of the actual requirements, in the convolution layer, the convolution kernel with a size of3×3 is used for the convolution operation to extract the complete features effectively with the minimum window. In the pooling layer, the pooling window with a step size of 2 and size of 2×2 is selected for pooling operation. Figure 10 is a schematic diagram of the pooling operation. Since the texture and contour characteristics of the target object were both obvious in this study, the maximum pooling was selected in this study.Figure 10
Schematic of pooling operation.In this study, the final output of the network is the classification result. However, in the convolutional layer and pooling layer, the image features are represented in the form of two-dimensional data, so the two-dimensional data should be converted into one-dimensional data in this process. In the last layer, the Softmax operation was added to make the output the probability of classification. According to the actual requirements of this study, the final output was set asS=S1,S2,S3. The Softmax formula is given by:
(3)Si=eai∑k=13eaki=1,2,3.In the three classification tasks, the sample label was set to the vector classification form,y=a,b,c, where a, b, and c are set to 0 or 1. In vector y, only the index position of the corresponding category is 1, and the rest is 0. For example, y=0,1,0 represents a sample label for category 2. At this point, the cost function of Softmax is expressed as
(4)L=−∑i=13yilogsi.By establishing the cost function of Softmax, it can be concluded that the loss of prediction when it is correct is smaller than that when it is wrong. For example, when the number of categories is 3, category 2 is the correct classification result. When the result output isP=0.1,0.7,0.2, the loss value is −log0.7. When the result output is P=0.7,0.1,0.2, the corresponding loss value is −log0.1. When the output of the result is P=0.1,0.2,0.7, the corresponding loss value is −log0.2. Therefore, the above conclusion was verified-the loss of prediction on the right time is smaller than the loss of prediction on the wrong time. By constructing this type of Softmax cost function, the model can predict the optimization goal more accurately.


## 4. Experimental Results and Analysis

This experiment was conducted in a computer environment consisting of an Intel i5-2400 CPU, 8 GB memory, and GTX 1050 Ti graphics card containing GPU. For car seat backrest detection system, Visual Studio2017 was selected as the software development environment to develop machine vision software. The C++ software development language was selected. Many kinds of machine vision algorithms in OpenCV were reasonably utilized, and the program was custom written to achieve the required functions necessary for this study.


### 4.1. Experiment Results and Analysis of Car Seat Backrest Classification

In the experiment, 50 workpieces were randomly selected from three kinds of car seat backrests for detection, the minimum logarithm of the matching points of the SIFT algorithm was 61, the mean value was 67, the average error matching rate was about 8.3%, and the standard variance was about3.14×10−3. The minimum logarithm of matching points in the SURF algorithm was 1367, the mean value was 1378, the average error matching rate was about 1.47%, and the standard variance was about 8.52×10−4. Considering the actual situation, the SURF algorithm was selected by this study’s researchers for best feature point matching.Because only three kinds of car seat backrests were identified and utilized, good detection results could be achieved when the convolution neural network had a small number of iterations. The recognition samples of the self-built data set were used as the input images of the convolutional neural network, and the unified size of the images was128×128, a total of 1500 images. Taking 80% of the whole data set as the training sample, the optimal network parameter configuration could be obtained when network parameters were constantly updated. The remaining 20% of the data was taken as the test sample to detect the network recognition effect, so as to obtain the recognition accuracy of the convolutional neural network. As shown in Figure 11, the learning rate size was set to 0.001, the batch block size was set to 32, and the number of iterations was 100.Figure 11
Change of recognition rate and loss curve of the training and test samples.
(a)
Recognition rate curve(b)
Loss functionAs can be seen from the change of recognition rate curve in Figure11, the recognition accuracy of the training sample (blue) reaches 100% and that of the test sample (red) was 99.56%. As the number of iterations increases, the loss decline curve of the sample quickly approaches 0, indicating that the model converges quickly.


### 4.2. Spring Hook Test

In the experiment, 20 Class A seat backrest workpieces were detected and the data were recorded. The test results of the spring hook are shown in Table2, among which No. 7, No. 12, and No. 17 are the Class A car seat backrests with misfitted or missing fittings.Table 2
Spring hook test results.
No.Eight regional locationsGround truth (15-75)Experimental abnormal angle valueCorrect number of test resultsActual resultExperimental resultQuantity of qualified positionsNumber of nonconforming locationsQuantity of qualified positionsNumber of nonconforming locationsNo. 58071—80.887No. 77171—10.688No. 127171—79.128No. 177180——7No. 208080——8If the components of car seat backrests are actually not installed to standard (that is, they have been misfitted or are missing), the test result was unqualified. If the visual interface showed “yes” in the item of “whether the seat backrest is qualified”, then the test was passed and standards met. On the contrary, it indicates the detection of an error in manufacture. For example in example No. 12 of test results of eight car seat springs (location 1 ~ 8, respectively), the position 1 ~ 7 placings of the seven spring hanger hook card slots and the angle at the upper were in the qualified range; however, the location of the eighth spring hanger hook to lateral card slot angle was below the scope of qualified, so No. 12 was actually an unqualified product. Through the system test, the experiment showed an accurate result by detecting the exact real world correctly fitted and incorrectly fitted components. In example test No. 17, the locations 1 ~ 8 points of the eight placings of the spring hanger hook card slots and the angle at the upper were in the qualified range, but after the system test, a visual inspection showed that one of the locations was not within the scope of qualified, therefore the results did not tally with the actual situation, namely, test results were only correct in 7 out of 8 cases. In the experiment, out of a total of 160 spring hook positions of 20 seat backrests, 158 positions were detected correctly, which meant that the detection accuracy of the spring hook positions of the final car seat backrests was 98.75%.


### 4.3. Experiment Results and Analysis of Class A Car Seat Backrest Quality Test

During the process of low-speed movement (1 m/s), fixed-point detection was carried out on the car seat backrest, and the detection accuracy of the system was 0.04 mm. In the experiment, 50 car seat backrests were tested, and the results are shown in Table3. When the detection accuracy of the system is 0.04 mm, the attributes of the car seat backrests can be accurately identified and judged, and the entire automatic detection and screening process can be completed within 3 s. For example, it can be seen from Table 3 that the real property of No. 1 is qualified, while the test property is qualified, so the detection is correct. For No. 9, the real property is qualified and the test property is unqualified, so the test is wrong. The final 50 car seat backrest quality detection accuracy was 98%.Table 3
Test data of 50 car seat backrests.
No.Real propertyDetection time (s)Test propertyRed light/green lightTest resultsNo. 1Qualified2.9QualifiedGreen lightYesNo. 2Qualified2.7QualifiedGreen lightYesNo. 3Qualified2.1QualifiedGreen lightYesNo. 4Qualified3.0QualifiedGreen lightYesNo. 5Qualified2.9QualifiedGreen lightYesNo. 6No qualified2.2No qualifiedRed lightYesNo. 7Qualified2.5QualifiedGreen lightYesNo. 8Qualified2.5QualifiedGreen lightYesNo. 9Qualified2.8No qualifiedRed lightNoNo.10Qualified3.0QualifiedGreen lightYes..................No.50Qualified2.7QualifiedGreen lightYes


## 4.1. Experiment Results and Analysis of Car Seat Backrest Classification

In the experiment, 50 workpieces were randomly selected from three kinds of car seat backrests for detection, the minimum logarithm of the matching points of the SIFT algorithm was 61, the mean value was 67, the average error matching rate was about 8.3%, and the standard variance was about3.14×10−3. The minimum logarithm of matching points in the SURF algorithm was 1367, the mean value was 1378, the average error matching rate was about 1.47%, and the standard variance was about 8.52×10−4. Considering the actual situation, the SURF algorithm was selected by this study’s researchers for best feature point matching.Because only three kinds of car seat backrests were identified and utilized, good detection results could be achieved when the convolution neural network had a small number of iterations. The recognition samples of the self-built data set were used as the input images of the convolutional neural network, and the unified size of the images was128×128, a total of 1500 images. Taking 80% of the whole data set as the training sample, the optimal network parameter configuration could be obtained when network parameters were constantly updated. The remaining 20% of the data was taken as the test sample to detect the network recognition effect, so as to obtain the recognition accuracy of the convolutional neural network. As shown in Figure 11, the learning rate size was set to 0.001, the batch block size was set to 32, and the number of iterations was 100.Figure 11
Change of recognition rate and loss curve of the training and test samples.
(a)
Recognition rate curve(b)
Loss functionAs can be seen from the change of recognition rate curve in Figure11, the recognition accuracy of the training sample (blue) reaches 100% and that of the test sample (red) was 99.56%. As the number of iterations increases, the loss decline curve of the sample quickly approaches 0, indicating that the model converges quickly.


## 4.2. Spring Hook Test

In the experiment, 20 Class A seat backrest workpieces were detected and the data were recorded. The test results of the spring hook are shown in Table2, among which No. 7, No. 12, and No. 17 are the Class A car seat backrests with misfitted or missing fittings.Table 2
Spring hook test results.
No.Eight regional locationsGround truth (15-75)Experimental abnormal angle valueCorrect number of test resultsActual resultExperimental resultQuantity of qualified positionsNumber of nonconforming locationsQuantity of qualified positionsNumber of nonconforming locationsNo. 58071—80.887No. 77171—10.688No. 127171—79.128No. 177180——7No. 208080——8If the components of car seat backrests are actually not installed to standard (that is, they have been misfitted or are missing), the test result was unqualified. If the visual interface showed “yes” in the item of “whether the seat backrest is qualified”, then the test was passed and standards met. On the contrary, it indicates the detection of an error in manufacture. For example in example No. 12 of test results of eight car seat springs (location 1 ~ 8, respectively), the position 1 ~ 7 placings of the seven spring hanger hook card slots and the angle at the upper were in the qualified range; however, the location of the eighth spring hanger hook to lateral card slot angle was below the scope of qualified, so No. 12 was actually an unqualified product. Through the system test, the experiment showed an accurate result by detecting the exact real world correctly fitted and incorrectly fitted components. In example test No. 17, the locations 1 ~ 8 points of the eight placings of the spring hanger hook card slots and the angle at the upper were in the qualified range, but after the system test, a visual inspection showed that one of the locations was not within the scope of qualified, therefore the results did not tally with the actual situation, namely, test results were only correct in 7 out of 8 cases. In the experiment, out of a total of 160 spring hook positions of 20 seat backrests, 158 positions were detected correctly, which meant that the detection accuracy of the spring hook positions of the final car seat backrests was 98.75%.


## 4.3. Experiment Results and Analysis of Class A Car Seat Backrest Quality Test

During the process of low-speed movement (1 m/s), fixed-point detection was carried out on the car seat backrest, and the detection accuracy of the system was 0.04 mm. In the experiment, 50 car seat backrests were tested, and the results are shown in Table3. When the detection accuracy of the system is 0.04 mm, the attributes of the car seat backrests can be accurately identified and judged, and the entire automatic detection and screening process can be completed within 3 s. For example, it can be seen from Table 3 that the real property of No. 1 is qualified, while the test property is qualified, so the detection is correct. For No. 9, the real property is qualified and the test property is unqualified, so the test is wrong. The final 50 car seat backrest quality detection accuracy was 98%.Table 3
Test data of 50 car seat backrests.
No.Real propertyDetection time (s)Test propertyRed light/green lightTest resultsNo. 1Qualified2.9QualifiedGreen lightYesNo. 2Qualified2.7QualifiedGreen lightYesNo. 3Qualified2.1QualifiedGreen lightYesNo. 4Qualified3.0QualifiedGreen lightYesNo. 5Qualified2.9QualifiedGreen lightYesNo. 6No qualified2.2No qualifiedRed lightYesNo. 7Qualified2.5QualifiedGreen lightYesNo. 8Qualified2.5QualifiedGreen lightYesNo. 9Qualified2.8No qualifiedRed lightNoNo.10Qualified3.0QualifiedGreen lightYes..................No.50Qualified2.7QualifiedGreen lightYes


## 5. Conclusion

In this paper, the car seat backrest classification and quality inspection system based on machine vision is proposed. The experiment results showed that the system can meet the requirements of car seat backrest quality tests in both accuracy and time. From the perspective of classification, car seat backrests are classified by combining the SURF algorithm and the convolutional neural network. The mismatching rate of feature points is ≤1.5%, and the final classification recognition rate is 99.56%. From the perspective of the quality test of Class A car seat backrests, when the system test accuracy is 0.04 mm, the attributes of the car seat backrest can be identified and judged accurately, and the whole automatic test and screening process can be completed within 3 s, with a test accuracy of 98%. Therefore, the car seat backrest classification and quality detection system proposed in this paper can meet the needs of modern industrial production and can be used to carry out qualified safety testing of car seat backrests quickly and accurately. The current system based on computer design is too large; in the future, embedded systems will enable equipment to further miniaturization, and it meets requirements of future intelligent machines.



---

*Source: 3106313-2022-03-15.xml*