# A Neural Network Model for Quality Prediction in the Automotive Industry

Anders Risan, Mohamed Kais Msakni, Peter Schütz

► To cite this version:

Anders Risan, Mohamed Kais Msakni, Peter Schütz. A Neural Network Model for Quality Prediction in the Automotive Industry. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2021, Nantes, France. pp.567-575, 10.1007/978-3-030-85914-5_60 hal-03897888

HAL Id: hal-03897888

https://inria.hal.science/hal-03897888

Submitted on 14 Dec 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# A Neural Network Model for Quality Prediction in the Automotive Industry*

Anders Risan¹, Mohamed Kais Msakni¹[0000-0003-3060-6970], and Peter Schütz¹[0000-0002-9466-0354]

Department of Industrial Economics and Technology Management,
Norwegian University of Science and Technology, 7491 Trondheim, Norway
andris@stud.ntnu.no; {kais.msakni,peter.schutz}@ntnu.no

**Abstract.** In this paper, we present a machine learning application for the automotive industry. We study the use of neural networks to predict the location of milled holes in a bumper beam using historical measurement data. The overall goal of the study is to reduce the time needed for quality control procedures as the predictions can supplement manual control measurements. Our preliminary results indicate that the neural network can generally capture the production process variations, but underestimates larger deviations from the specified location.

**Keywords:** Neural network · Quality control · Machine Learning · Manufacturing.

## 1 Introduction

The fourth industrial revolution, commonly referred to as Industry 4.0, marks a new era in global manufacturing, causing a rapid transformation of systems of production, management, and governance [13]. Technological solutions and data processing methods such as the Internet of Things (IoT), Cloud Computing, and Big Data Manufacturing have emerged, allowing a higher degree of connectivity. Together with advances in Artificial Intelligence and new possibilities offered by Machine Learning algorithms, decision-makers can extract valuable information from data and distribute it throughout entire value chains [14]. The technological advances following Industry 4.0 also enable new approaches for quality control by converting the collected data into manufacturing intelligence. Some of these approaches include the ability to perform predictive analysis, real-time quality monitoring, and corrections [16,18].

In this paper, we study the application of a neural network to predict the location of machined holes in a bumper beam based on historical measurement data. The produced bumper beams are measured in fixed intervals to ensure that the produced products satisfy the quality requirements, i. e. that all measurement points are within tolerance limits. In case of discovering a part that

<footnote>* This work was supported by the Research Council of Norway as part of the LeanDigital research project.</footnote>
violates the tolerance limits, the entire production since the last measurement is
scrapped. The goal of using a neural network to predict the quality of the pro-
duced products is to address the trade-off between the frequency of carrying out
quality control procedures and the risk of waste production. High-fidelity qual-
ity predictions would increase the duration of the time interval between physical
control measurements, thus reducing downtime in the production system.

The remainder of this paper is organized as follows: In Section 2, we provide an overview over applications of machine learning and neural networks in quality prediction. We introduce crash management systems and the bumper beam we develop the neural network for in Section 3. Section 4 covers the implementation and the performance of the neural network model. We conclude in Section 5.

## 2 Quality Control and Machine Learning

The purpose of quality control is to ensure that the final product meets the specified quality requirements. This is crucial for the long-term success of any manufacturing company. Despite the complexity of the production system, multi-factor and non-linear interactions between different processes and/or products, a predictive model can be used to predict the end quality of products from the available input data [15]. This information can then be used to improve the production system, e. g. by early calibration of process parameters, resulting in reduced waste production or stabilization of the production process.

The significant amount of data collected from sensors and other monitoring tools has made it possible to apply machine learning in quality control, e. g. predictive maintenance, anomaly detection, monitoring of product quality and machine conditions, as well as process optimization [13,21]. The orientation towards machine learning is, among others, pushed by the limited capacity of classical statistic techniques against complex big data sets. Conversely, machine learning systems tend to improve their performance as the amount of available data grows, making machine learning an adequate tool for extracting valuable information from mass data [1].

Neural networks are among the most widely used models in machine learn-
ing. They have been used for a wide range of applications within quality con-
trol in manufacturing environments. Notable examples include, amongst others:
Karayel [7] highlights the importance for manufacture to predict the surface
roughness before machining with a CNC lathe. The prediction can be done us-
ing a feedforward multi-layered neural network where the input parameters are
cutting depth, cutting speed, and feed rate. The predicted surface roughness
is then fed into the control system to obtain the optimal cutting parameters.
Tsai et al. [17] develop a supervised feedforward neural network for real-time
quality control of the surface roughness in milling cutting operations. Mar-
tin et al. [11] propose to automate the quality control process in resistance spot
welding. A supervised feedforward neural network is developed to replace a hu-
man expert that estimates the quality level of spot welds from ultra sonic oscillo-
grams. In a similar context, Zhao et al. [22] show that the power signal includes
useful information to determine the welding nugget states. This information
is used by a regression model and feedforward neural network to determine the
welding qualities for a real-time detection system. Wang et al. [19] propose a con-
volutional neural network to automate the process of detecting defects in parts.
This quality control process is done by visual inspections and aims at replacing
human agents to improve the production efficiency. Wang et al. [20] extend the
model for geometrically complex products using cloud-based platform.

In semiconductor manufacturing, virtual metrology is used to avoid the ex-
tra cost of physical measurements of wafers. It consists of predicting the wafers
properties using machine parameters and sensor data. Due to the high dimen-
tionality of data and complexity of the process, regression models fail to provide
good predictions. Thus, machine learning-based methods have been proposed
to detect faulty wafers [8]. Maggipinto et al. [10] propose a deep learning-based
model, Jia et al. [6] use the group method of data handling-type polynomial
neural networks, and Puggini and McLoone [12] develop an extreme learning
machine (a particular type of neural networks) for virtual metrology.

The contribution of this paper is to propose a quality prediction model in the
automotive industry where products are subject to strict requirements. Unlike
most models of the literature, the proposed neural network does not analyze the
current product on a real-time basis for validation but instead learns from pre-
viously produced and measured products to predict the quality of the following
product. By doing so, the production process is smoothed as early adjustments
can be anticipated and scrap production is reduced.

3 Quality Prediction for a Bumper Beam

3.1 Crash Management Systems

The main purpose of a car’s crash management system is to reduce the impact
of a collision; the objective is changing dependent on speed: at low speeds (<15
km/h), the goal is to minimize the cost of repairing the damage. At speeds be-
tween 15 and 40 km/h, the main aim is to protect any pedestrians involved in
the collision, while the protection of the passengers inside the car is becoming
the first priority at higher speeds [4]. Due to its role as an important passive
safety system, the crash management system has to satisfy strict requirements
regarding deflection and crash stability, set out by legislation, insurance compa-
nies, and customers [2]. To ensure that the system has the capability to absorb
the maximum amount of crash energy, the different parts are subject to rigorous
quality control throughout the production process.

## 3.2 Description of the Bumper Beam

The front crash management system usually consists of a fascia covering a
bumper beam and crash boxes for energy absorbance connecting the bumper
beam to the structure of the car [4]. In this study, we focus on the production
of the bumper beam.
The bumper beam is produced through stretch forming an extruded aluminum profile before machining holes for mounting the crash boxes and cutting it to length. A total of 20 holes have to be milled, each of which has a narrow tolerance range regarding its location in the beam. In predefined intervals, a beam is taken out of production after the machining process for quality control. During this quality control, the curvature of the beam and the location of the holes are measured. If the measured values violate the specified tolerance limits, the beam is rejected. Figure 1 illustrates different measurement points for the curvature and milled holes. The location of the leftmost hole in Figure 1, hole HH20, is predicted by the neural network discussed in Section 4.

<figure><img src="image_3.png" /><figcaption>Fig. 1: Measurement points on the beam. The blue rectangles represent the different measurement points of the bend. Hole HH20 is the leftmost hole in the beam.</figcaption></figure>

Hole HH20 has been selected for analysis for two main reasons: First, it serves as a reference hole for three other holes. As such, any deviation from its dedicated location translates into deviations for the holes referencing HH20. Being able to predict the location of HH20 therefore allows predicting the locations of the other holes. Second, the measure of linear correlation between the holes' historical measurements using the Pearson correlation matrix indicates that the predictive potential of curvature measurements for the location of holes is largest for HH20. The analysis of historical measurement data also shows that the measurements of the Y-coordinate of hole HH20 are very stable. The measurements of the X-coordinate of the same hole, however, are more volatile and vary a lot over time. Thus, we focus our study on predicting the X-coordinate of the future locations of HH20.

We use the available historical measurement data for the entire cluster to train a neural network model for predicting future HH20 locations. The input data is formed by the different measurements that showed high correlations with HH20. The neural network is used for the approximation of the regression function of the available data, which in this case are homogeneous and continuous. The choice of neural networks is based on their successful application in various domains, especially in manufacturing environments (see section 2). One of the strengths of neural networks is their ability to handle multidimensional and multivariate data while providing high accuracy results [5,21], as well as to estimate nonlinear functions not known analytically [3].
# 4 Implementation and Results

## 4.1 Training, Validation and Test Set

The data available includes 1,255 measurement reports that were collected over three years at specific intervals. These time series data are then divided into two sets. The first 70% of the data forms the first set used for training. The last 30% of the data constitutes the second set used to validate the neural network model. K-fold cross-validation method is applied to reduce the variance in the validation score with respect to the size of the training and validation sets. This generates 878 samples in the training and validation set; a 5-fold split is used, yielding a validation set size of 175 samples. We standardize the input data as the performance of neural networks is sensitive to scale. The standardization of the data is performed on the training, and the same scaler is later used on the testing set. By doing so, there is no information leak between training and testing sets. Also, the performance of the neural network is not affected by the scale between the value of the measurements and the deviations from the nominal values.

## 4.2 Neural Network Implementation

The neural network is implemented in Python 3.8.6 using the multi-layer perceptron regressor provided by the Scikit-Learn library. We use a neural network with two hidden layers of 20 neurons each and use the default parameters of MLPRegressor of Scikit-Learn. The output layer includes one neuron and converts the flow of the network to the final output. For the weight optimization, however, we use the LBFGS solver, an optimizer in the family of quasi-Newton methods and recommended setting for small datasets for faster converge and better performance.

The goal of the neural network model is to predict the location of hole HH20 based on past bend measurements. We compare the results from three cases that use variations of the available input. The neural networks are trained for each case separately. Case 1 uses 1-lagged measurements, i. e. only the last set of measurements. In Case 2, the neural network is provided with 1-lagged and 2-lagged measurements, i. e. the last two sets of measurements. The last case, Case 3, is fed with 1-lagged, 2-lagged and 3-lagged input data.

## 4.3 Model performance

We estimate the neural networks' performance on the testing set by computing the Mean Absolute Error (MAE), the Mean Squared Error (MSE), the Root Mean Squared Error (RMSE), and the coefficient of determination $R^2$ for the different cases. The results are presented in Table 1.

We see from Table 1 that the prediction quality of the different cases is quite similar, albeit with slightly better performance for Case 3. However, the improvements in the different performance metrics are small, indicating that using more lagged data has limited value for this production process.
<table><caption>Table 1: Performance metrics of the neural network model.</caption><thead><tr><th></th><th>Case 1</th><th>Case 2</th><th>Case 3</th></tr></thead><tbody><tr><td>MAE</td><td>0.21</td><td>0.21</td><td>0.21</td></tr><tr><td>MSE</td><td>0.08</td><td>0.08</td><td>0.07</td></tr><tr><td>RMSE</td><td>0.28</td><td>0.28</td><td>0.27</td></tr><tr><td>R<sup>2</sup></td><td>0.09</td><td>0.09</td><td>0.13</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Fig. 2: Performance of Case 3, comparing true and predicted location of HH20.</figcaption></figure>

Figure 2 plots the true values of HH20 against the predicted values of HH20 for Case 3. Overall, the neural network manages to track the movements in the time series quite well and predict most of the actual values with sufficient accuracy. However, it tends to smooth predicted values and has difficulties predicting spikes, here values below 953.25 or above 954.25.

<figure><img src="image_5.png" /><figcaption>Fig. 3: Residual histogram for all cases of the neural network.</figcaption></figure>

A good prediction should have uncorrelated residuals with zero mean [9]. The descriptive statistics for all models are presented in Table 2, with the corresponding histograms presented in Figure 3. Again, the different measures and visualization confirm that the prediction quality of the three cases is comparable, with Case 3 performing slightly better. The mean value of Case 1 is positive,
whereas the mean values of Case 2 and 3 are negative. This can be attributed to the fact that Case 1 sometimes predicts higher values than the actual values. Cases 2 and 3, on the contrary, are more restrictive and tend to smooth the predicted values. This observation can be confirmed by the *min* and *max* measurements. The mean is close to zero for all cases with a small deviation around it (about 0.27). From both visualization and descriptive statistics, the residuals appear to be close to Gaussian distribution.

<table><caption>Table 2: Descriptive statistics for the residuals.</caption><thead><tr><th></th><th>Case 1</th><th>Case 2</th><th>Case 3</th></tr></thead><tbody><tr><td>mean</td><td>0.10</td><td>-0.10</td><td>-0.09</td></tr><tr><td>std</td><td>0.27</td><td>0.27</td><td>0.26</td></tr><tr><td>min</td><td>-0.88</td><td>-0.97</td><td>-0.79</td></tr><tr><td>25%</td><td>-0.23</td><td>-0.24</td><td>-0.24</td></tr><tr><td>50%</td><td>0.07</td><td>-0.08</td><td>-0.08</td></tr><tr><td>75%</td><td>0.08</td><td>0.08</td><td>0.09</td></tr><tr><td>max</td><td>0.78</td><td>0.66</td><td>0.67</td></tr></tbody></table>

Overall, the inability to predict any of the spikes can be a challenge for using this neural network to predict the properties of a specific product: if extreme measurements cannot be predicted, the neural network will be overly optimistic regarding the production system's ability to produce parts that are within the specified tolerance limits. It might still be possible though, to use the neural network to detect trends indicating that the production process is no longer stable. Unfortunately, the available measurement data is currently not sufficient for such a study.

# 5 Conclusion

This paper studies the use of a neural network to predict the location of milled holes in a bumper beam. Three different cases have been analyzed where the neural network is trained on different amounts of historical measurement data. The predicted values match the true locations of the holes on the beam quite well. However, the neural network generally underestimates more extreme deviations from the dedicated position, both in terms of frequency of occurrence and magnitude of deviation.

As a first extension to the work presented in this paper, we will use the neural network for the multivariate case, i. e. to predict the coordinates of multiple holes. An important aspect that needs to be investigated further is also the ability to predict more extreme values to get a more accurate estimation of the probability of producing a beam violating its specifications. A data analysis could help to identify outliers (anomalies) in the production process and thus improve the ability to predict extreme values. Finally, the effect of including other data
than measurement data, e. g. alloy or temperature variations in storage, on the
quality of the prediction will also be investigated.

References

1. Chollet, F.: Deep Learning with Python. Manning Publications (2017)

2. Davoodi, M.M., Sapuan, S.M., Ahmad, D., Ali, A., and Mehdi Jonoobi, A.K.: Mechanical properties of hybrid kenaf/glass reinforced epoxy composite for passenger car bumper beam. Materials & Design **31**(10), 4927–4932 (2010)

3. Dreyfus, G.: Neural networks: methodology and applications. Springer Science & Business Media (2005)

4. European Aluminium: Aluminium Automotive Manual. European Aluminium Association, Brussels, Belgium (2015), https://www.european-aluminium.eu/resource-hub/aluminium-automotive-manual/, last accessed: 19 March 2021

5. Goldstein, M., Uchida, S.: A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data. PloS one **11**(4), e0152173 (2016)

6. Jia, X., Di, Y., Feng, J., Yang, Q., Dai, H., Lee, J.: Adaptive virtual metrology for semiconductor chemical mechanical planarization process using gmdh-type polynomial neural networks. Journal of Process Control **62**, 44–54 (2018)

7. Karayel, D.: Prediction and control of surface roughness in CNC lathe using artificial neural network. Journal of materials processing technology **209**(7), 3125–3137 (2009)

8. Kim, D., Kang, P., Cho, S., joo Lee, H., Doh, S.: Machine learning-based novelty detection for faulty wafer detection in semiconductor manufacturing. Expert Systems with Applications **39**(4), 4075–4083 (2012). https://doi.org/10.1016/j.eswa.2011.09.088, https://www.sciencedirect.com/science/article/pii/S0957417411014114

9. Kuhn, M., Johnson, K.: Applied Predictive Modeling, vol. 26. Springer New York (2013). https://doi.org/10.1007/978-1-4614-6849-3_2

10. Maggipinto, M., Beghi, A., McLoone, S., Susto, G.A.: Deepvm: A deep learning-based approach with automatic feature extraction for 2d input data virtual metrology. Journal of Process Control **84**, 24–34 (2019)

11. Martin, O., Lopez, M., Martin, F.: Artificial neural networks for quality control by ultrasonic testing in resistance spot welding. Journal of Materials Processing Technology **183**(2-3), 226–233 (2007)

12. Puggini, L., McLoone, S.: Extreme learning machines for virtual metrology and etch rate prediction. In: 2015 26th Irish Signals and Systems Conference (ISSC). pp. 1–6. IEEE (2015)

13. Roblek, V., Meško, M., Krapež, A.: A Complex View of Industry 4.0. SAGE Open **6**(2), 1–11 (2016)

14. Santos, C., Mehrsai, A., Barros, A., Araújo, M., Ares, E.: Towards Industry 4.0: an overview of European strategic roadmaps. Procedia Manufacturing **13**, 972–979 (2017)

15. Schmitt, J., Bönig, J., Borggräfe, T., Beitinger, G., Deuse, J.: Predictive model-based quality inspection using machine learning and edge cloud computing. Advanced Engineering Informatics **45**, 101101 (2020)

16. Tao, F., Qi, Q., Liu, A., Kusiak, A.: Data-driven smart manufacturing. Journal of Manufacturing Systems **48**(C), 157–169 (2018)
17. Tsai, Y.H., Chen, J.C., Lou, S.J.: An in-process surface recognition system based on neural networks in end milling cutting operations. International Journal of Machine Tools and Manufacture **39**(4), 583–605 (1999)

18. Wan, J., Tang, S., Li, D., Wang, S., Liu, C., Abbas, H., Vasilakos, A.V.: A manufac- turing big data solution for active preventive maintenance. IEEE Transactions on Industrial Informatics **13**(4), 2039–2047 (2017)

19. Wang, T., Chen, Y., Qiao, M., Snoussi, H.: A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology **94**(9-12), 3465–3471 (2018)

20. Wang, Y., Liu, M., Zheng, P., Yang, H., Zou, J.: A smart surface inspection system using faster r-cnn in cloud-edge computing environment. Advanced Engineering Informatics **43**, 101037 (2020)

21. Wuest, T., Weimer, D., Irgens, C., Thoben, K.D.: Machine learning in manufac- turing: advantages, challenges, and applications. Production & Manufacturing Re- search **4**(1), 23–45 (2016)

22. Zhao, D., Wang, Y., Liang, D., Ivanov, M.: Performances of regression model and artificial neural network in monitoring welding quality based on power signal. Jour- nal of materials research and technology **9**(2), 1231–1240 (2020)