# In-Process Noise Detection System for Product Inspection by Using Acoustic Data

Woonsang Baek, Duck Young Kim

► To cite this version:

Woonsang Baek, Duck Young Kim. In-Process Noise Detection System for Product Inspection by Using Acoustic Data. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2019, Austin, TX, United States. pp.413-420, 10.1007/978-3-030-29996-5_48. hal-02460470

HAL Id: hal-02460470

https://inria.hal.science/hal-02460470

Submitted on 30 Jan 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# In-process noise detection system for product inspection
by using acoustic data

Woonsang Baek¹[0000-0003-0353-2291] and Duck Young Kim¹*[0000-0003-0072-4693]

¹ Ulsan National Institute of Science and Technology, Ulsan, South Korea

*dykim@unist.ac.kr

**Abstract.** Objective quality inspection of products in manufacturing process is inseparable from sensor technologies. Inspection methods using analysis of vibration signals have advantages such as being non-destructive, accurate, fast for in-process application. This paper presents recent developments and applications of in-process product inspection which use vibration and acoustic data in various industry. In detail, the inspection system developed with accelerometer, laser vibrometer, laser ultrasonic sensor, acoustic emission sensor, and microphone are presented. An in-process noise detection system for car body parts inspection is introduced as a case study.

**Keywords:** In-process inspection, Vibration sensors, Acoustic sensors.

## 1 Introduction

Increasing awareness about quality of products have improved performance of in-process inspection in highly automated manufacturing process. Since the manufacturing process vary from molding, machining, and so on, numerous quality indicators, which are measurable physical manifestations of the products, have been suggested. Inspection with as many indicators as possible should provide plenty information of the products, however, limitations of takt time and cost hinder the usage of multi-indicators. Therefore, selecting proper quality indicators according to characteristics of the inspection process has been a critical issue. For instance, visual inspection with computer vision technology is one of the representative inspection method that measures dimensional error or color of the products. Not only for the manufactured products, but also it has been applied to inspection of raw products such as agricultural foods [1].

However, the visual inspection is not the best solution when it is necessary to inspect inside of the product. In this cases, inspection using vibration data can be an alternative solution since it has strength in detecting changes and dynamics inside of the products [2]. However, in the past decades, most of the inspection using vibration data have been applied to condition monitoring of industrial machineries, not for inspection of the products. The reason is that the vibration sensors should had to be contacted with the surface of the target. Moreover, developing apparatus to fix the vibration sensors at the surface is improper for industrial applications [3].
<figure><img src="image_2.png" /><figcaption>Fig. 1. Illustration of five inspection methods using vibration sensors</figcaption></figure>

Recently, advances in sensor technologies enabled various types of vibration sensors which are capable of remote measurements of vibration signals. Also, improved signal processing algorithms have increased precision of the inspection. Therefore, numerous researches adopted the vibration sensors for product inspection.

Our paper reviews the state-of-the-art technologies for in-process product inspection methods using the vibration sensors. In detail, inspection systems which installed ac-celerometer, laser vibrometer, laser ultrasonic sensor, acoustic emission sensor, and microphone are presented.

Moreover, the solution of the practical problem, that developing an in-process in-spection system for a car body part is introduced, based on the state-of-the-art technol-ogies. The presence of the irritating small noises that caused by assembly defects of the car body parts are determined by a noise detection algorithm.

## 2 Vibrations sensors for inspection

Product inspection methods using the vibration and acoustic data, assume that assembly errors, random discrepancies of components have potential to be manifested in abnor-mal vibratory behaviors. Traditionally, the vibration data have been measured by con-tact-type sensors, however, the contact type sensors require sensing part that is in con-tact with the products. Non-contact type sensors recently have aroused as an attractive
alternative due to its simple implementation. Post-processing analyses of vibration data
make use of the features in time domain, frequency domain, and cepstral domain [4].

Considering variety of the sensors, five major categories are created to classify the existing inspection methods. Inspection methods using each type of sensors are as follows: inspection using accelerometer; inspection using laser vibrometer; inspection using laser ultrasonics; inspection using acoustic emission sensor; inspection using microphone. Here, the selected representative papers are introduced as below.

## 2.1 Inspection using accelerometer

Accelerometer is a seismic type sensor that consists of a spring-damper system, and a piezoelectric crystal, to create an electric signal proportional to the acceleration of the measured vibration. The sensor is mostly installed at the surface of the product that being measured. Most of the previous studies have monitored the continuous vibration response of fixed machinery for condition monitoring. However, there are several applications that adopted the sensor for inspection of products.

The inspection system for microchip packages, which measures quality of sealing, had installed the accelerometer at the vibrating fixture. Signal processing techniques, such as PCA (Principal Component Analysis) and statistical process control were applied to determine the status of collected data [5]. In case of inspecting injection-molding process, checking existence of the flash that caused by imperfect plastic resin is the main concern. Accelerometers were installed at the fixture of the inspection system to collect the dynamic trend of the mold movement during the filling stage. Data captured by Z direction of the accelerometer was in particular analyzed to extract feature signal, and logistics regression model was applied for determination of the state [6].

Assembly process of automotive assembly with robot arm and gripper, had applied the accelerometer to monitor the quality of assembled product. The traditional assembly monitoring system consists of force and moment sensors, was supported by analysis of the vibration data. Various features in frequency domain were extracted, then three-layered neural network were applied for automatic classification of the defects [7].

## 2.2 Inspection using laser vibrometer

Laser vibrometer measures the vibration velocity of objects using Doppler effect while providing high spatial resolution in a wide frequency band. In addition, the laser vibrometer has its strength in easy application, which reduces the cost for developing the inspection station. In several industrial cases, the vibrometer has proven their advantages over the accelerometers, especially when the contact measurement is not available, or contaminate the target signals.

A comparative study of electric motor inspection measured the vibration data by using the accelerometer and the laser vibrometer. The study shown that the data measured by the laser vibrometer derived higher correlation with defects than that of the accelerometer. The research mentioned that the data of accelerometer was contaminated by the electromagnetic noise from the surface of the motor that distorts frequency spectra and compromise classification through frequency analysis [8]. For extracting
the quality indicator of fruits, especially for firmness, which is not easily measured by a visual observation, the power spectrum analysis was applied to detect differences in displacements inferior to the nanometer [9].

On-line inspection system of washing machines installed the laser vibrometer at right above the position that the products be fixed for repeatable inspection procedures. The system concentrated on detecting abnormal vibrations of the tub that usually generated by the faulty operation of motor. Determination of the state was conducted by measuring features in frequency domain after reducing the speckle noise from the collected signal [10]. In the other research, the similar inspection system was installed at the mobile robot for warehouse inspection, while analyzing the power spectrum of the collected signals for detection of abnormal vibration [11].

## 2.3 Inspection using laser ultrasonics

Laser ultrasonic sensor is a non-contact type sensor that have been widely used for non-destructive inspection of structures, and products. The sensor collects the vibration response of the surface where a brief laser pulses are induced by pulse generator. Then, a heterodyne, fiber-coupled laser interferometer measures the out-of-plane displacement of the surface. Since the laser ultrasonic sensor has its strength in measuring vibration signals of rough surfaces, it has been applied to various industrial inspections.

Inspection of chip-scale packages can be the representative case of the laser ultrasonics application for product inspection. The inspection systems were developed to measure unexpected cracks inside of the multilayer ceramic capacitors. Both features of time-domain and frequency domain were extracted, and compared for fault detection in real-time [12].

The laser ultrasonic sensor is also assumed to have potential advantages for the inspection of composites. Inspection of the products in aeronautic industry is one of the popular application, since the products are mostly large, complex, and assembled with various materials. An automated inspection system for composite wing plate installed the laser ultrasonic sensor at the mobile robot for full inspection, and developed an algorithm that detects cracks by using Laplacian filter and standing wave extraction [13]. In case of automotive industry, laser ultrasonic sensors were applied to inspection of defects that are generated during spot welding, friction stir welding, painting, and checking strength of adhesive bonds [14]. Synthetic Aperture Focusing Technique (SAFT) algorithm was developed and applied to the laser ultrasonic data to measure the defects of welded spots [15]. Inspection of painting focused on measuring thickness of painted area by obtaining a resonance spectroscopy with the laser ultrasonic sensor. The determination of the status was conducted by comparing the amplitude spectra with the model for propagation in multilayers.

## 2.4 Inspection using acoustic emission sensor

Acoustic emission sensor collects the transient elastic waves which are generated by the rapid release of energy from a localized source or source within a material. The sensors are widely applied to the field of condition monitoring with its advantage of
high sensitivity regarding dynamic processes or changes in a material. Although the
sensor has limited applicability as the other contact type sensors, there have been re-
searches applied the sensor for the in-process inspection of products.

Recently, additive manufacturing adopted the acoustic emission sensors for measur-
ing the internal defects rather than the traditional monitoring methods, such as temper-
ature monitoring [16]. The acoustic vibration signals were analyzed by extracting var-
ious characteristics of frequency domain, and signal processing methods, such as col-
laborative neural networks [17]. Similarly, inspection system for stir welding and laser
welding installed the sensor at the surface of the galvanized steel for detecting internal
dynamics by using signal processing methods [18].

In addition, inspection system for beverage container that checks the existence of
unwanted materials inside of the container installed the sensor. The pulse compression
method was applied for data preprocessing, and tomographic reconstruction method
was applied for defect detection [19].

## 2.5 Inspection using microphone

Similar to the acoustic emission sensor, a microphone was applied to the condition
monitoring of machineries during manufacturing process. The methods are useful in
certain cases, especially when it is impossible to access the machine since audio meas-
urements can be performed at a distance from the machine. However, the microphone
based methods were not developed and widely applied as the other vibration based in-
spection methods. This is due to the possibility of contamination of the acoustic signal
by unexpected noises such as shop-floor noises [20]. Therefore, application of the mi-
crophones always requires additional apparatus for reducing the shop-floor noise, or
signal processing methods for suppressing, or removing the noise.

Inspection system for cracks of rice grain after milling process installed the micro-
phones with an equalizer and PVC pipes for insulating the shop-floor noise. The mi-
crophones collect the acoustic signals that generated right after the grains crashed into
the plate [21].

In automotive industry, traditional inspection of irritating noises in car body parts
have been done by human ear. Recently, researches about automatic and objective
judgement of that issue have studied with the microphones [22]. The acoustic signals
were collected near a fixture, that vibrates the products to resemble the driving situa-
tion. The dominant sound quality metrics in a multiple linear regression equation was
applied for inspection algorithm [23].

# 3 In-process noise detection system for car body parts

Recently, annoying noises of car body parts have been critical issues for automotive
industry, since it can easily degrade the emotional satisfaction of drivers. The noises
are caused by a lot of potential sources, for instance, error in assembly operations, de-
fects in components, and dimensional control issues [22]. Acquisition of the annoying
noise has been conducted by the microphones [23]. However, the shop floor noises are known to easily contaminate the signals of interest as mentioned in section 2.5. Two methods have proposed to reduce the effect of the shop floor noises: conducting the inspection in an anechoic chamber, or applying the noise reduction methods. However, anechoic chamber is not used for full-inspection of real manufacturing process due to its expensiveness costs. The realistic solution for full-inspection are assumed to be the application of noise reduction methods. However, most of the inspection system about the annoying noise have utilized the anechoic chambers, while the noise reduction methods were not seriously studied.

Therefore, the author proposed an inspection system for car body parts, as shown in Fig.1. The system is applicable for manufacturing process by developing noise reduction system with two groups of microphones [24]. A pneumatic pusher and fixtures were installed for replicating a situation that a driver pushing a car door. A car door trim is slowly pressed down by the pneumatic pusher with a pressure of 10 kgf/cm². The groups of microphones, which are a microphone array and parabolic microphones, collect the acoustic signals during the operation of the pneumatic pusher. The microphones of the microphone array are installed right above the fixture that holds the product, where those of the parabolic microphones are installed at the outside of the system to concentrate on collecting the shop-floor noise. The microphones were grouped to collect real-time data of shop-floor noise, while the most of the existing noise reduction methods use trained shop-floor noise. With assumptions about the sound sources of two groups of microphones, spectral subtraction method and the modified Non-Negativity Matrix Factorization were developed for extracting the annoying noise from the contaminated signal.

<figure><img src="image_3.png" /><figcaption>Fig. 2. In-process noise detection system of the present work of the author</figcaption></figure>
## 4 Conclusion

The paper reviews the current quality inspection of products during manufacturing process. Since inspection methods using analysis of vibration signals have advantages such as being non-destructive, accurate, fast and applicable for in-process inspection, the focus of review is on the vibrational sensors. Applications of in-process product inspection by using five categories of the sensors are presented, which are inspection using accelerometer, laser vibrometer, laser ultrasonic sensor, acoustic emission sensor, and microphone are presented.

As a case study, the in-process noise detection system for inspection of car body parts is presented. The configuration of developed system, named in-process noise detection system, is briefly introduced. The system was developed for replacing human-based inspection by devising noise detection algorithms with acoustic data. Since the shop-floor of manufacturing line generates shop-floor noise that may contaminate the acoustic signals, noise reduction methods are also developed. The system installed two groups of the microphones to improve the noise reduction methods.

## References

1. Narendra, V., Hareesha, K.: Quality inspection and grading of agricultural and food products by computer vision-A review. International Journal of Computer Applications. **2**, 43-65 (2010)

2. Chui, Y., Barclay, D., Cooper, P.: Evaluation of wood poles using a free vibration technique. Journal of Testing and Evaluation. **27**, 191-195 (1999)

3. Rodriguez, R., Cristalli, C., Paone, N.: Comparative study between laser vibrometer and accelerometer measurements for mechanical fault detection of electric motors. In: Fifth International Conference on Vibration Measurements by Laser Techniques: Advances and Applications, pp. 521-530 (2002).

4. Aryan, P., Sampath, S., Sohn, H.: An overview of non-destructive testing methods for integrated circuit packaging inspection. Sensors **18**, 1-27 (2018)

5. Ostyn, B., Darius, P., De Baerdemaeker, J., De Ketelaere, B.: Statistical monitoring of a sealing process by means of multivariate accelerometer data. Quality Engineering. **19**, 299-310 (2007)

6. Zhang, J. Z.: Development of an in-process pokayoke system utilizing accelerometer and logistic regression modeling for monitoring injection molding flash. The International Journal of Advanced Manufacturing Technology. **71**, 1793-1800 (2014)

7. Mechefske, C. K., Sun, Q.: Failure detection in automotive light assemblies during vibration endurance testing. The International Journal of Advanced Manufacturing Technology. **51**, 799-810 (2010)

8. Cristalli, C., Paone, N., Rodríguez, R.: Mechanical fault detection of electric motors by laser vibrometer and accelerometer measurements. Mechanical Systems and Signal Processing **20**, 1350-1361 (2006)

9. Santulli, C., Jeronimidis, G.: Development of a method for nondestructive testing of fruits using scanning laser vibrometry. NDT. Net. **11**, 1-12 (2006)
10. Torcianti, B., Cristalli, C., Vass, J.: Non-contact measurement for mechanical fault detection in production line. In: IEEE International Symposium on Diagnostics for Electric Machines, Power Electronics and Drives, pp. 297-301. IEEE, Atlanta (2007).

11. Raffaeli, R., Cesetti, A., Angione, G., Lattanzi, L., Longhi, S.: Virtual planning for autonomous inspection of electromechanical products. International Journal on Interactive Design and Manufacturing. **6**, 215-231 (2012)

12. Erdahl, D. S., Ume, I. C.: Online-offline laser ultrasonic quality inspection tool for multilayer ceramic capacitors-Part I. IEEE Transactions on Advanced Packaging. **27**, 647-653 (2004)

13. Dubois, M., Drake Jr, T. E.: Evolution of industrial laser-ultrasonic systems for the inspection of composites. Nondestructive Testing and Evaluation. **26**, 213-228 (2011)

14. Blouin, A., Kruger, S., Lévesque, D., Monchalin, J.-P.: Applications of laser-ultrasonics to the automotive Industry. In: Proceedings to 17th World Conference on Non Destructive Testing, pp. 105-112. WCNDT, Shanghai (2008).

15. Lévesque, D., Dubourg, L., Mandache, C., Kruger, S., Lord, M., Merati, A., Jahazi, M., Monchalin, J. P.: Synthetic aperture focusing technique for the ultrasonic evaluation of friction stir welds. In: Proceedings of 2018 AIP Conference, pp. 263-270. ICAMSME, Incheon (2008).

16. Lu, Q. Y., Wong, C. H.: Additive manufacturing process monitoring and control by non-destructive testing techniques: challenges and in-process monitoring. Virtual and Physical Prototyping. **13**, 39-48 (2018)

17. Yoon, J., He, D., Van Hecke, B.: A PHM approach to additive manufacturing equipment health monitoring, fault diagnosis, and quality control. In: Proceedings of the Prognostics and Health Management Society Conference, pp. 1-9 (2014).

18. Gu, H., Duley, W. W.: Resonant acoustic emission during laser welding of metals. Journal of Physics D: Applied Physics. **29**, 550-604 (1996)

19. Ho, K., Billson, D., Hutchins, D.: Inspection of drinks cans using non-contact electromagnetic acoustic transducers. Journal of Food Engineering. **80**, 431-444 (2007)

20. Vilela, R., Metrolho, J., Cardoso, J.: Machine and industrial monitorization system by analysis of acoustic signatures. In: Proceedings of the 12th IEEE Mediterranean Electrotechnical Conference, pp. 277-279. IEEE, New York (2004).

21. Buerano, J., Zalameda, J., Ruiz, R.: Microphone system optimization for free fall impact acoustic method in detection of rice kernel damage. Computers and Electronics in Agriculture. **85**, 140-148 (2012)

22. Bernard, T., Cerrato-Jay, G., Dong, J., Pickering, D., Braner, L., Davidson, R., Jay, M.: The development of a sound quality-based end-of-line inspection system for powered seat adjusters. Technical report, SAE Technical Paper. 0148-7191.

23. Cook, V. G. C., Ali, A.: End-of-line inspection for annoying noises in automobiles: Trends and perspectives. Applied Acoustics. **73**, 265-275 (2012)

24. Baek, W., Kim, D. Y.: An in-process BSR-noise detection system for car door trims. In: IFIP International Conference on Advances in Production Management Systems, pp. 35-38. Springer, Cham (2018).