# Continuous acoustic data mining using machine learning

Théotime de la Selle, Stéphanie Deschanel, Jérôme Weiss

► To cite this version:

Théotime de la Selle, Stéphanie Deschanel, Jérôme Weiss. Continuous acoustic data mining using machine learning. EWGAE 2024, Sep 2024, Postdam (Allemagne), Germany. e-Journal of Nonde-structive Testing, 2024. hal-05354352

HAL Id: hal-05354352

https://hal.science/hal-05354352v1

Submitted on 7 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

Continuous acoustic data mining
using machine learning

Théotime de la SELLE¹, Stéphanie DESCHANEL², Jérôme WEISS¹

¹Institut des sciences de la Terre, Grenoble, France, Theotime.de-la-selle@univ-grenoble-alpes.fr

²Matéis (INSA Lyon), Lyon, France, Stephanie.deschanel@insa-lyon.fr

**Abstract.** In order to address the challenge of exploring new signals and recognizing sources based on both statistics and physics, a relevant time series representation tool, known as a deep scattering network, has been developed. A deep scattering network is a deep convolutional neural network that implements a cascade of convolutions with wavelet filters, a modulus function, and pooling operations. The advantage for unsupervised classification of time series is that deep scattering spectra are locally invariant to translation and preserve transient phenomena such as attack and amplitude modulation.

We show that an AE adapted scattering network, combined with reduction model and clustering algorithm, is an efficient tool to perform unsupervised investigation of continuous acoustic data by applying our method on AE streaming recorded during fatigue testing: low amplitude acoustic multiplets non registered by common AE procedure has been clustered successfully and the content of the continuous acoustic background can be automatically grouped into classes of similar physical mechanisms, e.g. frictions, plasticity or electronic and mechanical noise.

**Keywords:** acoustic emission, unsupervised machine learning, deep scattering network, continuous acoustic data, fatigue testing

**Introduction**

The analysis of continuous seismogram data in the form of time series is widely used because of the variety of signals with very different time and frequency scales. Increased computing power and advances in machine learning have paved the way for the exploration of the internal structure of seismic data, enabling signals to be grouped according to their sources and new signals to be discovered [1]. Initially only supervised, unsupervised clustering investigation methods requiring little human expertise in signal identification and not based on previously known data sets, offer now new prospects [2][3].
In the field of acoustic emission (AE), classical studies distinguish *discrete AE*, ie high-amplitude burst-type signals, from *continuous AE*, i.e. the superposition of low-amplitude AE signals from various sources and noise (environmental and electronic). In almost all papers on AE, only *discrete AE* is studied - by amplitude-threshold crossing - as the amount of data generated by continuous recording in a few seconds is considerable and there were no powerful scientific tools for analysing this vast mixture of signals and noise. However, the investigation of AE signals within the *continuous AE* part will permit the exploration of continuous emission mechanisms in materials, such as *mild plasticity* [4], or the detection of acoustic signatures already known thanks to some of their high-amplitude occurrences belonging to *discrete AE*, in the *continuous* part.

For this reason, we propose here a new *continuous acoustic data mining* method based on a similar analysis in the field of seismological. After having presented its workflow, in particular the principles of *deep scattering networks*, we show an application example of the method on experimental AE data streaming from fatigue tests on alloys.

# 1. Methods : unsupervised machine learning

## 1.1 Overall architecture

The data mining method introduced here takes part in the category of time series *sub-sequence grouping*. The goal is to group sliding windows extracted from a single time series – here a full AE datastreaming -, with the aim of finding similarities and differences between the extracted time windows around 1ms in our case. This objective is achieved through a classic unsupervised machine learning workflow: features determination (spectrograms of each extracted time windows), dimensionnality reduction and clustering. Each step are detailed in the three following sub-sections.

<figure><img src="image_3.png" /><figcaption>Figure 1: Figure 1: Schematic representation of the continuous acoustic data mining method. a) Continuous AE recording, sectioned into 1ms windows. b) Deep scattering spectrogram obtained as output from the two-layers deep scattering network. c) Feature matrix extracted from the deep scattering spectrogram using NMF analysis. d) Dendrogram computed by agglomerative hierarchical clustering based on a similarity measure in the feature space (clustering can also be performed by GMM using the method presented in section 1.4). Illustration adapted from [1].</figcaption></figure>
## 1.2 Deep scattering network

In order to achieve a relevant time series clustering, it is necessary to adopt a representation of the signal that is locally invariant with respect to translation and preserves transient phenomena such as attack and amplitude modulation. Indeed, Andén and Mallat [5] have shown that a spectrogram generated by the Fourier transform is not ideal for clustering purposes because it is not stable to temporal distortions, particularly at periods that are short relative to the duration of the analysis window. Such kind of stable representation can be obtained through a *deep scattering network* which applies a cascade of convolutions (wavelets filters), a modulus function and pooling operations to the extracted time series. From successive wavelet transforms (Gabor wavelets are used here [5]) and modulus operations, scalograms of different orders are produced. At each layer (2 is enough for audio signals [5] and seismograms [2]) of the network, scattering coefficients are extracted from scalograms by a pooling operation (low-pass filter) and then collected over all time windows to obtain the deep scattering spectogram, i.e. a scattering coefficient matrix. Naturally, the network has to be parametered with octaves numbers and wavelet banks for each layer plus the type of pooling. In fact, these parameters are physically-driven by temporal and frequential characteristics of both the data and the attempt ranges of potential interesting signals within the recording (see [2][6] for details). As a result, features generated by this network have also a physical meaning, unlike other deep networks. Finally, to reduce the effect of signal amplitude in favour of frequency and time variations on the dimensionnality reduction, we normalise the data.

## 1.3 Dimensionnality reduction

The information contained in the deep scattering spectrogram at the network output is locally invariant with respect to translation and preserves transient phenomena such as attack and amplitude modulation as it is requiered for clustering. However, this representation of time windows in the space of deep scattering coefficients is very redundant and high-dimensional, which is not suitable for clustering due to the curse of dimensionality. We therefore resort to a dimension reduction method.

Thus, we reduce the number of dimensions with a *Non-negative Matrix Factorization* (NMF), a method for factoring a matrix $M \in R^{n×d}$ of *n* observations described by *d* dimensions into two matrices $W \in R^{n×k}$ and $H ∈ R^{k×d}$ (with *k* the number of dimensions chosen) containing only positive or zero values and whose product *WH* is close to *M*. Remark that all other dimensionnality reduction methods would also work. NMF has the advantages to result in positive-value features, easily understandable in case of signals spectrograms. Applied to our analysis, the NMF allows us to form a low-dimensional latent space from the scattering coefficients space. However, as in all dimensionnality reduction methods, there is a compromise between the size of the latent space (number of dimension *k*) and the resulting information loss. Depending on the application of the method, e.g. detection of specific signals or overall investigation of the dataset, the strategy would differs depending on the need of interpretation of temporal and frequential variations in dimensions.

## 1.4 Clustering

Each observations of the dataset, i.e. the time windows, are ultimately clustered according to their temporal and frequential characteristics represented in the latent space, the output of the NMF. This clustering task can be achieved by several algorithms. We use
here the common *hierarchical clustering* method as it is suitable for unsupervised investigation and offers an overall view of the underlying data structure.

The hierarchical clustering algorithm aims to successively agglomerate observations - defined here as data points in a given feature space - on the basis of a distance measure (here euclidean distance). At initialization, all the observations in the dataset each form an independent cluster. Depending on the distance measure and the agglomeration strategy (here Ward's method), the closest clusters are assembled one after the other and the number of clusters gradually decreases until all observations are unified into a single cluster [JOH 67]. A dendrogram illustrates this process, representing the successive clusters as a function of the distance between the clusters, as for example in fig. 1 d). Depending on the truncation distance selected in the dendrogram, a different number of clusters is obtained.

## 2. Experiments and datasets

This method can be applied on any continuous acoustic recording from AE sensors (each channel independently). Because we have previously detected and characterise acoustic signatures of fatigue cracks by classical discrete AE approach on experimental fatigue testing in alloys [7][8], we analyse the datastreaming acquired during these tests with the method presented above and interpret the results through the known background.

<figure><img src="image_4.png" /><figcaption>Figure 2: Photo of the fatigue assembly showing the XC48 steel CT specimen and 4 AE sensors</figcaption></figure>

### 2.1 Materials, AE parameters and mechanical sollicitation

We performed fatigue crack growth tests on *compact tension* (CT) specimens in carbon steel XC48. A sinusoidal load-imposed sollicitation was applied at 5Hz with a load ratio $R=P_{min}/P_{max}=0.1$ (where $P_{min}$ and $P_{max}$ are the minimum and maximum applied loads) until the final rupture of the specimen. 4 sensors, two resonant *Nano30* sampled à 2 MHz and two broad-band *Micro200* sampled at 5MHz, monitored the tests and recorded – whitout any filters - AE hits from threshold crossing and AE datastreaming during all the tests.

### 2.3 Previous investigations of fatigue cracks acoustic signatures in experiments

We have shown that it is possible to automatically detect acoustic multiplets, i.e. highly similar signals (AE hits obtained by threshold crossing) triggered at almost same load level during hundred or thousands of cycles, in these tests and investigate their source mechanisms. These multiplets are specific signatures of fatigue crack revealing its presence as due to crack surfaces asperities repeated shocks. See further details about the detection algorithm [7] and source investigation [8]. From additional full-field
measurements during these tests by Digital Image Correlation (DIC), we also know that there was crack closure (contacts of crack surfaces) and intense plasticity at crack tip ; two potential AE sources that we may found in continuous AE data.

## 2.2 Dataset preprocessing

For clarity, we only analyse a short part (1 minute) of the full signal - lasting several hours -. We select a portion where multiplets were recorded and crack closure was observed. To compare resulting clustering from various frequency responses (resonant or brand-band) and sampling frequencies (2MHz or 5MHz), we extract the same portion for the two sensors (Nano30 and Micro200). The 1 minute continuous signal from Nano30 sensor is shown in fig. 3. We observe that AE hits are clearly visible and the amplitude of background signal is following the sinusoidal solliciation (5Hz). For some unknown reasons – maybe format conversion procedure – the continuous recording is interspersed by eletronic noise phases lasting few seconds visible in fig. 3 a) and b).

As the time window is set at 1ms, we obtain a dataset consisting of $N = t_{signal}/t_{window} = 20969$ observations ($t_{signal}$ the duration of the recording portion and $t_{window}$ the size of the time window) at the output of the deep scattering network.

<figure><img src="image_5.png" /><figcaption>Figure 3: Portion of a continuous EA recording (Nano30) from an imposed load fatigue test imposed load fatigue test ($R = 0.1$ at 5Hz) on an XC48 steel specimen. The red dotted lines indicate the level of the discrete EA acquisition threshold. a) Complete portion b) Enlargement of the area framed in black over 2.5s.</figcaption></figure>

# 3. Results

Our continuous acoustic data mining method was applied identically to the two independent datasets - one for each sensor - composed of 1 minute AE datastreaming acquired during fatigue crack growth test on XC48 steel CT specimen. The analysis of Nano30 recording is first presented and then compared to the Micro200 one.

## 3.1 Nano30 continuous data clustering

Clustering results are represented through a dendrogram (fig. 4) which reveals the underlying structure of the data depending on temporal and frequencial characteristics - identified by the method - in time windows. As the objective of clustering is to make dense and well separated clusters (in the latent space), a dendrogram cut around distance 240 appears naturally, creating 5 clusters. Note that cluster 0 may be separate into two or more sub-clusters (not studied here).
<figure><img src="image_6.png" /><figcaption>Figure 4: Unsupervised hierarchical clustering dendrogram (Ward's method) grouping the windows of the continuous acoustic recording of a fatigue test on an XC48 steel CT specimen. 5 *clusters* are extracted by dendrogram distance-cut.</figcaption></figure>

To visualise the clustering results, we can plot the detection rate per cluster for each of these 5 clusters simultaneously. To do this, we calculate the detection curve for each cluster (one-hot encoding) and then calculate the average of the results in a sliding window. Figure 5 shows the detection rates for each cluster. The periodicity of clusters 0, 1 and 4 can be seen in fig. 5.b). Their average detection rates are synchronised with the cyclic variations of the acoustic signal - and therefore of the mechanical load - and are repeated similarly at each cycle throughout the recording. Cluster 3 is strictly composed of time windows containing only electronic noise sporadically affecting the recording while cluster 2 contains the signals of acoustic multiplets (type II) detected by the previous study (see 2.3).

<figure><img src="image_7.png" /><figcaption>Figure 5: Averaged detection rates of the 5 clusters from the dendrogram section in figure 4. a) Detection rate of the complete portion. b) Continuous acoustic signal (grey) and time windows detection rate of the 5 clusters (coloured curves without units).</figcaption></figure>

### 3.1 *Micro200* continuous data clustering

The same analysis was performed on the second continuous acoustic dataset recorded by a *Micro200* sensor to compare the results. Thus, we plot fig. 6 representing averaged detection rates of the resulting 5 clusters similarly to fig. 5.b). The 5 clusters of both analysis (cluster ids are fully consistent even if the continuous acoustic signal (grey in figures) is totally different (Order and color differs). Note that time origin of the two portions are not exactly the same (time lag).
<figure><img src="image_8.png" /><figcaption>Figure 6: Averaged detection rates of the 5 clusters (colored curved) from continuous acoustic data mining on Micro200 AE recording (grey curve) during fatigue test on XC48 steel.</figcaption></figure>

# 4. Clusters interpretation

## 4.1 Two sensors comparison

This supplementary analysis on another frequency response serves to show the robustness and efficiency of the method as variations of sensitivity, band-pass, sampling frequency and positions of the sensor – which results in totally different continuous signals in fig. 5b) and 6 - do not affect significatively the clustering, even if the amplitude variations are very low compare to quantization step size. The high proximity of these clustering results despite the differences in continuous data invites to cross-interpret the frequencial and temporal contents of each clusters to characterise more deeply their respective sources (not studied here).

## 4.2 Investigation of sources associated with each cluster

Through the detection rate of each clusters, the waveforms of time windows, the dendrogram and the features constructed by the deep scattering network and the NMF, we try to associate and characterise the physical sources identified in previous studies (see 2.3) to signals content of time windows.

As noted in 3.1, cluster 2 is composed of 300 time windows containing type II multiplets signals provoked by repeated local fatigue crack surfaces contacts and constitutes the acoustic signatures of the presence of a fatigue crack. On the dendrogram (fig. 5), from a cut-off level of around 50 (Euclidean distance), we form two sub-clusters included in cluster 2, each containing a multiplet with waveforms quite distinct from those of the other. Fig. 7 shows the detection rates at regular time intervals (each cycle) and a few waveforms, enabling us to check the content of these two sub-classes. Thus, not only our method clustered the multiplets signals of the dataset, but also permit to separate this cluster in different multiplets.

Regarding the cluster 3 which contains all the windows of electronic noise - visually checked on waveforms (not plotted here) -, we notice that it is the most distanced cluster on the dendrogram, showing a radical difference in the underlying structure of the data between pur noise and AE signal.
<figure><img src="image_9.png" /><figcaption>Figure 7: Sub-clusters of cluster 2, each containing a multiplet. a) Detection rate over the entire length of the record for each of sub-clusters 1 and 2. b) Some windows from the two sub-clusters.</figcaption></figure>

The 3 remaining clusters 0, 1 and 4 represent more than 80% of the total number of windows. They contains the data commonly called *continuous AE* and cannot be investigated through their waveforms as no pattern emerge from it, in *Nano30* recording as well as in *Micro200* one. From their detection rates in fig. 5b), we see that each cycle (from minimum to minimum loads) is decomposed in a succession of the 3 clusters (4-0-1-0-4) meaning that their respective sources are activating sequentially at each cycle. To reject the idea of a basic mechanical noise level variation decomposed in 3 clusters, we have checked that the amplitude of background signal evolves slowly during all the fatigue test, showing that it is due to AE. Furthermore, it has been shown that there is a correspondence between the *continuous AE* and the plastic strain rate, a signature of plastic deformation resulting from the cumulative effect of many small and uncorrelated (in space and time) dislocation rearrangements, in cyclic loading tests on pure polycrystalline aluminium [WEI 19]. In our case, as the fatigue crack concentrates plastic deformation at crack tip and since this plastic flow occurs while the crack is sufficently opened, we associate this mechanisms to the cluster 1 containing windows around the maximum load. Then, as we know that there is crack closure, we associate cluster 0 - which windows are surrounding those of cluster 1 - to fatigue crack surfaces frictional mechanism during the opening and closing phase of the crack (debris have been observed on MEB fractography [SEL 24]), while the last cluster (4) with windows around minimum load refers to mechanical noise whose signal amplitude is constant over the entire test.

These hypothesis are reinforced by several arguments: from the dendrogram, we see that cluster 0 and 1 are the closest ones, which is coherent with the idea of a distinction between noise-based windows (cluster 4) and AE-based windows (clusters 0 and 1). In addition on the averaged detection rates curves (fig. 5 and 6), the transition between clusters 0 and 1 during each cycle are smoother than the one between clusters 0 and 4: it can be explained by the fact that the mechanisms at crack tip (plasticity and damage) and the crack surfaces frictions have a period of coexistence at each cycles in which the frictional sources decrease while plasticity and damage increase. Finally, we noticed that multiplets signals (cluster 2), which are crack surfaces asperities shocks, had occurred during cluster 0 associated to crack closure.
## Conclusion

Through a continuous acoustic data mining approach based on machine learning methods we show that it is possible to cluster high amplitude burst-type AE signals as well as background signal based on their acoustic contents (temporal and frequency characteristics) which directly depend on their respective physical sources. Especially, the presented method is based on a *deep scattering network* which build a representation of the signal that is locally invariant with respect to translation and preserves transient phenomena such as attack and amplitude modulation. Dimensionnality reduction by NMF and hierarchical clustering can then efficiently cluster the acoustic data and reveal its underlying structure. This method has been successfully tested on continuous AE datasets from two sensors of various frequency responses recorded during fatigue crack growth tests of XC48 steel already studied in previous studies. 5 clearly distinct clusters emerged from the application of the method on the two recordings and each of these clusters have been interpreted and associated to physical sources: crack surfaces frictions during crack closure or opening, plasticity or damage at crack tip, acoustic multiplets generated by crack surfaces asperities repeated contacts, mechanical and electronic noise.

From the association of sources to clusters obtain by continuous data mining, it is then possible to investigate further the signal content (temporal and frequency characteristics) of each cluster through the features of the latent space used for clustering or their respective deep scattering spectrograms. Such kind of analysis open the way for physical sources and acoustic signatures investigation.

## Bibliography

[1] Steinmann R., Seydoux L., Beaucé, Campillo M.
*Hierarchical Exploration of Continuous Seismograms With Unsupervised Learning.*
Journal of Geophysical Research: Solid Earth, vol. 127, no 1, 2022.

[2] Seydoux L., Balestriero R., Poli P., Hoop M. d., Campillo M., Baraniuk R.
*Clustering earthquake signals and background noises in continuous seismic data with unsupervised deep learning.*
Nature communications, vol. 11, no 1, 2020, page 3972, Nature Publishing Group UK London.

[3] Shi P., Seydoux L., Poli P.
*Unsupervised Learning of Seismic Wavefield Features: Clustering Continuous Array Seismic Data During the 2009 L'Aquila Earthquake.*
Journal of Geophysical Research: Solid Earth, vol. 126, 2021.

[4] Weiss J., Rhouma W. B., Richeton T., Deschanel S., Louchet F., Truskinovsky L.
*From Mild to Wild Fluctuations in Crystal Plasticity.*
Phys. Rev. Lett., vol. 114, 2015, page 105504, American Physical Society.

[5] Anden J., Mallat S.
*Deep Scattering Spectrum.*
IEEE Transactions on Signal Processing, vol. 62, no 16, 2014, p. 4114–4128.

[6] Steinmann R., Seydoux L., Campillo M.
*AI Based Unmixing of Medium and Source Signatures From Seismograms: Ground Freezing Patterns.*
Geophysical Research Letters, vol. 49, no 15, 2022.

[7] de la Selle T., Weiss J., Deschanel S.
*Acoustic multiplets detection based on DBSCAN and cross-correlation.*
Mechanical Systems and Signal Processing, vol. 211, 2024.

[8] de la Selle T., Réthoré J., Weiss J., Lachambre J., Deschanel S.
*Early detection and characterization of fatigue crack growth from acoustic emission repeaters and image correlation.*
Available at SSRN: https://ssrn.com/abstract=4758696.