FISHER: A Foundation Model for Multi-Modal Industrial Signal Comprehensive Representation

## Abstract

Abstract With the rapid deployment of SCADA systems, how to effectively analyze industrial signals and detect abnormal states is an urgent need for the industry. Due to the significant heterogeneity of these signals, which we summarize as the M5 problem, previous works only focus on small sub-problems and employ specialized models, failing to utilize the synergies between modalities and the powerful scaling law. However, we argue that the M5 signals can be modeled in a unified manner due to the intrinsic similarity. As a result, we propose FISHER, a F oundation model for multi-modal I ndustrial S ignal compre HE nsive R epresentation. To support arbitrary sampling rates, FISHER considers the increment of sampling rate as the concatenation of sub-band information. Specifically, FISHER takes the STFT sub-band as the modeling unit and adopts a teacher student SSL framework for pre-training. We also develop the RMIS benchmark, which evaluates the representations of M5 industrial signals on multiple health management tasks. Compared with top SSL models, FISHER showcases versatile and outstanding capabilities with a general performance gain up to 5.03%, along with much more efficient scaling curves. We also investigate the scaling law on downstream tasks and derive potential avenues for future works. FISHER is now open-sourced on https://github.com/jianganbai/FISHER

###### Index Terms:

## I Introduction

With the rapid development of the Internet of Things (IoT) and artificial intelligence (AI) technologies, AI-based supervisory control and data acquisition (SCADA) systems have been widely deployed in modern manufacturing, where production lines are continuously monitored by ubiquitous sensors via various modalities. By analyzing these streaming signals, one can detect anomalies, diagnose malfunctions, and optimize maintenance schedules, thereby reducing downtime and operational costs. Nowadays, the installation of SCADA systems do not present any major technical challenges thanks to the powerful IoT infrastructure. However, how to efficiently analyze these signals is an urgent challenge, due to the unique complexity of signal mechanisms and health management tasks. In this paper, we boil it down to the M5 problem:

- 1.
Multi-modal. Sensors observe the working status via multiple modalities: sound, vibration, voltage, current, temperature, etc.
- 2.
Multi-sampling-rate. Due to cost issues, the sensor sampling rate is often slightly greater than twice the signal bandwidth, therefore possessing great variations. As the data volume scales up, model need to cope with a wide variety of sampling rates.
- 3.
Multi-scale. The diverse working mechanisms of machines (sliding, rotation, static, etc.) lead to different signal patterns, requiring the model to conduct multi-scale analysis for different kinds of signals.
- 4.
Multitask. Anomaly detection, fault diagnosis, remaining useful life (RUL) estimation, etc.
- 5.
Minim fault. Fault data are often scarce. Thus the class distributions of health management tasks are often imbalanced.

Due to the complexity of the M5 problem, existing approaches only focus on small sub-problems, and models are trained specifically for each sub-problem, such as sound-based anomaly detection [1, 2, 3], vibration-based bearing fault diagnosis [4, 5], and vibration-based RUL estimation [6]. These models are typically trained on small datasets such that the sampling configurations can be fixed (e.g. fixed sampling rate, fixed test rig). While these specialized models achieve superior performance within their respective domains, they fail to leverage the potential synergies among different modalities and the potential gains when scaling up [7]. Moreover, they introduce extra burdens during model development and deployment, since each sub-problem must be dealt by an exclusive model.

Figure: (a) Model Performances on Different Tasks of the RMIS Benchmark
Refer to caption: x1.png

In this work, we argue that the M5 problem is mostly related to how the health state is perceived by multiple observation principles, rather than the health state itself. Although industrial signals are heterogeneous in appearance, the internal patterns of these data imply similarities that have yet to be exploited, which are listed as follows:

- 1.
Identical semantic information. Different industrial signals are perceptions of the same mechanical event by different physical laws and thus possessing identical semantic information.
- 2.
Similar generation principles. Sound and vibration, the two most common modalities of industrial signals, are essentially different observational forms of vibration, since sound signals are recorded by the oscillation of the microphone diaphragm.
- 3.
Similar analysis methods. Spectral analysis is widely employed for analyzing various industrial signals, such as Fourier transform, envelope spectrum and wavelet transform, indicating that these signals can be modeled in a unified manner.
- 4.
Similar malfunction patterns. Since machines are assembled from components, their failure patterns are often comparable. Typical malfunction patterns are anomalous frequencies and unexpected impulses.
- 5.
Shared features for multi-tasking. Since health management tasks are all related with malfunctions, a dense representation extracted by a powerful foundation model is sufficient to handle these tasks.

Motivated by the powerful scaling law [7], we assume that by scaling up (model size, data volume, computational resource, test time, etc.), these internal similarities can be uncovered by a unified model, which will be able to leverage the synergies between different modalities and machinery and generalize to various downstream tasks without fine-tuning. Thus, it is viable to overcome the multi-modal, multi-scale and multitask problem. Meanwhile, the mini fault problem can be dealt by the external knowledge injected during large-scale pre-training, which has been proved effective by the success of SSL models in anomaly detection tasks [2, 8]. Finally, the difference in the sampling rate barely affects the signal appearance rather than its essence. As long as the sampling rate satisfies the Nyquist Sampling Theorem, which is a natural law when designing the SCADA system, the information is fully retained in the signal. Thus, the multi-sampling-rate problem does not hinder unified signal modeling. However, it is noted that to leverage the power of large-scale data, the model must be able to cope with arbitrary sampling rates.

In this work, we boldly scale up a single model to uniformly characterize heterogeneous signals. We propose FISHER, short for Foundation model for multi-modal Industrial Signal compreHEnsive Representation, which models the increment of sampling rate as the concatenation of additional sub-band information. Specifically, the raw signal, regardless of the modality, is represented by short time Fourier transform (STFT) with fixed-duration window and hop size. The spectrogram is then split into sub-bands with predefined bandwidth and the model processes these sub-bands individually. The model is trained by a teacher student self distillation framework [9], where the student is guided by the representations of the teacher, and the teacher is an exponential moving average (EMA) version of the student.

Figure: Figure 2: Pipeline of FISHER. FISHER converts signals into STFT spectrograms and splits them into sub-bands with fixed bandwidth $w$. These sub-band are processed individually by the ViT backbone and the [CLS] embeddings are concatenated as the signal representations.
Refer to caption: x3.png

Figure: (a) 16 kHz
Refer to caption: extracted/6643448/imgs/subband/spec_16k.png

To comprehensively evaluate the model, we also develop the RMIS benchmark, short for Representation of M5 Industrial Signals. The RMIS benchmark currently supports two typical health management tasks, i.e. anomaly detection and fault diagnosis. Anomaly detection predicts whether the signal is anomalous without knowing any anomalous data in advance, while fault diagnosis identifies the specific fault type of the signal with labeled data provided for reference. For fault diagnosis tasks, we conduct sealed train-test split to ensure proper difficulty. To evaluate the inherent and generalization capabilities, the model is not fine-tuned on any downstream datasets, but instead directly use k-nearest neighbor (KNN) for inference.

Compared with multiple top SSL models, FISHER showcases versatile performances and efficient scaling properties. On the RMIS benchmark, FISHER achieves an overall score of 62.50%, which surpasses all baselines by at least 5.03%. Meanwhile, FISHER possesses a much more efficient scaling curve, where the performance grows consistently as the model size scales up, and FISHER-tiny with merely 5.5M parameters outperforms all baselines by 3.91%. We believe the success of FISHER mainly comes from its capability of adaptively utilizing the full signal bandwidth, while scaling up data volume and test-time scaling (TTS) are more promising ways for pre-training signal foundation models due to severe signal duplication. To eliminate the impact of split ratio, we compare the performance under multiple split ratios, which is strongly correlated with the performance under fixed split ratio.

The rest of the paper is organized as follows. Section [II](https://arxiv.org/html/2507.16696v1#S2) depicts the proposed model. Section [III](https://arxiv.org/html/2507.16696v1#S3) introduces the RMIS benchmark in detail. Section [IV](https://arxiv.org/html/2507.16696v1#S4) presents the experiment results and our conclusions, and Section [V](https://arxiv.org/html/2507.16696v1#S5) concludes the paper.

## II FISHER

### II-A Sub-band Modeling

In FISHER, the input signal is represented by STFT. In contrary with common sound pre-trained models [9, 10, 11, 12] that adopt the log-mel spectrogram as the signal representation, FISHER reverts to STFT since:

- 1.
Malfunctions often appear in high frequencies, which would be diluted in mel scale.
- 2.
The harmonic relationships of characteristic frequencies are essential, which would be smoothed in mel scale.

To deal with the multi-sampling-rate problem, the STFT window size $N$ is mapped to fixed time duration $t_{win}$. That is, let $f_{s}$ denote the signal sampling rate, then $N=t_{win}{\cdot}f_{s}$. In this way, the frequency resolution of STFT will be constant for arbitrary sampling rates:

$$ ${\Delta}f=\frac{f_{s}}{N}=\frac{f_{s}}{t_{win}{\cdot}f_{s}}=\frac{1}{t_{win}}$ (1) $$

where ${\Delta}f$ is the frequency gap between adjacent frequency grid. On the other hand, the STFT hop size is also mapped to fixed time duration $t_{hop}$, such that signals with the same time duration will have spectrograms with the same time shape regardless of the sampling rate.

To deal with the variable frequency shape, FISHER emphasizes the importance of sub-band information and considers it as the building blocks of the overall information. On the one hand, the information gain of a higher sampling rate lies in the additional sub-bands as depicted in Figure [3](https://arxiv.org/html/2507.16696v1#S1.F3). As known, all sensors employ anti-aliasing filtering to prevent signal aliasing. Therefore, the spectrogram does not contain any information about frequencies higher than half the sampling rate. On the other hand, sampling rates of common large-scale datasets, i.e. 16 kHz, 32 kHz, 44.1 kHz and 48 kHz, are integer multiples of a base frequency $f_{base}$, such as 2 kHz and 4 kHz, making sub-band a natural modeling unit for data with multiple sampling rates. Therefore, we take the sub-band as the unit for modeling, and build up the information of the whole spectrogram by concatenating sub-band information just like building blocks. That is, the signal representation is the concatenation of sub-band representations. The higher the sampling rate, the more informative the signal representation is.

We now describe the sub-band modeling process in detail, which is illustrated in Figure [2](https://arxiv.org/html/2507.16696v1#S1.F2). A batch of signals are first resampled to a batch-specific sampling rate $sr_{batch}$ to align the spectrogram shape for batching, where $sr_{batch}$ is randomly selected from all the harmonics of $f_{base}$ that are less than a maximum frequency $f_{max}$. The aligned signals are then converted to log amplitude STFT spectrograms of shape $(B,\ T,\ F)$, where $B$ is the batch size, and $T$ and $F$ are the time and frequency shapes respectively. Each spectrogram is then split into sub-bands with bandwidth $w$ and concatenated along the batch axis, thereby transferring the variability from the frequency axis to the batch axis. These sub-bands have a shape of $(B{\times}n,\ T,\ w)$, where $w=f_{base}{\cdot}t_{win}$ is the frequency gap on the spectrogram corresponding to $f_{base}$, and $n=\lfloor\frac{F}{w}\rfloor$ is the number of sub-bands. Then the sub-bands are processed individually by the network and their representations are concatenated afterwards.

### II-B Model Architecture

FISHER adopts an encoder-decoder architecture with a teacher student self distillation scheme for pre-training, which has been demonstrated effective in multiple self-supervised learning (SSL) models [9, 13, 14, 15]. FISHER comprises three sub-networks: a student encoder $E_{stu}$, a student decoder $D_{stu}$ and a teacher encoder $E_{tea}$. Both encoders are deep Transformer [16] networks and $D_{stu}$ is a shallow convolutional neural networks (CNN). $E_{stu}$ and $E_{tea}$ have the identical structure with the parameters of $E_{tea}$ being the EMA of the parameters of $E_{stu}$:

$$ $\theta_{E_{tea}}=\tau\theta_{E_{tea}}+(1-\tau)\theta_{E_{stu}}$ (2) $$

where $\theta_{E_{tea}}$ and $\theta_{E_{stu}}$ are the parameters of $E_{tea}$ and $E_{stu}$ respectively, and $\tau$ is the EMA decay factor. After applying band splitting on the STFT spectrogram, each sub-band is further split into a patch sequences [17]. For the student branch, the patch sequence is masked by inverse block masking adopted in EAT [9] with a large mask ratio of 80%, and masked patches are discarded. Here we also employ the mask cloning strategy in EAT to efficiently increase the batch size, while we constrain the maximum number of cloned sub-bands in a batch as $m_{b}$, rather than adopting fixed clone ratio as EAT. The unmasked patches are then appended with a [CLS] token at front and sent into $E_{stu}$. After encoded by $E_{stu}$, the output of the [CLS] token is selected as the student sub-band representation $s_{band}$, and the output of these unmasked patches are merged with the masked parts in the original spatial locations, where the values of the masked parts are initialized from normal Gaussian. $D_{stu}$ takes in the merged sequence and outputs the student patch representation $s_{patch}$. For the teacher branch, $E_{tea}$ processes the unmasked patch sequence, and the embeddings of all its layers are averaged to derive the teacher sub-band representation $t_{band}$ and teacher patch representation $t_{patch}$. The self distillation process is supervised from both the sub-band level and the patch level:

$$ $\begin{cases}&L_{band}=\|s_{band}-sg(t_{band})\|_{2}^{2}\\ &L_{patch}=\|s_{patch}-sg(t_{patch})\|_{2}^{2}\end{cases}$ (3) $$

where $sg(\cdot)$ denotes stop gradient. The final loss is the combination of the two losses:

$$ $L=L_{band}+L_{patch}$ (4) $$

It is noted that sub-bands are processed individually during training. During inference, only $E_{stu}$ is employed and its sub-band representations are concatenated to form the overall representation of the signal.

## III RMIS Benchmark

To evaluate the comprehensive representation capability of industrial signals, we develop the RMIS benchmark, which is still under development and currently comprises five anomaly detection datasets and 13 fault diagnosis datasets, whose key features are presented in Table [I](https://arxiv.org/html/2507.16696v1#S3.T1). To demonstrate the intrinsic versatility of the model, we evaluate the model by KNN inference without fine-tuning on any dataset. For each task, we calculate the arithmetic mean of corresponding dataset scores as the task score, and the overall benchmark score is the arithmetic mean of two task scores to eliminate the impact of data imbalance. We will open-source the RMIS benchmark in the near future.

**TABLE I: Key features of datasets in the RMIS benchmark. Split denotes whether an official train-test split is provided for the dataset.**
| Dataset | Modality | Num | Sampling | Volume | Split |
| --- | --- | --- | --- | --- | --- |
| Class | Rate |  |  |  |  |
| DCASE20 | Sound | 2 | 16 kHz | 153 h | ✓ |
| DCASE21 | Sound | 2 | 16 kHz | 165 h | ✓ |
| DCASE22 | Sound | 2 | 16 kHz | 139 h | ✓ |
| DCASE23 | Sound | 2 | 16 kHz | 50 h | ✓ |
| DCASE24 | Sound | 2 | 16 kHz | 49 h | ✓ |

### III-A Anomaly Detection

Anomaly detection is to predict whether a signal is normal or anomalous when no anomalies are provided for training, which emphasizes the scarcity of fault data. Since anomaly detection is mainly studied on acoustic signals, we evaluate the model on the datasets of the annual DCASE anomalous sound detection (ASD) challenge, including DCASE20 [18, 19, 20], DCASE21 [21, 22, 23], DCASE22 [24, 23, 25], DCASE23 [24, 23, 26, 27] and DCASE24 [24, 23, 28, 29]. We use the official split and evaluate the model by the challenge criteria, which is based on the area under the receiver operating characteristic (ROC) curve (AUC). For each dataset, we report the harmonic mean over the development subset and the evaluation subset. The readers are advised to refer to these cited works for further information of the DCASE datasets. All models adopt the identical KNN-based anomaly detection pipeline as AnoPatch [2], which is briefly introduced in Appendix [A](https://arxiv.org/html/2507.16696v1#A1) to make the paper more self-contained.

Figure: (a) normal
Refer to caption: extracted/6643448/imgs/vib_leak/wt_normal.png

### III-B Fault Diagnosis

Fault diagnosis is to identify the specific fault type or health state of a signal with labeled data provided in advance. Fault diagnosis datasets commonly focus on small components where fault data can be readily collected at low cost, such as bearing and gearbox. The fault diagnosis part in the RMIS benchmark is sourced from seven publicly available datasets: IDMT-ISA-COMPRESSED-AIR (IICA) [30], IDMT-ISA-ELECTRIC-ENGINE (IIEE) [31], the WT-planetary-gearbox-dataset (WTPG) [32], the Machinery Fault Dataset (MaFaulDa) [33], the UM-GearEccDataset (UMGED) [34], the SDUST dataset [35, 36, 37, 38] and the PU Dataset [39]. Detailed information of these datasets is presented in Appendix [B](https://arxiv.org/html/2507.16696v1#A2). To reveal the modality specific performance, we first divide these data sources by the modality, resulting in 13 datasets. For each dataset, we flatten all multi-channel signals into single-channel signals and split them into segments if they are longer than the analysis segment length, which is selected as 10 s to line up with baseline models. For KNN inference, $k$ is default to 5(^1^11We set k as 10 on IIEE for better performance.) for all models to form fair comparison. All datasets are evaluated by macro-average accuracy.

To ensure proper difficulty of the task, the RMIS benchmark conduct sealed train-test split for datasets without official train-test split, where segments of the same recording can not be partitioned into both the training and the test set. As presented in Figure [4](https://arxiv.org/html/2507.16696v1#S3.F4), industrial signals are sometimes extremely stationary along the time due to the constant working condition, causing the split segments to be highly identical. If random splitting were adopted, these highly similar segments would appear in both the training and test sets, causing the task to be exceedingly simple. Therefore, we trace the original recording and allocate all its segments into either the train set or the test set(^2^22For the PU dataset, the signal under each working condition is recorded for 20 times, resulting in 20 highly identical segments. Thus, we allocate these segments based on the working condition.). The train-test split ratio is default to 1:1, except for UMGED datasets which are 4:1. We ensure that all classes are presented in both sets and evaluate the model under 10 different splits. To eliminate the impact of fixed split ratio, the split ratio is further analyzed in Section [IV-E](https://arxiv.org/html/2507.16696v1#S4.SS5).

**TABLE II: Unique Hyperparameters of FISHER under Different Scales**
| Scale | Num | $f_{base}$ | $w$ | Hidden | Num | Batch | $m_{b}$^1 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Param | Size | Head | Size^1 |  |  |  |  |
| tiny | 5.5M | 4000 | 100 | 192 | 3 | 24 | 64 |
| mini | 10M | 4000 | 100 | 256 | 4 | 32 | 64 |
| small | 22M | 2000 | 50 | 384 | 6 | 16 | 128 |

## IV Experiment

### IV-A Details of FISHER

We currently release three versions of FISHER, namely FISHER-tiny (5.5M), FISHER-mini (10M) and FISHER-small (22M), which are in line with the hierarchy of ViT. Larger versions will be released in the near future. Table [II](https://arxiv.org/html/2507.16696v1#S3.T2) lists unique hyperparameters for each version. As for the shared hyperparameters, $t_{win}$ and $t_{hop}$ are fixed as 25 ms and 10 ms, $f_{max}$ is 32 kHz. All models contain 12 layers and the patch size is $16{\times}16$. All models are pre-trained on the combined dataset of Audioset [40], Freesound(^3^33[https://freesound.org/](https://freesound.org/)), MTG-Jamendo [41] and Music4all [42] with a total volume of 17k hours. We train each model for 400k steps on either four NVIDIA RTX 3090 GPUs or four NVIDIA RTX A6000 GPUs. For each model, we adopt a warm-up scheduler with a peak learning rate of 0.0005 and a warm-up step of 40k. During evaluation, FISHER directly processes the original signal without resampling.

### IV-B Baselines

We first compare common SSL models on the RMIS benchmark, where speech models are generally lower than audio models due to the inconsistency [2]. Thus, we employ five top audio pre-trained models as baseline models, namely AudioMAE [43], BEATs [10], EAT [9], CED [44] and DaSheng [45]. Details of these baselines are presented in Appendix [C](https://arxiv.org/html/2507.16696v1#A3). Since all baselines adopt 16 kHz input, industrial signals are resampled to 16 kHz before processed by these baselines. The detection pipelines of baselines are identical with FISHER.

**TABLE III: Results on the RMIS Benchmark ($\uparrow$)**
| Model | Scale | Anomaly Detection | Fault Diagnosis | All |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| DCASE | Mean | IICA | IIEE | WTPG | MaFaulDa | SDUST | UMGED | PU | Mean |  |  |  |  |  |  |  |  |  |  |  |  |  |
| 20 | 21 | 22 | 23 | 24 | sound | vib | bearing | gear | sound | vib | vol | cur | vib | cur |  |  |  |  |  |  |  |  |
| AudioMAE | base | 63.86 | 54.86 | 55.95 | 57.26 | 55.29 | 57.44 | 58.84 | 54.23 | 63.97 | 47.51 | 81.01 | 59.77 | 98.18 | 6.90 | 10.94 | 10.00 | 11.55 | 56.06 | 35.25 | 45.71 | 51.58 |
| BEATs | iter3 | 74.20 | 60.50 | 58.95 | 62.41 | 56.37 | 62.49 | 77.93 | 68.22 | 70.57 | 52.88 | 84.46 | 62.38 | 98.23 | 5.97 | 10.32 | 10.96 | 17.23 | 74.29 | 48.54 | 52.46 | 57.47 |
| EAT | base30 | 71.07 | 57.19 | 58.14 | 59.44 | 56.10 | 60.39 | 63.40 | 78.68 | 74.50 | 61.57 | 89.77 | 61.53 | 95.30 | 6.78 | 10.30 | 13.74 | 19.03 | 74.19 | 49.01 | 53.68 | 57.03 |
| large | 72.61 | 57.18 | 58.86 | 60.31 | 57.44 | 61.28 | 61.09 | 70.42 | 70.65 | 62.76 | 89.71 | 56.25 | 95.35 | 9.00 | 10.89 | 11.65 | 16.01 | 75.47 | 48.25 | 52.12 | 56.70 |  |
| CED | tiny | 66.99 | 56.17 | 56.75 | 60.19 | 56.40 | 59.30 | 47.93 | 75.33 | 89.43 | 53.80 | 84.56 | 59.02 | 96.06 | 8.09 | 12.96 | 10.32 | 12.65 | 71.58 | 49.97 | 51.67 | 55.48 |
| mini | 67.48 | 56.35 | 56.59 | 60.05 | 57.44 | 59.58 | 50.21 | 86.77 | 89.26 | 56.98 | 84.87 | 59.24 | 95.51 | 8.37 | 11.66 | 10.14 | 12.43 | 71.40 | 48.32 | 52.70 | 56.14 |  |
| small | 67.50 | 56.65 | 56.87 | 60.76 | 57.88 | 59.93 | 50.49 | 82.61 | 91.20 | 54.67 | 84.66 | 58.00 | 96.12 | 8.04 | 12.15 | 10.26 | 13.18 | 72.82 | 49.22 | 52.57 | 56.25 |  |
| base | 67.60 | 56.67 | 56.95 | 60.99 | 57.88 | 60.02 | 52.76 | 92.46 | 89.20 | 57.63 | 85.95 | 60.20 | 95.83 | 8.12 | 11.56 | 10.03 | 13.61 | 72.16 | 50.85 | 53.87 | 56.95 |  |
| DaSheng | base | 69.25 | 57.27 | 56.01 | 60.95 | 56.39 | 59.97 | 74.43 | 98.85 | 46.85 | 63.45 | 92.09 | 61.38 | 98.30 | 5.64 | 4.45 | 11.72 | 20.10 | 74.81 | 42.56 | 53.43 | 56.70 |
| 0.6B | 68.35 | 56.76 | 55.35 | 59.85 | 56.33 | 59.33 | 75.14 | 99.15 | 45.41 | 57.25 | 91.12 | 58.21 | 98.30 | 5.26 | 4.28 | 11.66 | 14.65 | 75.83 | 42.85 | 52.24 | 55.78 |  |
| 1.2B | 69.52 | 57.06 | 55.82 | 61.13 | 56.32 | 59.97 | 73.19 | 98.75 | 48.89 | 58.10 | 90.83 | 55.50 | 97.97 | 5.42 | 4.51 | 11.76 | 17.31 | 76.11 | 44.30 | 52.51 | 56.24 |  |
| FISHER | tiny | 70.86 | 58.76 | 56.40 | 58.62 | 53.63 | 59.65 | 85.30 | 98.77 | 83.84 | 75.79 | 92.58 | 71.81 | 99.20 | 11.88 | 23.58 | 21.12 | 28.27 | 72.22 | 56.09 | 63.11 | 61.38 |
| mini | 70.19 | 58.40 | 57.63 | 61.07 | 54.60 | 60.38 | 86.02 | 99.10 | 84.74 | 75.74 | 93.20 | 70.10 | 98.55 | 13.67 | 20.52 | 22.92 | 29.94 | 72.01 | 55.58 | 63.24 | 61.81 |  |
| small | 71.04 | 59.48 | 59.64 | 62.64 | 55.63 | 61.69 | 90.23 | 98.96 | 86.77 | 72.61 | 95.86 | 76.35 | 99.08 | 12.72 | 18.35 | 17.41 | 25.54 | 74.92 | 54.19 | 63.31 | 62.50 |  |

### IV-C Results on RMIS Benchmark

All models are evaluated on five anomaly detection datasets and 13 fault detection datasets of the RMIS benchmark. Table [III](https://arxiv.org/html/2507.16696v1#S4.T3) presents the numerical results, while Figure [5](https://arxiv.org/html/2507.16696v1#S5.F5) further depicts the performance curve on each dataset. On anomaly detection tasks, FISHER is generally the second best model, only slightly behind BEATs, the best performing model in previous DCASE challenges [46, 47]. On fault diagnosis tasks, FISHER outperforms all baselines by a wide margin, where even FISHER-tiny is 9.24% higher than the best baseline model. This is mainly attributed to the capability of FISHER to utilize the full bandwidth of the original signal, whereas baseline models must downsample the signal, which consequently results in information loss.

As for the general performances, FISHER is the most versatile model for industrial signal health management tasks. FISHER-small achieves the best overall score of 62.50% on the RMIS benchmark, demonstrating dominant performances. Although BEATs slightly outperforms FISHER-small by 0.8% on anomaly detection tasks, it falls significantly short on fault diagnosis tasks and the overall score is 5.03% lower than FISHER-small. Moreover, FISHER achieves better performances with much smaller model size. Even FISHER-tiny, with merely 5.5M parameters, surpasses all baseline models (most of which with 90M parameters) by at least 3.91%.

### IV-D Efficient Scaling

In this paper, we also investigate how to effectively scale on signal health management tasks, which has rarely been discussed in previous works. As known, the scaling law [7] emphasizes key factors for developing large AI models, namely model size, data volume, computational resource and test time. By scaling these factors up, the model performance will grow steadily and is predictable. While model size, data volume and computational resource must be scaled up cohesively in pre-training, TTS [48] focuses on adaptively adjusting test time resources and strategies for better performance.

On the RMIS benchmark, FISHER showcases much more efficient scaling properties against baseline models. As presented in Figure [1b](https://arxiv.org/html/2507.16696v1#S1.F1.sf2), the scaling curve of FISHER is constantly above the curves of all baseline models. Compared with EAT which also adopts a teacher student EMA pre-training scheme, FISHER-small outperforms EAT-base30 by 5.47%. Compared with CED which has a similar scale hierarchy, FISHER outperforms CED by 5.90%, 5.67% and 6.25% for tiny, mini and small respectively. This demonstrates the superiority of the pre-training scheme of FISHER for signal representation.

It is worthy to note that directly scaling up in pre-training has encountered bottlenecks. For all baselines, the performance grows steadily as the model size scales from tiny to base (around 100M), yet unexpectedly drops as the scaling continues, which seems to contradict the scaling law. As a comparison, on general audio understanding benchmarks such as Audioset [40], HEAR [49] and MMAR [50], the performance can further improve as the model size scales up from 100M to billions. We believe that this is due to the poor quality of signal data for pre-training large-scale models. As presented in Figure [4](https://arxiv.org/html/2507.16696v1#S3.F4), industrial signals are sometimes extremely stationary and invariant. Despite the seemingly ample volume of signal data incorporated in the pre-training dataset, these data exhibit a high level of similarity and only a small amount of data remains after deduplication. These efficient data are only sufficient for training models of limited scale, and the inflection point is probably situated around 100M. If the model size is further increased, the model will be overfitted on other types of data, causing the performance on signal representation tasks to drop gradually.

Therefore, greater emphasis should be accorded to data preparation when scaling up the model. The training data should be enlarged with more unique and non-duplicated signals, such as signals of new modalities, new machinery, new working conditions, etc. Thus, it is necessary to carry out data cleaning on a broader scale and with finer granularity. Meanwhile, due to the high tendency of signal repetition, the proportion of data volume in the training configuration of the signal model should be larger than that of the speech model. As an example, FISHER-tiny trained with excessive data, performs unexpectedly well on the RMIS benchmark.

On the other hand, TTS appears to be effective on the RMIS benchmark. As an example, FISHER utilizes extra resources for inferring signals with higher sampling rates, which can be regarded as a form of TTS. We believe health management tasks require deep thinking and long reasoning, just like the way how skilled workers take time to diagnose the malfunction. Therefore, TTS is believed to be a potential breakthrough point for health management tasks.

**TABLE IV: Area under the Multi-Split Curve for Datasets without Official Split ($\uparrow$)**
| Model | Scale | IICA | WTPG | MaFaulDa | SDUST | UMGED | PU | mean |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| sound | vib | bearing | gear | sound | vib | vol | cur | vib | cur |  |  |  |  |  |
| AudioMAE | base | 56.11 | 62.88 | 45.54 | 78.26 | 58.34 | 95.24 | 7.14 | 11.76 | 9.28 | 10.18 | 54.48 | 34.97 | 43.68 |
| BEATs | iter3 | 74.00 | 68.60 | 50.15 | 81.81 | 60.61 | 95.20 | 7.02 | 11.58 | 9.82 | 13.67 | 71.39 | 45.50 | 49.11 |
| EAT | base30 | 60.97 | 70.70 | 58.52 | 87.04 | 59.25 | 91.02 | 7.52 | 9.92 | 11.63 | 15.23 | 70.82 | 47.45 | 49.17 |
| large | 58.51 | 66.08 | 59.62 | 87.11 | 54.53 | 91.29 | 8.84 | 11.26 | 10.23 | 12.85 | 71.68 | 46.18 | 48.18 |  |
| CED | tiny | 46.91 | 85.57 | 50.46 | 81.97 | 57.46 | 93.01 | 7.72 | 13.50 | 9.31 | 10.74 | 68.35 | 47.57 | 47.71 |
| mini | 48.71 | 86.30 | 54.59 | 82.29 | 57.44 | 92.57 | 7.81 | 12.38 | 9.30 | 10.61 | 68.43 | 46.40 | 48.07 |  |
| small | 49.03 | 87.47 | 51.55 | 82.32 | 56.83 | 93.18 | 7.57 | 12.77 | 9.43 | 11.21 | 68.87 | 47.76 | 48.17 |  |
| base | 50.59 | 85.85 | 54.39 | 83.34 | 58.77 | 93.02 | 7.47 | 12.25 | 9.35 | 11.28 | 68.73 | 48.65 | 48.64 |  |
| DaSheng | base | 71.16 | 47.30 | 59.33 | 90.44 | 59.57 | 94.34 | 6.20 | 4.81 | 10.20 | 15.64 | 72.19 | 41.00 | 47.68 |
| 0.6B | 71.83 | 46.17 | 54.03 | 89.16 | 56.87 | 93.69 | 6.17 | 4.79 | 10.27 | 11.84 | 73.15 | 41.29 | 46.61 |  |
| 1.2B | 69.85 | 49.20 | 54.65 | 88.47 | 54.07 | 93.73 | 6.36 | 4.90 | 10.13 | 13.67 | 73.24 | 42.49 | 46.73 |  |
| FISHER | tiny | 82.12 | 79.37 | 72.37 | 89.78 | 68.68 | 97.98 | 11.46 | 18.79 | 16.82 | 22.08 | 68.68 | 52.97 | 56.76 |
| mini | 82.84 | 79.68 | 72.33 | 90.72 | 66.65 | 97.15 | 13.49 | 16.71 | 18.06 | 22.79 | 68.93 | 52.27 | 56.80 |  |
| small | 86.38 | 82.72 | 69.12 | 93.33 | 72.94 | 98.08 | 12.28 | 15.43 | 14.31 | 19.83 | 71.19 | 51.57 | 57.27 |  |

### IV-E Multiple Split Ratios

In the RMIS benchmark, 12 out of 13 fault diagnosis datasets do not provide official split. Since previous works adopt various split ratio, we further analyze the model performance under multiple split ratios to eliminate the impact of split ratio. Specifically, we first plot the performance curve under variable split ratios ranging from 0.05 to 0.95 (train set ratio), which is presented in Figure [6](https://arxiv.org/html/2507.16696v1#S5.F6). For each split ratio, the model is still evaluated under 10 different train-test splits. Then we estimate the area under the multi-split curve based on the trapezoidal rule, where the bigger the area is, the better the model is. As presented in Table [IV](https://arxiv.org/html/2507.16696v1#S4.T4), FISHER still holds significant advantages under multiple split ratios, achieving the largest area under the multi-split curve. Compared with baselines smaller than ViT-base (90M), FISHER is better than CED on all datasets except WTPG. Compared with baselines equal or larger than ViT-base (90M), FISHER-small with much smaller model size, still surpasses all models on 10 out of 12 datasets. This further demonstrates the superiority of FISHER.

Meanwhile, there is a strong correlation between the performance under fixed split ratio and the performance under multiple split ratios, since the score under fixed split ratio and the area under the multi-split curve has a high Pearson correlation of 0.999. Therefore, it is proper to evaluate the model under a balanced split ratio, and thus we fix the ratio as 1:1 in the RMIS benchmark if not otherwise stated.

## V Conclusion

In the paper, we hypothesized that the heterogeneous M5 industrial signals can be modeled in a unified model due to the intrinsic similarity. As a result, we proposed FISHER, which models the information gain of higher sampling rate as the concatenation of sub-band information. We also developed the RMIS benchmark to evaluate versatility on different health management tasks, where FISHER excels all baselines by a wide margin with much efficient scaling properties. How to derive more powerful representation of industrial signal and effectively scale on health management tasks will be the focus of our future work.

## References

- [1]
A. Jiang, X. Zheng, B. Han, Y. Qiu, P. Fan, W.-Q. Zhang, C. Lu, and J. Liu, “Adaptive prototype learning for anomalous sound detection with partially known attributes,” in *ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*.   IEEE, 2025, pp. 1–5.
- [2]
A. Jiang, B. Han, Z. Lv, Y. Deng, W.-Q. Zhang, X. Chen, Y. Qian, J. Liu, and P. Fan, “Anopatch: Towards better consistency in machine anomalous sound detection,” in *Interspeech 2024*, 2024, pp. 107–111.
- [3]
A. Jiang, Y. Shi, P. Fan, W.-Q. Zhang, and J. Liu, “Coopasd: Cooperative machine anomalous sound detection with privacy concerns,” *arXiv preprint arXiv:2408.14753*, 2024.
- [4]
H. Peng, J. Liu, J. Du, J. Gao, and W. Wang, “Bearllm: A prior knowledge-enhanced bearing health management framework with unified vibration signal representation,” in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 39, no. 19, 2025, pp. 19 866–19 874.
- [5]
G. Wang, D. Liu, and L. Cui, “Auto-embedding transformer for interpretable few-shot fault diagnosis of rolling bearings,” *IEEE Transactions on Reliability*, vol. 73, no. 2, pp. 1270–1279, 2023.
- [6]
B. Wang, Y. Lei, N. Li, and N. Li, “A hybrid prognostics approach for estimating remaining useful life of rolling element bearings,” *IEEE Transactions on Reliability*, vol. 69, no. 1, pp. 401–412, 2018.
- [7]
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” *arXiv preprint arXiv:2001.08361*, 2020.
- [8]
X. Zheng, A. Jiang, B. Han, Y. Qian, P. Fan, J. Liu, and W.-Q. Zhang, “Improving anomalous sound detection via low-rank adaptation fine-tuning of pre-trained audio models,” in *2024 IEEE Spoken Language Technology Workshop (SLT)*.   IEEE, 2024, pp. 969–974.
- [9]
W. Chen, Y. Liang, Z. Ma, Z. Zheng, and X. Chen, “Eat: Self-supervised pre-training with efficient audio transformer,” *arXiv preprint arXiv:2401.03497*, 2024.
- [10]
S. Chen, Y. Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, W. Che, X. Yu, and F. Wei, “BEATs: Audio pre-training with acoustic tokenizers,” in *Proceedings of the 40th International Conference on Machine Learning*, ser. Proceedings of Machine Learning Research, vol. 202.   PMLR, 23–29 Jul 2023, pp. 5178–5193.
- [11]
P.-Y. Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked autoencoders that listen,” *Advances in Neural Information Processing Systems*, vol. 35, pp. 28 708–28 720, 2022.
- [12]
H. Dinkel, Z. Yan, Y. Wang, J. Zhang, Y. Wang, and B. Wang, “Scaling up masked audio encoder learning for general audio classification,” *arXiv preprint arXiv:2406.06992*, 2024.
- [13]
A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, “Data2vec: A general framework for self-supervised learning in speech, vision and language,” in *International conference on machine learning*.   PMLR, 2022, pp. 1298–1312.
- [14]
M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, R. Howes, P.-Y. Huang, H. Xu, V. Sharma, S.-W. Li, W. Galuba, M. Rabbat, M. Assran, N. Ballas, G. Synnaeve, I. Misra, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski, “Dinov2: Learning robust visual features without supervision,” 2023.
- [15]
K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2022, pp. 16 000–16 009.
- [16]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” *Advances in neural information processing systems*, vol. 30, 2017.
- [17]
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in *International Conference on Learning Representations*, 2021.
- [18]
Y. Koizumi, S. Saito, H. Uematsu, N. Harada, and K. Imoto, “ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection,” in *Proceedings of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)*, November 2019, pp. 308–312.
- [19]
H. Purohit, R. Tanabe, T. Ichige, T. Endo, Y. Nikaido, K. Suefusa, and Y. Kawaguchi, “MIMII Dataset: Sound dataset for malfunctioning industrial machine investigation and inspection,” in *Proceedings of the Detection and Classification of Acoustic Scenes and Events 2019 Workshop (DCASE2019)*, November 2019, pp. 209–213.
- [20]
Y. Koizumi, Y. Kawaguchi, K. Imoto, T. Nakamura, Y. Nikaido, R. Tanabe, H. Purohit, K. Suefusa, T. Endo, M. Yasuda, and N. Harada, “Description and discussion on DCASE2020 challenge task2: Unsupervised anomalous sound detection for machine condition monitoring,” in *Proceedings of the Detection and Classification of Acoustic Scenes and Events 2020 Workshop (DCASE2020)*, November 2020, pp. 81–85.
- [21]
R. Tanabe, H. Purohit, K. Dohi, T. Endo, Y. Nikaido, T. Nakamura, and Y. Kawaguchi, “MIMII DUE: Sound dataset for malfunctioning industrial machine investigation and inspection with domain shifts due to changes in operational and environmental conditions,” *IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)*, pp. 21–25, 2021.
- [22]
Y. Kawaguchi, K. Imoto, Y. Koizumi, N. Harada, D. Niizumi, K. Dohi, R. Tanabe, H. Purohit, and T. Endo, “Description and discussion on dcase 2021 challenge task 2: Unsupervised anomalous detection for machine condition monitoring under domain shifted conditions,” in *Proceedings of the 6th Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021)*, Barcelona, Spain, November 2021, pp. 186–190.
- [23]
N. Harada, D. Niizumi, D. Takeuchi, Y. Ohishi, M. Yasuda, and S. Saito, “ToyADMOS2: Another dataset of miniature-machine operating sounds for anomalous sound detection under domain shift conditions,” in *Proceedings of the 6th Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021)*, Barcelona, Spain, November 2021, pp. 1–5.
- [24]
K. Dohi, T. Nishida, H. Purohit, R. Tanabe, T. Endo, M. Yamamoto, Y. Nikaido, and Y. Kawaguchi, “MIMII DG: Sound dataset for malfunctioning industrial machine investigation and inspection for domain generalization task,” in *Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop (DCASE2022)*, Nancy, France, November 2022, pp. 1–5.
- [25]
K. Dohi, K. Imoto, N. Harada, D. Niizumi, Y. Koizumi, T. Nishida, H. Purohit, R. Tanabe, T. Endo, M. Yamamoto, and Y. Kawaguchi, “Description and discussion on DCASE 2022 challenge task 2: Unsupervised anomalous sound detection for machine condition monitoring applying domain generalization techniques,” in *Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop (DCASE2022)*, Nancy, France, November 2022, pp. 1–5.
- [26]
K. Dohi, K. Imoto, N. Harada, D. Niizumi, Y. Koizumi, T. Nishida, H. Purohit, R. Tanabe, T. Endo, and Y. Kawaguchi, “Description and discussion on DCASE 2023 challenge task 2: First-shot unsupervised anomalous sound detection for machine condition monitoring,” *In arXiv e-prints: 2305.07828*, 2023.
- [27]
N. Harada, D. Niizumi, D. Takeuchi, Y. Ohishi, and M. Yasuda, “First-shot anomaly detection for machine condition monitoring: A domain generalization baseline,” *In arXiv e-prints: 2303.00455*, 2023.
- [28]
T. Nishida, N. Harada, D. Niizumi, D. Albertini, R. Sannino, S. Pradolini, F. Augusti, K. Imoto, K. Dohi, H. Purohit, T. Endo, and Y. Kawaguchi, “Description and discussion on DCASE 2024 challenge task 2: First-shot unsupervised anomalous sound detection for machine condition monitoring,” *In arXiv e-prints: 2406.07250*, 2024.
- [29]
N. Harada, D. Niizumi, D. Takeuchi, Y. Ohishi, and M. Yasuda, “First-shot anomaly detection for machine condition monitoring: A domain generalization baseline,” *Proceedings of 31st European Signal Processing Conference (EUSIPCO)*, pp. 191–195, 2023.
- [30]
D. Johnson, J. Kirner, S. Grollmisch, and J. Liebetrau, “Compressed air leakage detection using acoustic emissions with neural networks,” in *INTER-NOISE and NOISE-CON Congress and Conference Proceedings*, vol. 261, no. 1.   Institute of Noise Control Engineering, 2020, pp. 5662–5673.
- [31]
S. Grollmisch, J. Abeßer, J. Liebetrau, and H. Lukashevich, “Sounding industry: Challenges and datasets for industrial sound analysis,” in *2019 27th European signal processing conference (EUSIPCO)*.   IEEE, 2019, pp. 1–5.
- [32]
D. Liu, L. Cui, and W. Cheng, “A review on deep learning in planetary gearbox health state recognition: methods, applications, and dataset publication,” *Measurement Science and Technology*, vol. 35, no. 1, p. 012002, 2023.
- [33]
F. M. L. Ribeiro, “Mafaulda-machinery fault database,” 2016. [Online]. Available: [https://www02.smt.ufrj.br/~offshore/mfs/page_01.html#SEC2](https://www02.smt.ufrj.br/~offshore/mfs/page_01.html#SEC2)
- [34]
J. Li, H. Chen, X.-B. Wang, and Z.-X. Yang, “A comprehensive gear eccentricity dataset with multiple fault severity levels: Description, characteristics analysis, and fault diagnosis applications,” *Mechanical Systems and Signal Processing*, vol. 224, p. 112068, 2025.
- [35]
J. Wang, X. Zhang, Z. Zhang, B. Han, X. Jiang, H. Bao, and X. Jiang, “Attention guided multi-wavelet adversarial network for cross domain fault diagnosis,” *Knowledge-Based Systems*, vol. 284, p. 111285, 2024.
- [36]
X. Zhang, J. Wang, Z. Zhang, B. Han, H. Bao, and X. Jiang, “Integrated decision-making with adaptive feature weighting adversarial network for multi-target domain compound fault diagnosis of machinery,” *Advanced Engineering Informatics*, vol. 62, p. 102730, 2024.
- [37]
B. Han, X. Jiang, J. Wang, and Z. Zhang, “A novel domain adaptive fault diagnosis method for bearings based on unbalance data generation,” *IEEE Transactions on Instrumentation and Measurement*, vol. 72, pp. 1–11, 2023.
- [38]
B. Han, Z. Yang, Z. Zhang, H. Bao:, J. Wang, Z. Liu, and S. Li, “A novel rolling bearing fault diagnosis method based on generalized nonlinear spectral sparsity,” *Measurement*, vol. 198, p. 111131, 2022.
- [39]
C. Lessmeier, J. K. Kimotho, D. Zimmer, and W. Sextro, “Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A benchmark data set for data-driven classification,” in *PHM society European conference*, vol. 3, no. 1, 2016.
- [40]
J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in *2017 IEEE international conference on acoustics, speech and signal processing (ICASSP)*.   IEEE, 2017, pp. 776–780.
- [41]
D. Bogdanov, M. Won, P. Tovstogan, A. Porter, and X. Serra, “The mtg-jamendo dataset for automatic music tagging,” in *Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML 2019)*, Long Beach, CA, United States, 2019.
- [42]
I. A. P. Santana, F. Pinhelli, J. Donini, L. Catharin, R. B. Mangolin, V. D. Feltrim, M. A. Domingues *et al.*, “Music4all: A new music database and its applications,” in *2020 International Conference on Systems, Signals and Image Processing (IWSSIP)*.   IEEE, 2020, pp. 399–404.
- [43]
P.-Y. Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked autoencoders that listen,” in *NeurIPS*, 2022.
- [44]
H. Dinkel, Y. Wang, Z. Yan, J. Zhang, and Y. Wang, “Ced: Consistent ensemble distillation for audio tagging,” in *ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, 2024.
- [45]
H. Dinkel, Z. Yan, Y. Wang, J. Zhang, Y. Wang, and B. Wang, “Scaling up masked audio encoder learning for general audio classification,” in *Interspeech 2024*, 2024.
- [46]
Z. Lv, A. Jiang, B. Han, Y. Liang, Y. Qian, X. Chen, J. Liu, and P. Fan, “Aithu system for first-shot unsupervised anomalous sound detection,” DCASE2024 Challenge, Tech. Rep., June 2024.
- [47]
A. Jiang, X. Zheng, Y. Qiu, W. Zhang, B. Chen, P. Fan, W.-Q. Zhang, C. Lu, and J. Liu, “Thuee system for first-shot unsupervised anomalous sound detection,” DCASE2024 Challenge, Tech. Rep., June 2024.
- [48]
Q. Zhang, F. Lyu, Z. Sun, L. Wang, W. Zhang, W. Hua, H. Wu, Z. Guo, Y. Wang, N. Muennighoff *et al.*, “A survey on test-time scaling in large language models: What, how, where, and how well?” *arXiv preprint arXiv:2503.24235*, 2025.
- [49]
J. Turian, J. Shier, H. R. Khan, B. Raj, B. W. Schuller, C. J. Steinmetz, C. Malloy, G. Tzanetakis, G. Velarde, K. McNally *et al.*, “Hear: Holistic evaluation of audio representations,” in *NeurIPS 2021 Competitions and Demonstrations Track*.   PMLR, 2022, pp. 125–145.
- [50]
Z. Ma, Y. Ma, Y. Zhu, C. Yang, Y.-W. Chao, R. Xu *et al.*, “Mmar: A challenging benchmark for deep reasoning in speech, audio, music, and their mix,” *arXiv preprint arXiv:2505.13032*, 2025.

## Appendix A Details of Anomaly Detection Tasks

For ASD tasks in the RMIS benchmark, all models adopt the identical KNN-based anomaly detection pipeline as AnoPatch [2] after extracting the embeddings, where normal embeddings from the training set form memory banks, and the anomaly score of each query embedding from the test set is the average distance to the nearest neighbors. The distance metric is selected as cosine distance and $k$ is kept as 1 for all datasets. To reveal the intrinsic capability of the model, we do not tune the hyperparameters of KNN on each ASD dataset. For the DCASE20 dataset, anomaly detection is conducted per machine id, where a memory bank is constructed for embeddings of each machine id, and each query embedding is inferred by the memory bank with the same machine id. For the rest DCASE datasets, anomaly detection is conducted per section. Since domain shift is involved, two memory banks are constructed for each section, one for the source embeddings and the other for the target embeddings.

## Appendix B Details of Fault Diagnosis Tasks

Table [V](https://arxiv.org/html/2507.16696v1#S5.T5) presents the detailed task setup for fault diagnosis datasets, including detailed class mapping. For bearing fault diagnosis tasks, fault types are commonly similar, where NC, IF (some denote as IR), OF (some denote as OR), RF corresponds to normal condition, inner race fault, outer race fault and rolling element fault respectively.

## Appendix C Baseline Details

Five baselines are compared in this paper. We now describe the details of these baselines. For each baseline, we follow the suggested embedding procedure and select the top performing open-sourced checkpoints on the RMIS benchmark to form fair comparisons.

For AudioMAE [43], we utilize the official base checkpoint. During evaluation, we extract the patch embeddings from the last ViT layer of the encoder and take the mean along the sequence to form the signal representation.

For BEATs [10], we utilize the official iter3 checkpoint, which outperforms the iter1, iter2 and iter3+ checkpoints on the RMIS benchmark. The representation procedure is identical with AudioMAE.

For EAT [9], we utilize the official base30 and large checkpoints. During evaluation, we take the [CLS] embedding as the signal representation.

For CED [44], we utilize checkpoints of all four scales, which is similar with the scale hierarchy of FISHER. Therefore, CED is most favored for comparison to reveal the superiority of the pre-training scheme. The representation procedure is also identical with AudioMAE.

For DaSheng [45], we utilize the official base, 0.6B and 1.2B checkpoints to evaluate the performances of models with much larger size. During evaluation, we extract the frame-level embeddings from the last layer and mean-pool these embeddings to retrieve the signal representation.

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 2
presentation: 2
questions: .
soundness: 2
strengths: The paper introduces a foundation model for multi-modal signal representation. This is the first attempt to learn a multi-modal, multi-sampling-rate for various signal datasets.
summary: The paper presents a foundation model for multi-modal signal representations including sound, vibration, voltage, current, temperature, and etc.
weaknesses: My primary concern lies in the lack of novelty. I could not identify any clear novel contribution beyond the use of multi-modal training.

**[No Ablation Study]**
The paper does not include any ablation studies, making it difficult to understand why the proposed model outperforms others. It would be beneficial to provide ablation results for factors such as multi-modality, multi-scale datasets, model architecture, and sequence length.

**[Input Size]**
During training, FISHER uses a fixed 10-second segment, which limits the model’s usability. Could you clarify how the model handles longer input signals during evaluation? A 10-second limit seems too short for practical applications. 

I'm not an expert in this topic, specifically in industrial signal, so please lower the weight of my score accordingly.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: - What’s the model sizes of baselines? In Appendix, authors described some but not all model sizes of baselines. To better understand the efficient scaling curves of FISHER, it'll be great if authors could display the number of parameters for all baselines. 
 - Should we train FISHER with pretrained weights instead of training from scratch? In AST (https://arxiv.org/abs/2104.01778), with the pretrained ViT weights, the performance of AST is further boosted.
soundness: 3
strengths: - Good problem formulation: clearly formalizing the M5 problem is a valuable contribution.
summary: The paper proposed the M5 problem, and presents FISHER, a foundation model that represents signals as STFT sub-bands, treats increments in sampling rate as concatenation of additional sub-band information, and uses a teacher–student self-supervised pretraining framework. To evaluate, the authors introduce RMIS benchmark spanning 19 datasets across four modalities. FISHER outperforms 15 state-of-the-art encoders by at least 3.23% on average, showing efficient scaling curves.
weaknesses: - Limited novelty: many modules in this paper can be found in references. For example, ViT, features emerging, and EMA are off-the-shelf modules. It’ll be great if the authors could explain more about what’s the unique contribution and novelty in this paper.

---

confidence: 2
contribution: 2
initial_rating: 4
presentation: 2
questions: To help clarify some of the points above, I would appreciate your response to the following:
1. Could you please elaborate on the practical batching mechanism for inputs that have different sampling rates and thus produce a variable number of sub-bands?
2. Regarding the training process, what was the specific motivation for including the unmasked patches in the distillation loss, and what effect did you observe from this choice?
3. To make the paper more self-contained, would it be possible to provide a brief, high-level explanation of the "mask cloning strategy" and its specific role in your framework?
4. I would be very interested to hear your thoughts on the exceptional performance of the BEATs model in the anomaly detection tasks. Do you have a hypothesis for this result, and what might it suggest about the nature of those specific tasks?
soundness: 3
strengths: The authors should be commended for tackling a problem of significant practical importance. Creating a single model for the diverse and heterogeneous world of industrial signals is a valuable goal, and this work makes a convincing step in that direction. The key strength of the paper, in my view, is the conceptual novelty of the sub-band approach. It's an elegant and intuitive way to handle the variable-length nature of signals sampled at different rates. This strong methodological contribution is backed by a very thorough empirical evaluation. The scale of the comparison, across the new RMIS benchmark and against 15 different encoders, lends significant weight to the authors' claims. Furthermore, the development of the RMIS benchmark itself stands out as a valuable contribution that will surely benefit the wider research community.
summary: This paper presents FISHER, a new foundation model aimed at creating a unified representation for industrial signals, a domain the authors characterize with the "M5 problem." The central and most compelling idea is to reframe the challenge of variable sampling rates by modeling signals as a composition of frequency sub-bands. This allows for a consistent processing architecture, which is trained via a teacher-student self-distillation scheme. To properly assess the model's capabilities, the authors have also constructed a new and comprehensive benchmark, RMIS. The experimental evaluation is strong.
weaknesses: While the paper is promising, its clarity could be significantly improved in several areas, which currently hinders a full appreciation of the work. My primary concern is that the paper is not fully self-contained. For example, a key component of the training, the "mask cloning strategy," is mentioned without explanation, forcing readers to consult the external EAT paper to understand the methodology. Similarly, the crucial process of how signals yielding a variable number of sub-bands are actually batched together is left ambiguous. Beyond these clarity issues, some key design choices lack justification. It's unclear why the distillation loss is computed on unmasked patches, a deviation from common practice that warrants an explanation. The paper would also benefit from a deeper analysis of its own results. The strong performance of the BEATs model on anomaly detection is a fascinating finding, and the lack of discussion around why this might be the case is a missed opportunity for richer scientific insight.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1.	It seems that the M5 Industrial Signals are independently fed into the model, with each modality processed and trained separately, without cross-modal interaction during pretraining. Could the model potentially benefit from frequency-aware attention or cross-band interaction mechanisms, rather than pure concatenation?
2.	How are sub-band boundaries handled? Is there any information leakage or overlap between adjacent sub-bands?
3.	Does using a fixed STFT window and hop size across signals with different sampling rates risk under-resolving low-rate signals or over-smoothing high-rate ones?
4.	How was the $f_{base}$ (bandwidth unit) chosen, and how sensitive is the performance to this choice?
5.	Why was KNN inference used exclusively instead of fine-tuning or linear probing?
soundness: 3
strengths: 1.	Industrial signal modeling is an underexplored domain compared to speech/audio; addressing the M5 heterogeneity problem with a scalable foundation model is meaningful.
2.	The idea of modeling higher sampling rates as concatenated sub-band information is simple yet effective.
3.	The proposed RMIS benchmark covers diverse tasks and modalities, potentially serving as a valuable testbed for future research.
summary: This paper proposes FISHER, a foundation model for multi-modal industrial signal representation, aiming to address the heterogeneity of industrial data, which the authors summarize as the M5 problem. The authors also introduce a new benchmark, RMIS, to evaluate general representation ability via KNN inference without fine-tuning.
weaknesses: 1.	The core architecture and SSL setup (teacher-student distillation via EMA, ViT backbone) are nearly identical to existing work, raising concerns that the novelty lies mainly in the data preprocessing stage.
2.	The novelty of the approach is limited, since the method essentially reweights gradients in a way similar to prior balancing strategies, without a clearly distinguished contribution.
3.	Figures could better illustrate sub-band composition to enhance interpretability.