# A Review of Neural Network-Based Emulation of Guitar Amplifiers
## 3 Neural Network-Based Methods
### Loss Functions  
Training neural networks is an optimization problem in which we often aim to minimize a given loss function. This loss represents the distance between the prediction and the target and must therefore accurately depict the perceptual difference between signals. This is often not the case with objective losses such as the Mean-Squared Error (MSE) defined as:  
\[\mathcal{L}_{\text{MSE}}=\frac{1}{N}\sum_{i=0}^{N-1}(y[i]-\hat{y}[i])^{2}. \tag{13}\]  
The models used for amplifier modelling process raw audio in the time domain and the losses used are computed directly with the signal's waveform whose shape does not perfectly correspond to human perception. For example, different frequencies are not perceived in the same way to the human ear, with certain frequencies appearing louder than others [52]. This difference in loudness is represented by equal-loudness contours. To improve the accuracy of a given network, spectral information can also be included into the loss computations but this approach presents its own problems and most of the loss functions used remain time domain-based. The most widely known loss for training neural networks is the MSE described previously. This loss is one of the most used for training networks in distortion circuit simulation. It was used in one of the first articles presenting recurrent networks for amplifier simulation [31] as well as in all of the gray-box methods presented previously, including a normalized variant used in [41] in order to stabilize the initial training of the network. \begin{table}
\begin{tabular}{c c} \hline \hline
**Data Set** & **Details** \\ \hline Data set from [50] & Five different playing styles of guitar sounds passing through different amplifiers Used in [23; 35; 36] at 44.1 kHz \\ \hline IDMT-SMT-Audio Effects [40] & 44.1 kHz used in [28] and downsampled to 16 kHz in [37; 38; 39] \\ \hline Freesound [47] & Input signal sent through SPICE simulations at 44.1 kHz. Used in [25] \\ \hline IDMT-SMT-Guitar/Bass [48; 49] & Used in [23; 26; 33]. 44.1 kHz input signals sent through the analog target devices \\ \hline SignalTrain [51] & Used in [28] at 44.1 kHz \\ \hline Custom & \begin{tabular}{c} Guitar signals sent through SPICE model of the analog circuit at 44.1 kHz used in [43] \\ \end{tabular} \\ \hline Custom & \
\begin{tabular}{c} 90-s logarithmic sine sweep ranging from \\ 20 Hz to 20 kHz with constant amplitude \\ followed by 90 s of white noise at 48 kHz \\ used in [4] \\ \end{tabular} \\ \hline Custom &
\begin{tabular}{c} 10-s logarithmic sine-sweeps combined with \\ low-level white noise. This combined signal \\ was increased in amplitude linearly from 0 to \\ 1 over 1/2 the length of the signal. \\ Min freq = 20 Hz and max = 10 kHz at \\ 192 kHz [41] \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 4: _Cont._Similar losses include the Root Mean Squared Error (RMSE) used in [30]:  
\[\mathcal{L}_{\text{RMSE}}=\sqrt{\frac{\sum_{i=0}^{N-1}(y[i]-\hat{y}[i])^{2}}{N}} \tag{14}\]  
and the Normalized Root Mean Squared Error (NRMSE) used in [35]:  
\[\mathcal{L}_{\text{NRMSE}}=\sqrt{\frac{\sum_{i=0}^{N-1}(y[i]-\hat{y}[i])^{2}}{ \sum_{i=0}^{N-1}y[i]^{2}}}. \tag{15}\]  
Despite having relatively widespread use, the MSE-based losses lack perceptual accuracy. Another loss function frequently encountered in the state-of-the-art of amplifier simulation methods using neural networks is the Error-to-Signal Ratio (ESR) defined as:  
\[\mathcal{L}_{\text{ESR}}=\frac{\sum_{i=0}^{N-1}(y[i]-\hat{y}[i])^{2}}{\sum_{i =0}^{N-1}y[i]^{2}}. \tag{16}\]  
Variants of this loss have been used in the works of [PERSON] et al. [25; 26] and [PERSON] et al. [23; 33] with pre-emphasis filtering for better perceptual accuracy. In both [PERSON] et al. papers, the high-pass pre-emphasis filter with the transfer function \(H(z)=1-0.95z^{-1}\) was used to train their WaveNet architecture as without this filtering it was found that the model struggled at higher frequencies.