# A Review of Neural Network-Based Emulation of Guitar Amplifiers
## 3 Neural Network-Based Methods
### Architectures
The computations carried out in a GRU are the following [34]:  
\[\text{Reset gate:}\ \mathbf{r}^{(t)}=\sigma(W_{rx}\mathbf{x}^{(t)}+W_{ rh}\mathbf{h}^{(t-1)}+\mathbf{b}_{r}) \tag{9}\] \[\text{Update gate:}\ \mathbf{z}^{(t)}=\sigma(W_{zx}\mathbf{x}^{(t)}+W_{ zh}\mathbf{h}^{(t-1)}+\mathbf{b}_{z})\] (10) Figure 1:0 (11) Figure 1:1  
The architecture used is comprised of a single recurrent layer followed by a fully connected one. Preliminary experiments showed that adding extra recurrent layers had little effect on the audio quality of the output. This method was used to model a pedal (Big Muff) and a combo amplifier (Blackstar HT-1) in [33]. In terms of objective quality, the most accurate RNN outperformed the WaveNet for the pedal and the most accurate WaveNet outperformed the RNN for the amplifier, with LSTM outperforming the GRU in terms of accuracy with roughly the same processing time. In a follow-up article [23], further comparisons between the recurrent networks from [33] and various WaveNet configurations were carried out for the modelling of two vacuum tube amplifiers: a Blackstar HT-5 Metal and a Mesa Boogie 5:50 Plus. This study includes the results from previous articles on the feedforward WaveNet and the LSTM network for distortion pedal emulation, and extends this with the inclusion of the vacuum tube amplifier models. Various configurations of each architecture were trained and compared in order to gauge both the real-time capabilities (using C++ implementations) and the audio quality. The largest configurations were WaveNet3 with gated activation from [27], 18 layers, and 16 channels, and LSTM96 with 96 recurrent units. The smallest configurations were WaveNet1 with gated activation, 10 layers, and 16 channels, and LSTM32 with 32 recurrent units. The LSTM provides better processing speeds than the WaveNet models; however, the largest WaveNet can better model the highly nonlinear HT5M amplifier. Analog distortion effects can be emulated by various configurations of neural network architectures [35]. In this work, eight architectures are tested and compared, one of which is the subject of a 2018 article [36]. These architectures include four LSTM networks, one hybrid convolutional LSTM, one plain Deep Neural Network (DNN) in the form of a MLP, one CNN, and one hybrid convolutional RNN. The most notable architectures of this work are:  
* The parametric LSTM in which the input dimensions are extended in order to taken into account the amplifier parameters. * The convolutional LSTM in which two stacked 2D convolution layers are used to redimension the input signals in order to parallelize the computations on GPU for accelerated processing. * The sequence-to-sequence LSTM which outputs a buffer instead of a single sample, again to accelerate processing. All architectures presented vary in terms of accuracy and real-time performance but the hybrid convolutional LSTM presented the best Computation Time (CT) and accuracy trade-off. Another category of hybrid method includes autoencoder architectures such as those presented in the works of [PERSON] et al. [37, 38, 39]. The general structure of an autoencoder comprises three stages, namely, an encoding front-end, a latent space containing the new representation of the input data, and a decoding back-end. This structure enables the model to learn an approximate copy of the input, forcing it to prioritize useful properties of the data [22].