## 2 Background  
### Modeling of Nonlinear Audio Effects  
Since a nonlinear system cannot be characterized by its impulse response, frequency response or transfer function [1], digital emulation of distortion effects have been extensively researched [39]. Different methods have been proposed such as _memoryless static waveshaping_[11], where systemblack-box modeling approaches where an entire problem can be taken as a single indivisible task which must be learned from input to output. The desired output is obtained by learning and processing directly from the incoming raw audio, thus reducing the amount of required prior knowledge and minimizing the engineering effort [45]. End-to-end deep neural networks (DNNs) for audio effects modeling have been recently explored for linear and nonlinear, time-varying and time-invariant audio effects with long temporal dependencies. Equalization matching is achieved in [6] and nonlinear modeling in [7], where the network is capable of modeling an arbitrary combination of linear and nonlinear audio effects with short-term memory. Nevertheless, the network of [7] does not generalize to transformations with long temporal dependencies such as modulation based audio effects. The model is divided into three parts: adaptive front-end, latent-space and synthesis back-end, and follows an adaptive convolutional architecture together with dense layers and trainable activation functions as nonlinear waveshapers. Several linear and nonlinear time-varying and time-invariant audio effects were modeled in [8], following the adaptive convolutional architecture from [7]. The structure of the synthesis back-end is modified and RNNs are incorporated into the latent-space in order to explore their capabilities when  
The **latent-space DNN** contains two layers. Following the filter bank architecture, the first layer is based on locally connected dense layers and the second layer consists of a fully connected (FC) layer. The DNN modifies the latent representation \(\mathbf{Z}\) into a new latent representation \(\hat{\mathbf{Z}}\) which is fed into the synthesis back-end. The first layer applies a different dense layer to each row of the matrix \(\mathbf{Z}\) and the second layer is applied to each row of the output matrix from the first layer. In both layers, all dense layers have 64 hidden units, are followed by the _softplus_ function and are applied to the complete latent representation rather than to the channel dimension. The **synthesis back-end** accomplishes the nonlinear task by the following steps. First, \(\hat{\mathbf{X}}_{2}\), the discrete approximation of \(\mathbf{X}_{2}\), is obtained via unpooling the modified envelopes \(\hat{\mathbf{Z}}\). Then the feature map \(\hat{\mathbf{X}}_{1}\) is the result of the element-wise multiplication of the residual connection \(\mathbf{R}\) and \(\hat{\mathbf{X}}_{2}\). This can be seen as an input filtering operation, since a different envelope gain is applied to each of the frequency band decompositions obtained in the front-end. The second step is to apply various waveshapping nonlinearities to \(\hat{\mathbf{X}}_{1}\). This is achieved with a a DNN with smooth adaptive activation functions (DNN-SAAF). The DNN-SAAF consists of 4 FC dense layers. All dense layers are followed by the _softplus_ function with the exception of the last layer. Locally connected Smooth Adaptive Activation Functions (SAAFs) [50] are used as the nonlinearity for the last layer. SAAFs consist of piecewise second order polynomials which can approximate any continuous function and are regularized under a Lipschitz constant to ensure smoothness. Overall, each function is locally connected and composed of 25 intervals between \(-1\) to 1. We tested different standard and adaptive activation functions, such as the parametric and non parametric rectifier linear unit (_ReLU_), hyperbolic tangent, sigmoid and fifth order polynomials. Nevertheless, we found stability problems and non optimal results when modeling nonlinear effects.