# Singing-Voice Timbre Evaluations Based on Transfer Learning
## 4 Transfer Learning Model for Timbre Evaluation
### Transfer Learning Model  
#### 4.2.1 Musical Instruments Timbre-Evaluation Network  
This research develops a model based on the CRNN model architecture proposed by [PERSON] et al. (2016). The model in this research consists of three parts, retaining the first two parts of the original model--convolutional layer and recurrent layer. The third part is changed from a coding layer to regression layer. After the convolutional layer is used to filter the mel spectrogram in order to extract complex features, the recurrent layer is used to analyze the features; finally, the regression layer is used to calculate the scores. The network structure is shown in Figure 5.  
Figure 4: The architecture of the contrast model. Figure 5: The architecture of the musical-instrument timbre-evaluation model. The parameters of each network layer are redesigned, inspired by the filter shape design raised by [PERSON] et al. [8]. The convolutional kernel size is \(3\times 1\), and the number of filters is 64. Pooling layers are added after the convolutional layer to extract features, and the window size is designed as a rectangle with small widths, lengths, and heights in order to retain more information about the vertical direction. Unlike general networks, this research designs only one convolutional layer, and its main purpose is to adapt to a small dataset. Moreover, the input image is a transient frequency feature, which is easier to learn than an entire spectrogram image. The recurrent layer is a single GRU layer, which can effectively handle the sequential input. A dropout layer is added after the GRU layer to avoid model overfitting. Finally, the regression layer is used to predict timbre scores. The activation function is the commonly used softmax, which also requires mapping the score labels to the range of 0 to 1 when processing data. This research tested four different activation functions, and the results proved that softmax was the most suitable for the task. The loss results for the test set are shown in Table 2. Among them, relu and selu converged poorly. Relu completely overfit, and its output is all one value. Sigmoid is prone to overfitting, and its loss-of-training set outperforms softmax but the validation and test sets are not as good as softmax. Figure 6 shows an example of the one-feature model. \begin{table}
\begin{tabular}{c c} \hline \hline
**Activation Function** & **Average Absolute Error Loss of Test Set** \\ \hline Relu + BN & \\ \hline selu & 3.84 \\ \hline Sigmoid & 1.70 \\ \hline softmax & 1.25 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average absolute error loss of the test set for different activation functions. Figure 6: Loss of the slim model: (**a**) model loss when the activation function is relu and batch normalization, (**b**) model loss when the activation function is selu, (**c**) model loss when the activation function is sigmoid, and (**d**) model loss when the activation function is softmax. Considering the slightly discrete distribution of the scores in the dataset and the situation that one sample corresponds to multiple score labels, we defined the loss function as Equation (1). The coefficient of the mean absolute error is multiplied by the standard deviation so that the penalty is large for samples with concentrated ratings and small for samples with scattered ratings. In the loss function, y_pred is the predicted score, y_true is the average of the actual scores, alpha is the standard deviation, and k is a constant. After testing, this research selected k to be 100. \[\text{loss}=\frac{\text{k}}{\alpha^{2}}(\text{y}_{\text{pred}}-\text{y}_{ \text{true}})^{2} \tag{1}\]  
The dataset was divided into a ratio of 80% training set, 10% test set, and 10% validation set. Since different features have different concerns, we construct 16 models to learn the scores for 16 feature distributions. The accuracy of evaluation is improved by adjusting the learning rate of each model to adapt to different features.