# Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks
## 4 Experiments and Results
### Experimental Setups
#### 4.1.2 Hyperparameters  
To obtain the best experimental results, the hyperparameters were calibrated in detail using the validation set. The hyperparameters were heuristically determined by the system developers, whereas the feature parameters were calculated within the model and they can be determined from the data. In deep learning, the representative hyperparameters include the learning rate, epoch, batch size, and dropout rate.  
To find the lowest loss \(L\), the weights \(W\) were updated with a learning rate \(\eta\). The weight \(w_{t+1}\) in the next time step was updated by subtracting the multiplication of gradient \(\frac{\theta L}{\theta w}\) and \(\eta\) from the current weight \(w_{t}\), as described in (3). The weights were optimized by the Adam optimizer [21].  
\[w_{t+1}=w_{t}-\eta\times\frac{\theta L}{\theta w} \tag{3}\]  
Setting an appropriate learning rate is very important, as the learning rate affects variations in the loss value and helps to find the global minimum loss value. After conducting several experiments by calibrating the learning rate, this study determined the most appropriate learning rate to be 0.0001.  
Epoch means a count of how often the total training samples have been passed forward and backward through the model. As each epoch proceeds, the loss value is expected to gradually decrease. However, there is a period in which the model does not show any significant improvement. Thus, an appropriate number of epochs needs to be determined when investigating the improvement in performance. This study confirmed that 100 epochs were sufficient to achieve the lowest loss.  
Batch size refers to the number of training samples that are sequentially entered into the training stage, and it is generally set to a power of two such as 2, 4, 8, 16, 32, etc. In general, a larger batch size makes the training faster but it requires more memory for calculation. Thus, the batch size should be determined considering the memory capacity of the computer. In this study, the batch size was empirically set to 32. Thus, 32 training samples were in a single batch.  
Dropout is a type of regularization technique [22]. It helps to prevent the overfitting problem, which arises when the network excessively fits into the training data and thus fails to correctly recognize test data. Overfitting problem often occurs when the network has a large capacity and the amount of training data is too small to meet the capacity. Hence, this issue needs to be handled in the accepted speech recognition task, where a smaller amount of accented speech data have been provided. Dropout deactivates some neurons (nodes) to reduce the capacity of the network, making the network simpler. This operation is performed according to a dropout rate, which is the ratio of the deactivated nodes to all nodes. In this study, the dropout rate was empirically set to 0.1.  
The last hyperparameter in this study was a domain adaptation parameter \(\lambda\) used in DANN. When the gradient of the domain classifier \(\frac{\theta L_{d}}{\theta w_{d}}\) is backpropagated into the downstream of the ASR model, the gradient is multiplied by \(\lambda\), which can be set between 0 and 1. The parameter controls the influence of domain adversarial training. A higher value makes the effect of the domain adversarial training more dominant. However, an excessive dependence on domain adversarial training may lead to performance degradation. Thus, the parameter should be set precisely via experiments. In this study, \(\lambda\) was empirically set to 0.01.