# Deep Error-Correcting Output Codes
## 3 Deep Error-Correcting Output Codes (DeepECOCs)
### DeepECOCs
Here, \(Z_{i}\) denotes the hidden layer learned from some coding strategies. We designed this architecture because we wanted to make the autoencoder and denoising autoencoder have the same structures as DeepECOCs. In order to make DeepECOCs adapt to this structure, we used the dense and sparse coding design methods that controlled the codeword length. The denoising rate and dropout rate were set to 0.1, the batch size was set to 100, the learning rate was set to 0.01, and the number of epochs was set to 80,000. Figure 5 shows the comparison results. We can observe that DeepECOCs achieve comparable results with the autoencoder and denoising autoencoder, but perform better than the single-layer ECOC framework. Figure 6 shows the experimental results with a new deep architecture. We can observe that DeepECOCs outperform other compared methods. In addition, DeepECOCs with the ECOCONE (initialized by DECOC) coding strategy achieved the best result. Note that, in this scenario, all the deep networks shared the same architecture. Figure 4: Classification accuracy with different numbers of hidden layers on the USPS data set. Figure 5: Classification accuracy on the MNIST data set. The architecture is 784-500-500-2000-10. ### Classification on the CMU Mocap Data Set

The CMU mocap data set includes three categories, namely, jumping, running and walking. We chose 49 video sequences of four subjects. For each sequence, the features were generated using [PERSON]'s method [53], with a dimensionality of 93. Figure 7 shows one video sequence from the CMU mocap data set. We used 3 hidden layers for all deep learning methods. The learning rate was set to 0.1 and the epoch was set to 400. The other parameters were the same as in the previous experiments. The average classification accuracy based on the 10-fold cross validation are reported in Table 5.

From Table 5, we can easily observe that DeepECOCs achieved the best results in the CMU mocap data set, and DeepECOCs with a one-versus-one coding strategy far surpass other methods. ### Classification on the CIFAR-10 Data set

The CIFAR-10 data set consists of 60,000 \(32\times 32\) colour images in 10 classes, with 6000 images per class. There are 50,000 training images and 10,000 test images. Figure 8 shows some samples from the 10 categories. For the purpose of reducing computational cost, we attempted to create a compact representation of the data using an efficient local binary patterns algorithm. As a result, the representation of dimensionality 36 and 256 were adopted and the data were normalized to [0, 1], as well. We also used 3 hidden layers for all deep learning methods. The learning rate was set to 0.1, and the epoch was set to 4000. Other parameters were the same as the previous experiments. The classification accuracy are reported in Table 6.

Figure 6: Classification accuracy on the MNIST data set. The architecture is 784-\(Z_{1}\)-\(Z_{2}\)-\(Z_{3}\)-10. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Problem** & **AE** & **DAE** & **V1** & **V2** & **V4** & **V5** \\ \hline CMU & 0.6171 & 0.6422 & **0.8030** & 0.6364 & 0.7652 & 0.6030 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The classification accuracy on the CMU mocap data set. The best results for each scenario are highlighted in bold face. Figure 7: One video sequence from the CMU mocap data set. From Table 6, we can observe that DeepECOCs achieved the best results. Since we compressed the data using the local binary pattern algorithm, the results are not comparable with the state-of-the-art results. However, for the compared methods, they are fair. In this case, DeepECOCs with the ECOCONE (initialized by one-versus-one) coding strategy achieved better results than the autoencoder and denoising autoencoder. This means that DeepECOCs are general models to handle real world applications and can often achieve desirable results.