# Deep Error-Correcting Output Codes
## 3 Deep Error-Correcting Output Codes (DeepECOCs)
### DeepECOCs
In addition, from all of the above experiments, we can basically conclude that DeepECOCs with the ECOCONE (initialized by one-versus-one) coding strategy often perform very well. ### Combining CNNs with DeepECOCs

In this experiment, we combined CNNs and DeepECOCs to improve the performance of the original CNNs. The CIFAR-10 data set was used. We first trained a ResNet for 200 epochs, as the baseline model (denoted as ResNet-baseline). Then, we trained it from scratch for 50 epochs, and subsequently, used the flattened output of the last convolutional layer to train DeepECOCs. Here, the architecture of DeepECOCs was \(m\)-505-225-220-10, where \(m\) was the dimension of the flattened features, and the coding strategy of the ECOC modules was the dense coding. Finally, the whole CNNs+DeepECOCs model (denoted as ResNet-DeepECOCs) was fine-tuned for 80 epochs. Table 7 shows the obtained results. As we can observe, DeepECOCs improve the performance of the original CNNs. ## 5 Conclusions

In this paper, we propose a novel deep learning model, called deep error-correcting output codes (DeepECOCs). DeepECOCs extend the traditional ECOC framework in a deep architecture fashion. Extensive experiments on 16 data sets from the UCI machine learning repository, the USPS and MNIST handwritten digits, the CMU motion capture \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Problem** & **AE** & **DAE** & **V1** & **V2** \\ \hline CIFAR(36) & 0.3501 & 0.3678 & 0.4530 & 0.3895 \\ CIFAR(256) & 0.4352 & 0.4587 & 0.4936 & 0.4521 \\ \hline Problem & V3 & V4 & V5 & V6 \\ \hline CIFAR(36) & 0.4031 & **0.5089** & 0.4517 & 0.4752 \\ CIFAR(256) & 0.5236 & **0.5588** & 0.4589 & 0.5224 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The classification accuracy on the CIFAR-10 data set. The best results for each scenario are highlighted in bold face. Figure 8: CIFAR-10 color images from the 10 categories. Each column corresponds to one category. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Methods** & **ResNet-Baseline** & **ResNet-DeepECOCs** \\ \hline Accuracy & 0.9098 & **0.9208** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The classification accuracy obtained on the CIFAR-10 data set. The best result is highlighted in boldface. data set and the CIFAR-10 data set, demonstrate the superiority of DeepECOCs over the related ECOC, feature learning and deep learning methods. In future work, we would like to develop new coding strategies that may improve the learnability of DeepECOCs on broader applications than we have tested in this paper. Moreover, for the training of the base classifiers, parallel computing is one way to improve the learning efficiency. Hence, we will implement parallel computing to speed up the optimization of DeepECOCs. Additionally, we plan to combine DeepECOCs with Transformer models [31], using the attention mechanism of Transformer models to extract useful features for DeepECOCs, in contrast to the combination of DeepECOCs and CNNs introduced in Section 3.4. In this case, we may achieve the state-of-the-art learning results on different tasks, such as object recognition and detection. Conceptualization, L.-N.W. and G.Z. ; methodology, L.-N.W., Y.Z. and G.Z. ; software, H.W. ; validation, L.-N.W. and J.D. ; formal analysis, G.Z. ; investigation, L.-N.W. ; resources, J.D. ; data curation, H.W. ; writing--original draft preparation, L.-N.W., Y.Z. and G.Z. ; writing--review and editing, H.W., Y.Z., J.D. and G.Z. ; visualization, Y.Z. ; supervision, G.Z. ; project administration, J.D. ; funding acquisition, G.Z. All authors have read and agreed to the published version of the manuscript. This work was partially supported by the National Key Research and Development Program of China under Grant No. 2018 AAA0100400, HY Project under Grant No. LZY2022033004, the Natural Science Foundation of Shandong Province under Grants No. ZR2020 MF131 and No. ZR2021 ZD19, the Science and Technology Program of Qingdao under Grant No.