# Deep Neural Networks Classification via Binary Error-Detecting Output Codes
## 3 Results and Discussion
### Performance on CIFAR-10  
This section describes our main experiments. We evaluated various configurations of network architecture, output coding, and classification decision in terms of classification accuracy and reliability. Experiments were performed on the CIFAR-10 dataset. Accuracy is the most used measure of classification performance. In the overall methodology, the classification procedure assigns each neural network output to the one class (i.e., to the one codeword) during the training and the testing. Introducing the error-detecting codes, we can extend the decision space to the whole codespace. In this paper, we keep assigning network output to one codeword during the training. During the testing, the network output can be assigned to any code from the whole codespace. We use 10 bits output codes for all combinations reported in Table 4, except Hadamard coding (described in Section 2.1), which requires a minimum of 15 bits to cover 10 classes and to ensure the properties described in [26]. It means that the whole codespace is 1024 for CRC7 and 10 for one-hot encoding. We use accuracy to measure performance when only class-codewords (output codes belonging to classes) are used. When we use the whole codespace during the decision, we can end up with output codes that do not belong to any class. In this case, the error detection capability is used, and the sample is rejected as untrustworthy. We use the term \"full codespace accuracy\" (FCA) to measure accuracy when full codespace is used. Although the paper's primary goal is to examine the properties of binary coding, the decision-making method greatly influences the correctness of the classification and the ability to use full codespace. As shown in Table 4, we compared the performance of bit-by-bit decision (represented by the \"bit threshold\" approach) and holistic word decisions represented by Euclid distance (applied on Hadamard code), softmax (used on one-hot code), as well as a proposed Zadeh decision (applied for both one-hot and CRC code). We first tested all approaches with CNN1 architecture. For each combination, we ran 10 repetitions of training and reported the result with the best accuracy. Then, we selected four best performing approaches, according to their accuracy, and tested them on CNN2 and ResNet20 architecture. Here, we ran five repetitions of training for each architecture and reported the best result according to best accuracy. Our primary interest is to examine the possibility of using binary coding (represented by CRC7) with error detection capability. We first tested the bit thresholding approach (Equation (5)) as it is a straightforward decision-making approach. For this approach, the mean squared error was used during the training and bit thresholding during the testing. Then we tested Zadeh decision on both CRC7 and one-hot encoding. Using the Zadeh decision, we approached one-hot encoding as a particular binary coding case where class-codewords have one-hot properties. We used one-hot encoding with softmax as a baseline method for comparison. Inspired by [11], we also tested CRC7 coding with softmax. It allowed to combine the advantages of binary codes and the softmax decision but did not use error detection capabilities. Last, we tried the approach from [26], where the Euclid decision is used on Hadamard coding. Figure 2: Accuracy–reliability characteristic for CRC codes (left), CRC7 class-codewords (right). The results indicate a predominance of the holistic style of decision (Zadeh, softmax, Euclid) over decisions on individual bits (bit threshold), and we expect increasing advantages with an increasing number of bits. We can see that CRC7 with Zadeh decision achieves similar accuracy to one-hot encoding with softmax. In one-hot encoding with softmax, the code structure and the decision rule support each other. We know that only one is equal to \"1\" within bits, and the output values are normed and evaluated to keep this knowledge. Even during the network training, it supports the winning bit in its increasing.