# Deep Neural Networks Classification via Binary Error-Detecting Output Codes
## 4 Conclusions  
We examined the possibilities to use binary coding as the output coding in neural networks. We do not directly embed training dataset patterns into the coding space. It allows us to use the existing knowledge in the coding theory to design class-codewords instead. We use linear block codes for the possibility of the separation of class information from the checksum part of the codeword. We show that this property allows speeding up the decoding part of the classification. During the decoding phase, the similarity of the predicted output and the class-codewords usually needs to be computed. In the proposed approach, it is enough to take information (class) bits of the truest codeword (a rounded network normalized output) if the codeword gives the zero syndrome. We have proven that this simple procedure returns the truest decision from the Zadeh fuzzy logic point of view. Moreover, the truth-value of the decision can be notified to the user. We studied the error detection capabilities of binary coding. Error detection means that if the network's output does not have the properties of the codewords assigned to the classes, such outcomes are untrustworthy and thus rejected during the classification. It is a different approach to assess the trustworthiness of the prediction compared to the softmax case, where the threshold on the winning output's confidence value is used. Of course, we can combine both approaches, and the paper shows an advantage of such symbiosis. It enables us to reach a higher classification reliability if we are willing to accept more unclassified patterns. We proposed a novel classification decision method based on the Zadeh fuzzy logic. We can interpret the neural network's output value to measure the statement's truth that the pattern belongs to a given class. During the network training, we aim to maximize the truthiness of the correct answer. In the well-established one-hot encoding with the softmax decision, minimizing the error at the output corresponding to the valid class leads to minimizing deviations in the rest of the outputs. The decision based on Zadeh fuzzy logic has a similar ability. Minimizing the maximum error in the required codeword also tends to suppress the errors in the rest of the outputs. We chose CRC codes as representative of linear block codes. We tested different approaches to the classification decision using the CRC output coding and showed that the proposed Zadeh decision method achieves the best accuracy. Moreover, the Zadeh decision with CRC achieves comparable results to one-hot encoding with softmax while maintaining the error detection capability. Further experiments on reliability and out-of-distribution data confirm that the proposed method achieves similar performance as one-hot encoding with softmax for high accuracy levels. This paper performed a direct comparison between CRC coding with Zadeh decision and one-hot encoding with softmax. For this reason, we only used 10-bit codewords for 10-class classification. Contrary to the one-hot encoding, linear block codes (and binary codes in general) do not limit the size of the codewords, giving us more freedom in output coding strategies. They open up further possibilities to find better binary outputs codes. One way to use this freedom is to construct more compact output codes. Binary codes can encode \(n\) classes to a maximum of \(\log_{2}(n)+1\) output bits. This feature can be practical, especially for tasks with a large number of classes. Another way is to use a large codespace to improve performance or enhance error detection and/or correction abilities. Authors in [25] show that the longer target Hadamard code achieved higher accuracy than the shorter one. They also argue that longer codes consistently perform better than the shorter codes given the same training epoch. In this paper, we focused on a 10-label classification task represented by CIFAR-10 dataset. Although we consider the results to be very promising, using only a 10-class dataset can be considered a limitation of this study.