# Deep Neural Networks Classification via Binary Error-Detecting Output Codes
## 1 Introduction
Second, contrary to ECOC, we use error detection with a new ability to reject untrustworthy decisions. We propose an approach that replaces the combination of softmax and categorical cross-entropy to train models with binary coding combined with fuzzy Zadeh logic [13; 14]. We use linear block codes from coding theory [15; 16] as the output coding. Here, codewords are divided into two parts: information bits and check bits. The information part can reflect a data structure while the check part distributes the codewords across the codespace to obtain the error detection performance. In addition to comparing the accuracy of the proposed method against traditional one-hot output encoding with a softmax activation function, we study the trained models' reliability. Trust and safety is an important topic in machine learning [17; 18]. In the case of a neural network, a standard approach to deciding whether or not to trust a classifier's decision is to use the classifiers' reported confidence scores, e.g., probabilities from the softmax layer [19]. We define reliability as a percentage of correctly classified samples among all samples for which the model is confident to make a prediction. It means that we have three types of results: correct, incorrect, and abstain (reject). The typical approach to improve reliability is to use a threshold value on the classifier's confidence score. It allows the classifier to abstain from predicting if the resulting confidence is lower than the predefined threshold. This way, a classification network can indicate when it is likely to be incorrect. Modern deep neural networks are poorly calibrated, i.e., the probability associated with the predicted class label does not reflect its ground truth correctness likelihood [20; 21]. It can be improved using calibration methods, such as temperature scaling [20; 21], but even if the neural network is calibrated, the ranking of scores may not be reliable [22]. The error detection ability of binary output codes provides an additional tool to improve reliability--a threshold on the Hamming distance between the predicted output code and codewords assigned to classes. For one-hot codes, the Hamming distance between all codes is equal to two, which does not allow correcting errors or improving reliability. We tested the reliability on in-distribution and out-of-distribution data (data far away from the training samples). We show that our proposed method for both in-distribution and out-of-distribution data achieves approximately the same results in a case of maximum accuracy as the traditional one-hot encoding with softmax when used on a 10-class classification problem. It opens up new possibilities and research directions, especially regarding how to find optimal binary codes (in terms of their size and inner structure) and architectures that use their full potential. We take advantage of the error detection ability to indicate a codeword outside an agreed code subspace without evaluating a confidence metric. In the case of out-of-distribution data, we do not use any advanced techniques to tackle this problem [23; 24] as we are interested in the natural properties of the proposed method. We leave this to future research.