  DNN structures are continuously developing and achieving high performances in
classification problems. Also, it is observed that success rates obtained with
DNNs are higher than those obtained with traditional neural networks. In
addition, one of the advantages of DNNs is that there is no need to spend an
extra effort to determine the features; the CNN automatically extracts the
features from the dataset during the training. Besides their benefits, the DNNs
have the following three major drawbacks among the others: (i) Researchers have
struggled with over-fitting and under-fitting issues in the training of DNNs,
(ii) determination of even a coarse structure for the DNN may take days, and
(iii) most of the time, the proposed network structure is too large to be too
bulky to be used in real time applications. We have modified the training and
structure of DNN to increase the classification performance, to decrease the
number of nodes in the structure, and to be used with less number of hyper
parameters. A minimum distance network (MDN) following the last layer of the
convolutional neural network (CNN) is used as the classifier instead of a fully
connected neural network (FCNN). In order to strengthen the training of the
CNN, we suggest employing Walsh function. We tested the performances of the
proposed DNN (named as DivFE) on the classification of ECG, EEG, heart sound,
detection pneumonia in X-ray chest images, detection of BGA solder defects, and
patterns of benchmark datasets (MNIST, IRIS, CIFAR10 and CIFAR20). In different
areas, it has been observed that a higher classification performance was
obtained by using the DivFE with less number of nodes.
