# Intelligent Neural Network Schemes for Multi-Class Classification
## 1 Introduction  
Classification is one of the most important techniques for solving problems [1, 2]. Infinitely many problems can be viewed as classification problems. In daily life, telling a female person from a male one is a classification problem, which is probably one of the earliest problems people face to solve. Giving grades to a class of students could be an uneasy classification task for a teacher to work with. Deciding the disease a patient may have based on the symptoms is a difficult classification task for a doctor. Classification is also useful in engineering applications [3, 4, 5, 6], e.g., mechanical systems, mechanics and design innovations, applied materials in nanotechnologies, etc. For example, classification of structures, systems, and components is very important to safety for fusion applications [7]. The product quality has been found to be influenced by the engineering design, type of materials selected, and the processing technology employed. Therefore, classification of engineering materials and processing techniques is an important aspect of engineering design and analysis [8]. In general, classification consists of two phases, training and testing, as shown in Figure 1. In the training phase, a collection of labeled objects is given. After attribute extraction, a set of training instances is obtained, and they are used to train a model. In the testing phase, the trained model is used to evaluate any input objects to determine which categories the input objects belong to. Normally, there are \(m\) categories, \(m\geq 2\), such as Medicine, Finance, History, Management, and Education, involved in the multi-class classification problem [9]. The data to be dealt with can be either single-label or multi-label. If the given object belongs to one category, it is called single-label classification. On the other hand, it is called multi-label classification if the given object can belong to two or more categories [10; 11]. In this paper, multi-class classification is concerned with a given set of \(N\) training instances, \(\big{\{}\mathbf{x}_{1},\mathbf{y}_{1}\big{\}},\big{\{}\mathbf{x}_{2},\mathbf{ y}_{2}\big{\}},\ldots,\big{\{}\mathbf{x}_{N},\mathbf{y}_{N}\big{\}}\), where  
* \(\mathbf{x}_{i}=(x_{1,i},x_{2,i},\ldots,x_{n,i})\), \(1\leq i\leq N\), is the input vector of instance \(i\). There are \(n\) attributes, with real attribute values \(x_{j,i}\), \(1\leq j\leq n\) in \(\mathbf{x}_{i}\). * \(\mathbf{y}_{i}=(y_{1,i},y_{2,i},\ \ldots,\ y_{m,i}\ )\), \(1\leq i\leq N\), is the input vector of instance \(i\). There are \(m,m\geq 2\), categories, category \(1\), category \(2\), \(\ldots\), category \(m\). For instance, \(i\), \(y_{j,i}\ =+1\) if the instance belongs to category \(j\) and \(y_{j,i}\ =-1\) if the instance does not belong to category \(j\), \(1\leq j\leq m\). Note that \((a_{1},\ a_{2},\ \ldots,a_{k})\) indicates the \(k\)-vector \([a_{1}\ a_{2}\ \ldots\ a_{k}]^{T}\). The aim of this paper is, given the set of \(N\) training instances, to decide which categories a given input vector, \(\mathbf{p}\ =\ (p_{1},\ \ldots,\ p_{n})\), belongs to. For single-label classification, one and only one \(+1\) appears in every target vector, while for multi-label classification, two or more entries in any target vector can be allowed to be \(+1\). Traditionally, statistical methods were used for multi-class classification [12; 13]. Recently, machine learning techniques have been proposed for solving the multi-class classification problem. The decision tree (DT) algorithm [14; 15] uses a tree structure with if-then rules, running the input values through a series of decisions until it reaches a termination condition. It is highly intuitive, but it can easily overfit the data. Random forest (RandForest) [16] creates an ensemble of decision trees and can reduce the problem of overfitting. The naive Bayesian classifier [2] is a probability-based classifier. It calculates the likelihood that each data point exists in each of the target categories.