## 1 Introduction  
After the great success of machine learning (ML) methods in the classical application domains such as computer vision and speech recognition, they recently proved successful in other areas as well. The tasks suited to ML techniques can be divided into three main categories. First is the classification of objects based on rules inferred from a training set of examples (supervised learning). Second is the classification of objects without access to any examples, and the rules are found based only on a given distance metric. This technique (unsupervised learning) typically results in a cluster system minimizing and maximizing the intra- and inter-cluster distances, respectively. The third category (reinforcement learning) deals with learning how to interact with an unknown environment in order to reach a given goal. This method jointly addresses the problem of estimating the behavior of the environment and simultaneously learns a policy, which dictates actions to be taken in order to meet the goal. A good introduction to these ML topics can be found in [1, 2]. In this paper, we focus on the first category, i.e., on supervised learning, implemented with an artificial neural network (ANN). The ANN mimics the structure of a brain in a sense, in that it consists of a number of individual neurons, each having multiple real inputs and one real output. Each neuron is parameterized by a set of weights and a bias. The operation performed by each neuron is a simple weighted sum of its inputs, the addition of the bias, and the application of a nonlinear activation function. Typically, the structure of an ANN consists of a number of layers of neurons. The output of a neuron is connected to the inputs of all neurons in the subsequent layer. The total number of layers and number of neurons in each layer are the design parameters of the network. A special role is given to the input and output layer, which are always present (ANNs with hidden layers are termed deep neural networks). In the input layer, there is one neuron for each dimension of the input data. In the output layer, we have typically a neuron for each individual classification class, where we can apply a hard metric (for example, a max function) for a hard decision or a soft metric (soft-max function) to get a soft output, which can be interpreted as a probabilistic classification output. Learning is performed by applying examples from the training set and optimizing the parameters of individual neurons w.r.t. a fidelity metric of the ANN output and the desired output given by the training set. This optimization can be efficiently implemented by the backpropagation algorithm, which performs a stochastic gradient descent on the fidelity metric. This is the basic principle behind ML and ANN in particular. One of the issues with such a simple approach is the large number of parameters that arise with high-dimensional input data. With a large number of parameters, the learning process takes longer and requires a larger training set. One way of eliminating this problem is to preprocess the data and reduce its dimensionality while maintaining the information needed for classification. This process is called feature extraction and is very dependent on the particular data source and application. Much work has been done to develop sophisticated methods for such feature extraction, for example convolutional layers for image recognition [3], cepstral coefficients for speech recognition [4] or modulation recognition [5] and the bag-of words technique, developed for feature extraction of textual data [6], to name a few. Another issue, which is also related to the number of parameters of the network, is the problem of over-fitting. The core of all ML methods is the ability to extract generalizing classification rules out of a set of particular examples: the training set. Over-fitting occurs when insignificant details of the training data are learned and degrade the performance of classifying data not present in the training set.