Saturated perceptrons for maximum margin and minimum misclassification error
This Letter discusses the application of gradient-based methods to train a single layer perceptron subject to the constraint that the saturation degree of the sigmoid activation function (measured as its maximum slope in the sample space) is fixed to a given value. From a theoretical standpoint, we show that, if the training set is not linearly separable, the minimization of an L-p error norm provides an approximation to the minimum error classifier, provided that the perceptron is highly saturated. Moreover, if data are linearly separable, the perceptron approximates the maximum margin classifier.
minimum misclassification error single layer perceptron large margin classifier