On the Classification Consistency of HighDimensional Sparse Neural Network Artificial neural networks (ANN) is an automatic way of capturing linear and nonlinear correlations, spatial and other structural dependence among features and output variables. This results good performance in many application areas such as classification and prediction from magnetic resonance imaging, spatial data and computer vision tasks. Most commonly used ANNs assume availability of large training data compared to dimension of feature vector. However, in modern days applications, such as MRI applications or in computer visions the training sample sizes are often low, and may be even lower than the dimension of feature vector. In this paper, we consider a single layer ANN classification model that is suitable for low training sample. Besides developing the sparse architecture, we also studied theoretical properties of our machine. We showed that under mild conditions, the classification risk converges to the optimal Bayes classifier risk (universal consistency) under sparse group lasso regularization. Moreover, we proposed a variation on the regularization terms. A few examples in popular research fields are also provided to illustrate the theory and methods.

Sparse Wavelet Networks A wavelet network (WN) is a feedforward neural network that uses wavelets as activation functions for the neurons in its hidden layer. By predetermining the wavelet positions and dilations, the WN can turn into a linear regression model. The common approach for the construction of these WN families is to use leastsquares type algorithms. In this letter, we propose a novel approach by formulating a WN as a sparse linear regression problem, which we call a sparse wavelet network (SWN). In this WN, the problem of calculating the unknown inner parameters of the network becomes that of finding the sparse solution of an underdetermined system of linear equations. Our sparse solution algorithm is a nonconvex sparse relaxation approach inspired by smoothed L0 (SL0), a distinguished sparse recovery algorithm. The proposed SWN can be applied as a tool for the prediction and identification of dynamical systems.