  We review mathematical foundations of convolutional neural nets (CNNs) with
the goals of: i) highlighting connections with techniques from statistics,
signal processing, linear algebra, differential equations, and optimization,
ii) demystifying underlying computations, and iii) identifying new types of
applications. CNNs are powerful machine learning models that highlight features
from grid data to make predictions (regression and classification). The grid
data object can be represented as vectors (in 1D), matrices (in 2D), or tensors
(in 3D or higher dimensions) and can incorporate multiple channels (thus
providing high flexibility in the input data representation). For example, an
image can be represented as a 2D grid data object that contains red, green, and
blue (RBG) channels (each channel is a 2D matrix). Similarly, a video can be
represented as a 3D grid data object (two spatial dimensions plus time) with
RGB channels (each channel is a 3D tensor). CNNs highlight features from the
grid data by performing convolution operations with different types of
operators. The operators highlight different types of features (e.g., patterns,
gradients, geometrical features) and are learned by using optimization
techniques. In other words, CNNs seek to identify optimal operators that best
map the input data to the output data. A common misconception is that CNNs are
only capable of processing image or video data but their application scope is
much wider; specifically, datasets encountered in diverse applications can be
expressed as grid data. Here, we show how to apply CNNs to new types of
applications such as optimal control, flow cytometry, multivariate process
monitoring, and molecular simulations.
