# Model Predictive Control of Non-Linear Systems Using Tensor Flow-Based Models
## 2 Developing a System Model with Tensor Flow  
A successful identification of a process's discrete model is a task highly dependent on the choice of its structure and number of variables. This task becomes even more challenging when it is necessary to employ non-linear models to approximate the system's input/output behavior. In the present work, it is used an ANN based on the _TensorFlow_ implementation, using a regression variable's vector according to the Auto-Regressive model with eXogenous inputs (ARX) linear model structure--an approach referred in literature as Neural Network Auto-Regressive with eXogenous inputs (NNARX) [22]. The ANN is organized as a Multilayer Perceptron (MLP), an architecture known to be a proper choice for black-box modeling and system identification due to its scalability and universal approximator capability.  
Figure 1: System Architecture for the implementation of a Tensor Flow model-based control system.  
Regarding model structure, we set the activation functions of the hidden layer nodes to be hyperbolic tangent and in the output node as a linear activation function. This setup enables a good balance between model dimensionality and fitness of the model to non-linear systems. In what concerns to the parameters tunning, this procedure is accomplished through the execution of an optimization problem. Optimization is the task of minimizing or maximizing a cost function \(f(x)\) by varying \(x\). When training neural networks, it is frequently to use the mean squared error as cost function, as defined by \(J\) in Equation (1), and our goal is to find the set of parameters that minimize its value:  
\[J(W,Z^{N})=\frac{1}{2N}\sum_{n=1}^{N}[y(n)-\hat{y}(n|W)]^{2}+\lambda W^{\top}W, \tag{1}\]  
where \(N\) is the number of samples, \(Z^{N}\) is the set of data containing the inputs, \(y(n)\) is the expected output, \(\hat{y}(n|W)\) is the output calculated by the ANN, \(W\) is a vector of all ANN weights and \(\lambda\) is the regularization parameter that penalizes weights with high values. This parameter must be a positive value and its magnitude must be selected so it does not become the driving factor of the cost function as the main objective is to minimize the error between the training data set and the output generated by the neural network.  
The gradient descent is often the chosen iterative optimization algorithm to find a non-linear function minimum [23]. However, one of the disadvantages of this method is that it becomes time-consuming if the data set is large and the network has multiple internal layers. Nevertheless, to have an ANN with good generalization, a huge amount of data is required making the training a heavy task. To overcome this limitation, the Stochastic Gradient Descent (SGD) was developed as an extension of the gradient descent [24], in which the main difference is that instead of using all the samples available in the training data set, it uses only a smaller portion of this data in each step of the algorithm, making the training faster. This mini-batch of samples are drawn uniformly from the training data set. We use the SGD algorithm to train our neural network.