# Deep Learning Forecasting for Supporting Terminal Operators in Port Business Development
## 3 Materials and Methods
### Methods
#### 3.1.1 Fully Connected Neural Networks  
In feedforward networks, the information propagates along the synapses from the input to the output layer in only one direction. Each layer transforms the data, and successive transformations are chained to produce a composition of functions.  
We denote by \(\mathbf{x}=(x_{1},x_{2},\ldots,x_{d})^{\top}\in\mathcal{X}\) the vector of features. Mathematically, a \(q\in\mathbb{N}\)-dimensional feedforward (or fully connected) neural network layer can be seen as a vectorial function that maps the training samples in a new space. The coordinates of the data in the new space are the output of the neurons in the layer, and each unit computes the nonlinear function \(z_{i}(\mathbf{x})\) as follows:  
\[z_{i}(\mathbf{x})=\phi\bigg{(}w_{i,0}+\sum_{l=1}^{d}w_{i,l}x_{l}\bigg{)}\qquad i=1, 2,\ldots,q,\]  
where \(w_{i,l}\in\mathbb{R}\) are coefficients to learn and \(\phi(\cdot)\) denotes the activation function. The most popular choices of the activation function in the machine learning literature are the sigmoid, hyperbolic tangent, or rectified linear units, where the latter is known as a ramp function outside the machine learning parlance. In matrix form, the output \(\mathbf{z}(\mathbf{x})=(z_{1}(\mathbf{x}),z_{2}(\mathbf{x}),\ldots,z_{q}(\mathbf{x}))^{\prime}\in \mathbb{R}^{q}\) can be also expressed as  
\[\mathbf{z}(\mathbf{x})=\phi(\mathbf{w}_{0}+W\mathbf{x}), \tag{1}\]  
where \(\mathbf{w}_{0}=\{w_{i,0}\}_{1\leq i\leq q}\in\mathbb{R}^{q}\) and \(W=\{w_{i,l}\}_{1\leq i\leq q,1\leq l\leq d}\in\mathbb{R}^{q\times d}\).  
If the network has a single layer, the new features \(\mathbf{z}(\mathbf{x})\) are used directly to calculate the output. Otherwise, multiple layers are staked. The output of each layer represents the input of the next one and so on for the following layers. A network with several layers is called a deep network. Considering a deep network architecture with \(m\in\mathbb{N}\) layers, the mechanism behind can be formalised as follows:  
\[\mathbf{z}^{(1)}(\mathbf{x}) =\phi_{1}\bigg{(}\mathbf{w}_{0}^{(1)}+W^{(1)}\mathbf{x}\bigg{)}\] \[\mathbf{z}_{j}^{(2)}(\mathbf{x}) =\phi_{2}\bigg{(}\mathbf{w}_{0}^{(2)}+W^{(2)}\mathbf{z}^{(1)}(\mathbf{x}) \bigg{)}\] \[\ldots\] \[\mathbf{z}^{(m)}(\mathbf{x}) =\phi_{m}\bigg{(}\mathbf{w}_{0}^{(m)}+W^{(m)}\mathbf{z}^{(m-1)}(\mathbf{x}) \bigg{)},\]  
where \(q_{k}\) indicates the number of units in the \(k\)-th layer (with \(q_{0}=d\)), \(\mathbf{w}_{0}^{(k)}\in\mathbb{R}^{q_{k}}\), \(W^{(k)}\in\mathbb{R}^{q_{k}\times q_{k-1}}\) are the coefficients to optimise, and \(\phi_{k}(\cdot)\) is the activation function applied by the \(k\)-th layer. The network learning is carried out by optimising the coefficient matrices and vectors according to a specific loss function. It poses the minimisation of a high-dimensional non-convex function, and the backpropagation algorithm is generally used. The coefficients are iteratively adjusted by minimising the error of the network outputs with respect to some reference values.  
An extensive description of neural networks and backpropagation can be found in [32].