# From Ethnographic Research to Big Data Analytics--A Case of Maritime Energy-Efficiency Optimization
## 4 Applying Big Data Analytics for Maritime Energy Efficiency
### Constructing Prediction Models
#### 4.2.1 Multi-Layer Perception (MLP)  
The multi-layer perception (MLP) is organized in layers, each layer consisting of multiple nodes (neurons). Each neuron has input connections and creates an output that is propagated to other nodes in the following layer. Two following layers are fully connected to each other. Weights are assigned to each connection (\(w_{P\text{q}}\)\({}^{(0)}\) with \(p=0...m\)\({}^{(1\text{-}1)}\) and \(q=0...m\)\({}^{(1)}\)). The output depends on the weighted inputs and an activation function. The number of nodes and the used activation function can vary between layers. For convenience the 0 th layer is named _input layer_, the (L + 1) th _output layer_, and all other layers are _hidden layers_. The number of nodes in each layer (m\({}^{(0)}\) + 1) is called the dimension of layer l. Values set in the input layer are typically described as input vector x = (x0...xD)\({}^{\text{T}}\) and values of the output layer as vector \(\hat{y}\) = (y0...yC). Weights for layers are described via matrices. The dimension of the input layer is determined by the number of input dimensions from the data sets, see Equation (2). A classic multi-layer perceptron introduced by [PERSON] [52] is described in Figure 5.  
\[y_{q}^{(l)}=\text{a}\;(\sum\
olimits_{i=0}^{p}w_{i\text{q}}^{(l)}*\;y_{i}^{(l- 1)}) \tag{2}\]  
A data row can be split into an input and an output vector. For a model that predicts fuel consumption, Fuel Mass Flow Heavy Fuel Oil Main Engine (HFO ME) Group 1+2 and Fuel Mass Flow Maritime Diesel Oil (MDO) ME Group 1+2 are defined as \(y_{0}\) and \(y_{1}\). A choice of parameters such as AWS knots, LWS (longitudinal water speed) knots, relative wind speed (m/s), and others are chosen for x (see Table 1 in the Appendix A for explanation of parameter abbreviations). To use the model for predictions, x is set and the network will compute \(\hat{y}\) using all \(w^{(\text{L})}\). Cost (\(\hat{y}\), y) defines a cost function that measures the difference between expected output \(y\) and calculated output \(\hat{y}\) (prediction). Training the model means to find a configuration for all weight matrices \(w^{(\text{L})}\) such that the sum of costs for all predictions over the set of training inputs is minimal. Training is performed using back-propagation--an algorithm in which the error of a prediction is propagated backwards into the weight matrices, depending on the weight's contribution to the error as described by [PERSON] et al. [53]. The initial hyper parameters have to be chosen before the model is trained. In this paper hyper parameters are:  
* activation function (logistic, arctan, softsign, rectifier, softplus, gaussian,...);
* learning rate (initial value, time decay);
* learning batch size;
* cost function;
* number of hidden layers;  
Figure 5: Multi-layer perception (MLP) with L hidden layers. * size of hidden layer;
* choice of input parameters. This opens a rather large search space to perform a brute-force search. Through a few benchmarks, an activation function was chosen to be the sigmoid function for all layers, except the output layer, for which we chose the identity function. Our cost function is the MSE (mean-squared error) between normalized predictions and normalized expected outputs. The learning rate is set to an initial value of 0.01 and then determined by the Adam algorithm [54]. Batch sizes were initially tested for values 1, 5, 15, 50, 100, 250, and 450. For further tests 50, 150, and 450 were used. To determine the set of input parameters, number of hidden layers and their dimension we followed the flow-chart depicted in Figure 6.  
The six different sets were evaluated. Their input parameters are grouped into six groups, from A to F. They were listed in Table 4.  
Figure 7 depicts the MSE between predictions of MLP models and validation data, while testing different models with different parameters. The MSE values are the average over 5 runs per model configuration (5-fold). Each configuration was trained with batch sizes of 50, 150 and 450. One can see,