# Efficient Implementation of Multilayer Perceptrons: Reducing Execution Time and Memory Consumption
## 2 Materials and Methods  
### Proposed Approach  
In the event of saving the neural network configuration, it is necessary to emphasize how to store the weights of the connections, because this is what occupies most space and the other data hardly take up any [8]. The first option we find is to save the weights in a square matrix where one of the dimensions would be the input neurons and the other the output dimensions. With this technique, we can keep all the weights without any problem, the only thing we have are several positions that are at zero, pointing out that there are no connections between them.  
A way of not consuming so much memory would be to eliminate those values that are useless. By not storing values that are never actually used, we will manage to avoid having so many cache failures and we will be able to take better advantage of the location of the data [9; 10]. To achieve this, we can look at the literature where we would find out that this is an issue that occurs sometimes, and the matrices with values which do not serve us are called the sparse matrix [11]. What is performed with this is to eliminate all the positions that do not work for us by storing the matrix otherwise. Although there are different implementations of using the sparse matrix, all of them have something in common, that is, they are only profitable to use if you have a lot of unusable values and massive data. Generally, useful data should be around 10% at most, and you should have several thousands of records. If we analyze the weight matrices of our neural networks, the weights occupy approximately 30% of the total matrix, and we also do not meet the condition of several thousands of records since we are not close to that number. This fact will be discussed in detail in later sections.  
We should therefore analyze the feedforward networks, as, like it says in the literature, these networks can be stored in an upper triangular matrix, as shown in Figure 2b [12; 13].  
Through this, an important memory saving method is achieved, since half of the storage is used, with the triangular matrix, although there are still many values that are not employed. This is due to the fact that the connectivity of an MLP is feedforward from layer to layer, implying that those neurons that are in the same layer, or more than one layer away, will not be linked by a weight.  
Furthermore, if we look at how these networks are stored, it is possible to observe that they follow a common pattern (see Figure 2). These are collected in small subarrays, which are defined by the PEs in one layer and in the next. Since it is feasible to know how many weights each of these networks has, we can store these submatrices, a method which is based on the present paper.  
As described above, the proposed solution will involve storing only the values that are useful in an array. If, for instance, we use a network with 4 neurons in the input layer, 2 in the output layer and 3 in the only hidden layer, as per the example in Figure 2, it is shown how much memory is used to store the network. In Table 1, it is also possible to see how much space is used.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Approach** & **Cells** & **Saving** \\ \hline Matrix & 81 & - \\ Superior triangular matrix & 45 & 44.4\% \\ Proposed approach & 18 & 77.7\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Amount of memory used to store the weights of the different approximations used in Figure 2.