# Robustness of Artificial Neural Networks Based on Weight Alterations Used for Prediction Purposes
## 3 Methodology
### Replication of the Most Important Neurons/Connections  
Next, we present a method in order to achieve robustness for ANNs based on the replication of the most important neurons/connections, addressing the issue of robustness based on redundancy. Different methods have previously been used for robustness, but these are often restrictive. Most cases are based on replicating complete ANN architecture or replicating a certain number of complete layers in the ANNs. Based on past studies, ANNs are not always fault-tolerant and indicate the need for more robust methods [3]. Based on the results presented in the previous section for each individual ANN, as well as the PARSEC benchmark suite workloads, we managed to identify the most important neurons/connections in the presence of weight (bit) errors. Our method focuses on the replication of these parts in order to achieve robustness for the developed ANNs. In order to check how many redundant neurons/connections were needed for the different ANN sizes to achieve good robustness levels, different numbers of neurons were replicated and simulated. Figure 6 presents the results from simulations of different numbers of neuron replications for different ANN sizes. Figure 6: _Cont._  
The results show that good robustness results are achieved for the 4 \(\times\) 5 ANN with 12 neuron replications. For the 4 \(\times\) 6 ANN, good results are achieved with 14 neuron replications. For the 5 \(\times\) 6 ANN, good robustness results are achieved with 17 neuron replications. Figure 7 presents the structure of the neural network with redundant neurons/connections (redundant neurons/connections are shown in red). Figure 8 shows the complete process/steps of our method. Based on the previous results, since the weight alterations are already known, we determined the most important neurons/connections for specific ANNs under different PARSEC workloads. Next, we replicated the above important neurons/connections of the  
Figure 8: Steps for replicating the most important neurons/connections for robustness. Figure 6: Simulation results for different ANN sizes and different numbers of neuron replication. Figure 7: Structure of the neural network with redundant neurons/connections. specific ANN, changing the architecture of the ANN in order to increase its robustness. Based on the new and altered ANN architecture, we carried out simulations to check the prediction accuracy of different ANNs. Different simulations are developed with randomly chosen weights and bits, as explained before. We tested all the possible random weight bit/bits alterations for one and more weight errors. Using the new prediction simulation results, we verified the robustness of the ANNs. In our method, robustness is achieved via the actual ANN process itself by using the high connectivity of the neurons in the ANNs. Moreover, our method tries to reduce the redundancy needed (in comparison to previous methods) in order to minimize expenses in terms of the additional units and links. Our method is easier to implement in practice in comparison to the other methods that require replications of more neurons/connections (even entire ANNs), and this makes them impractical in reality. Figure 9 shows that the robust method analyzed above (with redundant neurons/connections) helps to increase the robustness of the ANNs used for prediction purposes. The figure shows the average percentages of correct predictions for each of the above-mentioned cases with redundancy in comparison to cases with no weight faults. The below results show that, by replicating the most important neurons/connections, ANNs can retain a high robustness level (around 98% prediction accuracy); these results are reproducible for larger networks too. The number of input-hidden-output units remains the same in our study, with only the most important neurons/connections changing. By replicating only these, we kept the overheads to a minimum in comparison to the previous studies that replicated the complete ANN or complete layers x times.