## 2 Artificial Neural Networks
### The Activation Function of an Artificial Neural Network  
The activation function is a function that receives an input signal and produces an output signal after the input exceeds a certain threshold. That is, neurons receive signals and generate other signals [66]. The neuron start is only performed when the sum of the total inputs is greater than the neuron threshold limit, then the output will be transmitted to another neuron or environment. This threshold limit determines whether the neuron is activated or not, the most common activation or transfer functions are the linear, binary step, piecewise linear, sigmoid, Gaussian and hyperbolic tangent functions [67].  
Table 1 shows the activation functions commonly used in NNs. The behavior of neurons is defined by these functions. If it transfers a function that is linear and the network is multi-layered, it can be represented as a single-layer network, since it is product of weight matrices of each layer and will only produce positive numbers over the entire range of real numbers. On the other hand, non-linear transfer functions (sigmoid function) between layers allow multiple layers to provide new capabilities, adjusting the weights to obtain a minimum error in each set of connections between layers [68; 69]. Linear functions are generally used in the input and output layers, while non-linear activation functions can be used for the hidden and output layers [70].  
The most used non-linear activation functions are sigmoid and hyperbolic tangents. Hyperbolic and sigmoid tangents are mainly used because they are differentiable and make them compatible with the back-propagation algorithm. Both activation functions have an \"S\" curve, while their output range  
Figure 3: The basic scheme of a neuron.  
is different [71]. The sigmoid function is the most used activation function in ANNs. This function varies from 0 to +1, although the activation function sometimes seeks to oscillate between \(-1\) and +1, in which case the activation function assumes an antisymmetric form with respect to the origin, defining it as the hyperbolic tangent function [72; 73; 74].  
Direct implementation of sigmoid and hyperbolic tangent functions in hardware is impractical due to its exponential nature. There are several different approaches to the hardware approximation of activation functions, such as the piecewise linear approximation. Linear part approximations are slow, but they are the most common way of implementing activation functions [75]. In addition, this uses a series of linear segments to approximate the trigger function. The number and location of these segments are chosen so that errors and processing time are minimized [76].  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **Graphic** & **Function** \\ \hline Linear & & \(f(\xi)=a\cdot\xi+b\) \\ \hline Binary step & 1 & if \(\xi\geq 0\), & then \(f(\xi)=1\), \\  & & if \(\xi<0\), & then \(f(\xi)=0\), \\ \hline Piecewise linear & 1 & if \(\xi\  
The Gaussian activation function can be used when finer control over the activation range is needed [69]. Furthermore, it can uniformly perform continuous function approximations of various variables [77].