# Application of Neural Networks for Water Meter Body Assembly Process Optimization
## 2 Materials and Methods
### Assumptions of a Neural Network
Moreover, in the model of the neuron there may exist additional, independent signal inputs, with constant value +1. This signal, same as the rest of the signals, is scaled by its own weight, which is calculated during training process. The neural network consists of three layers: input (18 neurons), output (1 neuron) and one hidden (3 neurons). The network architecture, mainly the size of the hidden layer, was selected empirically, taking into account the accuracy of the results prediction. Performance of the ANN is highly affected by selection of activation functions. Those functions, based on the sum of weighted signals, set neuron's output. The most common activation functions are: linear, where argument is passed directly to the output; logistic (S-shaped curve with output range from 0 to 1); hyperbolic (with output values within the range of \(-1\) to 1); and exponential. Selecting the correct ANN learning algorithm has a crucial impact on the network performance. Learning algorithm iteratively changes the weights assigned to neurons inputs, trying to minimize the output error calculated in each iteration, based on the data set, which were fed to the network. In this paper, from a variety of ANN learning techniques, the main focus was put on the method of the steepest descent (SD), the BFGS (Broyden-Fletcher-Golfarb-Shanno) method and the conjugate gradient method (CG). The steepest descent method is a modification of the simple gradient descent method. Both methods are iterative optimization algorithms used to find extremum of the function. In contrast to the simple gradient descent method, where search direction proceeds with constant arbitrary selected step size, in the SD method, step size can vary, to ensure the biggest decrease in the value of the function at the new point. Characteristic feature in SD method in each search direction is orthogonal to the one from previous iterations. Implementation of SD requires significant number of searches, which is resource intensive and time consuming. Mentioned issues can be overcome using conjugate directions method. The goal of that algorithm is to determine the search direction on the multidimensional error surface. This algorithm can be applied only for equations with positive-defined matrixes. Then, a straight line is drawn over the error surface in this direction and the minimum of the error function is determined for all points along the straight line. After finding the minimum along the initially given direction, a new search direction is determined from the location of this minimum and the whole process is repeated. Thus, there is a constant shift towards decreasing values of the error function until a point with the minimum of the function is found. It should be ensured that the second derivative determined in this direction is zero during the subsequent learning steps. To keep the second derivative value of zero, the conjugate directions to the previously chosen direction are determined, since moving in the conjugate direction does not change the fixed (zero) value of the second derivative computed along the previously selected direction. In determining the conjugate direction, it is assumed that the error surface is parabolic in shape. The Broyden-Fletcher-Goldfarb-Shanno algorithm is the second order algorithm that belongs to quasi-Newton algorithm class. BFGS finds the direction of function descent by iterative improves the approximation of the Hessian matrix (matrix of second order partial derivatives) of the loss function. Algorithm does not need to calculate the precise inverse of Hessian matrix in each iteration (it requires only its approximation), thereby it requires less computing power. For radial networks, standard learning procedures are applied, including center determination using k-means method, deviation determination using k-neighbor method, and then optimization of output layer. The k-means method is a method that aims to find and extract groups of similar objects (clusters). Thus, k different clusters will be created.