## 2 Load Profiling Mathematical Background
### Clustering Algorithms
#### 2.2.4 Neural Network-Based Clustering Algorithms  
The artificial neural networks used in clustering are based on the concept of competitive learning or on energy function minimization. The latter is employed in Hopfield Neural network, which is a recurrent neural network with full weight connection among the neurons [57; 73]. When an input is presented in the network, the weights are re-arranged in order to reach the minimum energy state. The weights represent the distances between patterns and centroids. The competitive learning operates differently. The competition refers to the neurons response to the input pattern. The neurons have the capability of affecting positively or negatively, or even not affecting at all, the other neurons. The neuron that wins the competition has the highest activation value. The weight update is held  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & \(\mathbf{a_{i}}\) & \(\mathbf{a_{j}}\) & \(\mathbf{\beta}\) & \(\mathbf{\gamma}\) \\ \hline Single Linkage (SL) & 0.50 & 0.50 & 0 & 0.50 \\ Complete Linkage (CL) & 0.50 & 0.50 & 0 & 0.50 \\ Unweighted Pair Group Method Average (UPGMA) & 0.50 & \(\frac{N_{i}}{N_{i}+N_{j}}\) & 0 & 0 \\ Weighted Pair Group Method Average (WPGMA) & 0.50 & 0.50 & 0 & 0 \\ Weighted Pair Group Method Centroid (WPGMC) & 0.50 & 0.50 & \(-0.25\) & 0 \\ Unweighted Pair Group Method Centroid (UPGMC) & \(\frac{N_{i}}{N_{i}+N_{j}}\) & \(\frac{N_{i}}{N_{i}+N_{j}}\) & \(\frac{N_{i}N_{j}}{(N_{i}+N_{j})^{2}}\) & 0 \\ Minimum Variance Method (MVM) or the [PERSON]’s method & \(\frac{N_{i}+N_{j}}{N_{i}+N_{j}+N_{i}}\) & \(\frac{N_{i}+N_{j}}{N_{i}+N_{j}+N_{i}}\) & \(-\frac{N_{i}}{N_{i}+N_{j}+N_{i}}\) & 0 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Coefficients of the hierarchical agglomerative algorithms.  
in a way that includes the addition of the input vector. The neural network that is based on the Adaptive Vector Quantization (AVQ) algorithm is composed by an input layer and an output layer [27]. A \(D\)-dimensional input \(y_{n}\) is presented in the input layer. The winning neural is activated by receiving the value \"1\" while the rest receive the value \"0\" [94]. The weight update \(w_{k}\) of the winning neuron \(k\) at iteration \(t\) is given by:  
\[w_{k}(n+1)=w_{k}(n)+\eta(t)(y_{n}-w_{k}(n))z_{k} \tag{15}\]  
where \(n\) is the number of patterns that have been presented in the input layer during iteration \(t\), \(w_{k}(n)\) is the weight of the \(k\)th neuron at iteration \(t\), \(\eta\) is the learning rate that is a decreasing function of time and depends of the following parameters: Initial value \(\eta_{0}\) and total number of epochs \(T\). The parameter \(z_{k}\) corresponds to the output of the \(k\)th neuron and is given by:  
\[z_{k}=\left\{\begin{array}{ll}1&\text{if }d_{eucl}(y_{n},w_{k}(n))\leq d_{ eucl}(y_{n},w_{s}(n)),\ s=1,...,K\\ 0&\text{otherwise}\end{array}\right. \tag{16}\]  
The SOM is the most commonly used unsupervised machine learning neural network. The input patterns are arranged on a surface based on their similarity. Each neuron is connected with weights with the input layer and receives a complete copy of the input pattern [95]. A neuron positively affects the neighbouring neurons and negatively the most distant ones. A competition takes place among the neurons in response of the input pattern. The weight update \(w_{k}\) of the winning neuron \(k\) at iteration \(t\) is given by:  
\[w_{k}(t+1)=w_{k}(n)+a(t)h_{c}^{(k)}(y_{n}(t)-w_{k}(t)) \tag{17}\]  
where \(a\) is the learning rate and \(h_{c}^{(k)}\) is the neighbourhood kernel around the winning neuron \(k\).