# Clustering Analysis of Wind Turbine Alarm Sequences Based on Domain Knowledge-Fused Word2 vec
## 3 Basic Methods
### Word2 vec  
Word embedding techniques can be used to convert words from plain text into real-valued representations. Word2 vec, being a form of word embedding, has proven to be exceptionally proficient in producing numeric representations for textual data. Transforming textual content into numerical representations facilitates subsequent analysis using algorithms such as cluster analysis [25]. Based on this functionality, Word2 vec has been used in the domain of fault diagnosis for analyzing text from maintenance records [26]. Regarding alarm data, Word2 vec has been applied to learn its vector representation, enabling the simultaneous capture of relationships between alarms [21; 22]. Figure 3: The percentage distribution of daily alarm trigger numbers in different segments. Figure 2: The average daily alarm trigger number and the maximum daily alarm trigger number for each wind turbine. Skip-gram model serves as an implementation technique for Word2 vec. Four key terminologies are employed in the model, namely, word, vocabulary, sentence, and corpus. Words are the fundamental units in textual descriptions. A vocabulary is a collection of all available words. A sentence is composed of multiple words and is used to describe an event. A corpus represents the collection of all available sentences. The objective of the training is to obtain vector representations for words based on all the sentences in the corpus, such that these vector representations can capture the correlation of the neighboring words within sentences. The skip-gram model is a neural network architecture, which is shown in Figure 4. The input layer consists of one-hot encoded vectors representing the target word. Hence, it has a dimension of \(V\) corresponding to all the words in the vocabulary. One input word is expressed as \(w_{I}\). Following the input layer, a hidden layer of dimension \(N\) is connected. In this case, \(N\) represents the dimensionality of the word embedding. Between the input layer and the hidden layer, there exists a transformation facilitated by a weight matrix \(W_{V\times N}\). Each row \(\mathbf{v}_{w_{I}}\) within this matrix corresponds to a vector representation of a respective word, and these vector representations are continually refined during training. The values across each dimension of the vector fall within the range of \(-1\) to \(1\). Finally, the output layer consists of \(C\) panels, each panel containing \(N\) nodes. Here, \(C\) corresponds to the length of the contextual window, which represents the number of neighboring words under consideration. \(N\) signifies the dimension of the word embedding. The hidden layer and the output layer are connected via a weight matrix \(W^{\prime}_{V\times N}\), which remains consistent across each panel. When the semantic window length is \(c\in\mathbb{C}=\{1,2,3,\ldots C\}\), the output of the \(j\)-th word in the vocabulary is denoted as \(w_{c,j}\). Its corresponding vector representation is denoted as \(\mathbf{v}^{\prime}_{w_{j}}\in\mathbb{R}^{N}\), which is a row vector of the output weights \(W^{\prime}_{V\times N}\). The objective of the skip-gram model training is to discover word vector representations that aid in predicting context words within sentences. For a given input word \(w_{I}\), the probability associated with the actual \(c\)-th context word \(w_{O,c}\) being \(w_{c,j}\) is represented as  
\[p(w_{c,j}=w_{O,c}|w_{I})=\frac{\exp(u_{c,j_{c}^{*}})}{\sum_{j^{ \prime}=1}^{V}\exp(u_{j^{\prime}})}, \tag{1}\]  
where \(u_{c,j}=u_{j}=\mathbf{v}^{\prime}_{w_{j}}{}^{T}\cdot\mathbf{h}\) represents the net dot product of the \(j\)-th word in the \(c\)-th panel, \(j_{c}^{*}\) is the index of the actual output context word occurring at the \(j\)-th index of the \(c\)-th, \(j^{\prime}\) is the index for words of the output layers. Figure 4: The Neural network architecture of the skip-gram model. Multiple panels in the output layer are represented by different colors.