## 3 Bin2 Vec: A Convolutional Neural Network-Based WBM Coloring Scheme
### Word2 Vec and Bin2 Vec  
#### 3.2.1 Word2 Vec  
Word2 Vec is the most widely used neural network-based distributed representation method that represents a word by a numerical vector in a fixed dimension [27; 28]. Traditional one-hot encoding cannot preserve the semantic relationship because a word is represented by \(|V|\)-dimensional vector (\(|V|\) is the number of unique words in the dataset) with only one element with the value 1, and all other elements being 0. In contrast, Word2 Vec can preserve the semantic relationship between words. There are two Word2 Vec structures: continuous bag-of-words (CBOW) and Skip-Gram that have the exactly reversed structures. CBOW takes the neighborhood words as the input of the network and is trained to correctly predict the targeted word, whereas Skip-Gram takes the targeted word as the input of the network and is trained to correctly predict the neighborhood words. We adopted the Skip-Gram structure, as shown in Figure 3[27], since it is known to be more effective in learning infrequent words. The bin codes in this study are equally important regardless of the frequency of their appearance. The Skip-Gram model consists of the input, projection, and output layers. Assume that we are attempting to find the \(d-\)dimensional word vectors for a total of \(|V|\) words in the dataset (\(d<<|V|\)), and the window size (the number of neighborhood words to predict in either direction from the targeted word) is set to \(c\). The input layer takes the one-hot encoding vector of the targeted word, whereas the output layer consists of the one-hot encoding vectors of the words surrounding the targeted words. The projection layer consists of \(d\) hidden nodes such that the hidden weight matrix \(\mathbf{W}\) between the input layer and projection layer becomes \(|V|\times d\), and the hidden weight matrix \(\mathbf{W}^{\prime}\) between the projection layer and output layer becomes \(d\times|V|\). After the training process is completed, either the row vector of \(\mathbf{W}\) or the column vector of \(\mathbf{W}^{\prime}\) can be used as word vectors. \begin{table}
\begin{tabular}{c c} \hline
**Value** & **Description** \\ \hline \(-2\) & Non-wafer part \\ \(-1\) & Untested portion \\ \(0\) & Not all results from 1 to 9999 \\ \(1\)\(\sim\)9999 & Code indicating a set of EDS tests that the die does not pass \\ \hline \end{tabular}
\end{table} 
Table 1: Values that can be assigned to WBM image pixels. Figure 2: Example of WBM. The purpose of the Skip-Gram model is to maximize the probability of generating the actual surrounding words for a given target word. More specifically, given a sequence of training words \(w_{1},\ w_{2},\ w_{3},\ \cdots,\ w_{T}\), the objective function of the Skip-Gram model is to maximize the average log probability as shown in Equation (1):  
\[\max\ \ \frac{1}{T}\sum_{t=1}^{T}\sum_{-c\leq j\leq c,j\
eq 0}\log p(w_{t+j}|w_{t}). \tag{1}\]  
The probability of generating the surrounding (output) words given the target (context) word is computed using the softmax function as shown in Equation (2):  
\[p(o|c)=\frac{\exp(u_{0}^{T}v_{c})}{\sum_{w=1}^{W}\exp(u_{0}^{T}v_{c})}, \tag{2}\]  
where \(u\) is the column vector of the hidden weight matrix \(\mathbf{W}^{\prime}\), and \(v\) is the column vector of the hidden weight matrix \(\mathbf{W}\). The Skip-Gram model learns the weight matrices \(\mathbf{W}\) and \(\mathbf{W}^{\prime}\) by stochastic gradient descent optimization method. There are two practical drawbacks of Equation (2). First, it is burdensome to compute the denominator because all the words must be used to compute the softmax function. Second, frequent words such as \"the\" or \"it\" are over-trained, whereas infrequent words cannot be trained sufficiently. To resolve these issues, the negative sampling technique [29] is commonly used.