# UnifiedCut: A Simple and Efficient Neural Model for Thai, Burmese and Khmer Word Segmentation
## 3 Methodology
### UnifiedCut Model
#### 3.1.2 Transformer Encoder  
We use a transformer encoder [25] to learn contextual features from an input sliding window of n-gram embeddings \(E\in\mathbb{R}^{n\times d_{m}}\) of length \(n\),  
\[E=[\mathbf{e}_{1},\mathbf{e}_{2},\cdots,\mathbf{e}_{n}]. \tag{3}\]  
The sequence of the input n-gram embeddings is fed into the transformer encoder. The transformer layer consists of two sub-layers, which are named the multi-head attention layer and the position-wise feed-forward layer. The encoder employs residual connections around each of the sub-layers, which is followed by layer normalization [26].  
Before applying the self-attention to the input sequence, the model firstly transforms the input \(E^{j}\) into query \(Q\in\mathbb{R}^{n\times d_{k}}\), keep \(K\in\mathbb{R}^{n\times d_{k}}\), and value \(V\in\mathbb{R}^{n\times d_{k}}\) sequences via  
Figure 1: Overview of the UnifiedCut architecture. The UnifiedCut model performs word segmentation by using multiple n-grams of characters as input. For each sliding window over the input sequence, n-gram embeddings are concatenated and fed into a Transformer encoder to capture contextual features. The encoder outputs are then flattened and passed through fully connected layers with ReLU activation. Finally, a classifier predicts the segmentation label (B or I) for each target character. This architecture enables efficient segmentation by utilizing sliding windows and multi-head attention for parallel processing.  
separate linear transformation. Then, we calculate the self-attention outputs \(H\in\mathbb{R}^{n\times d_{v}}\) by dot-product self-attention  
\[H=\text{Attention}(Q,K,V)=\text{softmax}(\frac{QK^{\top}}{\sqrt{d_{k}}})V. \tag{4}\]  
Multi-head attention enables the model to simultaneously focus on information from different representation subspaces at various positions. This allows our model to effectively learn multi-view word segmentation features from the surrounding context. The output of the multi-head attention layer is a projection formed by concatenating the outputs of all attention heads,  
\[\begin{split}\text{MultiHead}(Q,K,V)=\text{Concat}(\text{head} _{1},\cdots,\text{head}_{h})W^{O},\\ \text{where }\text{head}_{i}=\text{Attention}(QW^{Q}_{i},KW^{K}_{i},VW^{V}_{i}),\end{split} \tag{5}\]  
where \(W^{Q}_{i}\in\mathbb{R}^{d_{\text{m}}\times d_{v}}\), \(W^{K}_{i}\in R^{d_{\text{m}}\times d_{k}}\), and \(W^{V}_{i}\in\mathbb{R}^{d_{\text{m}}\times d_{v}}\) are the projection parameters of head \(i\). \(W^{O}\in\mathbb{R}^{hd_{v}\times d_{\text{model}}}\) denotes the output projection matrix. For each parallel attention head, we use \(d_{k}=d_{v}=d_{\text{m}}/h\).  
After the multi-head attention layer, a LayerNorm(X + MultiHead(X)) layer is applied to the sum of the output and residual connection of X.  
In addition to attention sub-layers, the encoder block contains a fully connected feed-forward network applied to each position separately and identically. The feed-forward network consists of two linear transformations with a ReLU activation in between.  
\[\text{FFN}(z)=\text{ReLU}(W_{1}z+b_{1})W_{2}z+b_{2}, \tag{6}\]  
where \(W_{1}\), \(b_{1}\), \(W_{2}\), are the \(b_{2}\) projection parameters, and \(z\) denotes the output of the previous layer normalization. Another layer normalization is applied after the feed-forward network, which makes the outputs \(o_{i}\) of the encoder block.  
We derive the contextual prediction feature from the multiple input n-grams by flattening the transformer's encoder output into a single vector,  
\[h=\text{Concat}([o_{1},o_{2},\cdots,o_{n}]). \tag{7}\]