SlowGuess/ACL_2020_OCR huggingface dataset:
Yuanhe Tian $^{\text{♥}}$ , Yan Song $^{\text{♣†}}$ , Fei Xia $^{\text{♥}}$ , Tong Zhang $^{\text{◇}}$ , Yonggang Wang $^{\text{♣}}$
$\hat{\mathcal{O}}$  University of Washington, \*Sinovation Ventures
$\diamond$ The Hong Kong University of Science and Technology
$\text{空}\{\mathrm{yhtian,}$  fxia  $\} @uw.edu$  \*clksong@gmail.com
$\diamond$ tongzhang@ust.hk  $\spadesuit$ wangyonggang@chuangxin.com
Contextual features always play an important role in Chinese word segmentation (CWS). Wordhood information, being one of the contextual features, is proved to be useful in many conventional character-based segmenters. However, this feature receives less attention in recent neural models and it is also challenging to design a framework that can properly integrate wordhood information from different wordhood measures to existing neural frameworks. In this paper, we therefore propose a neural framework, WMSEG, which uses memory networks to incorporate wordhood information with several popular encoder-decoder combinations for CWS. Experimental results on five benchmark datasets indicate the memory mechanism successfully models wordhood information for neural segmenters and helps WMSEG achieve state-of-the-art performance on all those datasets. Further experiments and analyses also demonstrate the robustness of our proposed framework with respect to different wordhood measures and the efficiency of wordhood information in cross-domain experiments.
Unlike most written languages in the world, the Chinese writing system does not use explicit delimiters (e.g., white space) to separate words in written text. Therefore, Chinese word segmentation (CWS) conventionally serves as the first step in Chinese language processing, especially for many downstream tasks such as text classification (Zeng et al., 2018), question answering (Liu et al., 2018), machine translation (Yang et al., 2018), etc.
In the past two decades, the mainstream methodology of CWS treated CWS as a character-based
sequence labeling task (Tseng et al., 2005; Song et al., 2006; Sun and Xu, 2011; Pei et al., 2014; Chen et al., 2015; Zhang et al., 2016; Chen et al., 2017; Ma et al., 2018; Higashiyama et al., 2019; Qiu et al., 2019), where various studies were proposed to effectively extract contextual features to help better predicting segmentation labels for each character (Zhang et al., 2013; Zhou et al., 2017; Higashiyama et al., 2019). Among all the contextual features, the ones measuring wordhood for n-grams illustrate their helpfulness in many nonneural CWS models (Sun et al., 1998; Xue and Shen, 2003; Feng et al., 2004; Song and Xia, 2012).
Later, following the track of the sequence labeling methodology, recent approaches with neural networks are proved to be powerful in this task (Chen et al., 2015; Ma et al., 2018; Higashiyama et al., 2019). However, since neural networks (e.g., LSTM) is considered to be able to provide a good modeling of contextual dependencies, less attention is paid to the idea of explicitly leveraging wordhood information of n-grams in the context as what had previously been done in non-neural models. Although some studies sidestepped the idea by incorporating contextual n-grams (Pei et al., 2014; Zhou et al., 2017) or word attention (Higashiyama et al., 2019) into the sequence labeling process, they are limited in either concatenating word and character embeddings or requiring a well-defined word lexicon. Therefore, it has not been fully explored what would be the best way of representing contextual information such as wordhood features in neural CWS models. Moreover, consider there are various choices of wordhood measures, it is also a challenge to design a framework that can incorporate different wordhood features so that the entire CWS approach can be general while being effective in accommodating the input from any measures.
In this paper, we propose WMSEG, a neural framework with a memory mechanism, to improve
*Partially done as an intern at Sinovation Ventures.
$^{1}$ WMSEG (code and the best performing models) is released at https://github.com/SVAIGBA/WSeg.
Figure 1: The architecture of WMSEG. “ $\mathcal{N}$ ” denotes a lexicon constructed by wordhood measures. N-grams (keys) appearing in the input sentence “部分居民生活水平” (some residents’ living standard) and the wordhood information (values) of those n-grams are extracted from the lexicon. Then, together with the output from the text encoder, n-grams (keys) and their wordhood information (values) are fed into the memory module, whose output passes through a decoder to get final predictions of segmentation labels for every character in the input sentence.
CWS by leveraging wordhood information. In detail, we utilize key-value memory networks (Miller et al., 2016) to incorporate character n-grams with their wordhood measurements in a general sequence labeling paradigm, where the memory module can be incorporated with different prevailing encoders (e.g., BiLSTM and BERT) and decoders (e.g., softmax and CRF). For the memory, we map n-grams and their wordhood information to keys and values in it, respectively, and one can use different wordhood measures to generate such information. Then for each input character, the memory module addresses all the n-grams in the key list that contain the character and uses their corresponding values to generate an output vector to enhance the decoder for assigning a segmentation label to the character. Experimental results from five widely used benchmark datasets confirm that WMSEG with wordhood information can improve CWS over powerful baseline segmenters and outperform previous studies, where state-of-the-art performance is observed on all the datasets. Further experiments and analyses are also performed to investigate different factors affecting WMSEG's performance.
2 The Proposed Framework
Following previous studies, we regard CWS as a character-based sequence labeling task. The architecture of WMSEG is illustrated in Figure 1, where
the general sequence labeling paradigm is the top part with a memory module inserted between the encoder and the decoder. The model predicts a tag (e.g., tag  $B$  for the 1st character in a word) for each character, and the predicted tag sequence is then converted to word boundary in the system output. The bottom part of the figure starts with a lexicon  $\mathcal{N}$ , which is simply a list of n-grams and can be built by various methods (see Section 2.1). Given an input sentence  $\mathcal{X} = x_{1}x_{2}\dots x_{i}\dots x_{l}$ , for each character  $x_{i}$  in  $\mathcal{X}$ , our approach uses the lexicon  $\mathcal{N}$  to generate (keys, values) for  $x_{i}$  and send it to the memory module. In all, the process of WMSEG to perform CWS can be formalized as
$$
\widehat {\mathcal {Y}} = \underset {\mathcal {Y} \in \mathcal {T} ^ {i}} {\arg \max } p (\mathcal {Y} | \mathcal {X}, \mathcal {M} (\mathcal {X}, \mathcal {N})) \tag {1}
$$
where  $\mathcal{T}$  denotes the set of all types of segmentation labels, and  $l$  stands for the length of the input sentence  $\mathcal{X}$ . The output  $\mathcal{V}$  is the corresponding label sequence for  $\mathcal{X}$  with  $\widehat{\mathcal{V}}$  representing the best label sequence according to the model.  $\mathcal{M}$  is the memory module proposed in this paper that consumes  $\mathcal{X}$  and  $\mathcal{N}$  and provides corresponding wordhood information for  $\mathcal{X}$  to maximize  $p$ .
In the rest of this section, we describe the construction of the n-gram lexicon, the proposed wordhood memory networks, and how it is integrated with different encoders and decoders, respectively.
2.1 Lexicon Construction
To build the wordhood memory networks, the first step is to construct the lexicon  $\mathcal{N}$  because the keys in the memory module are built upon  $\mathcal{N}$ , where each n-gram in  $\mathcal{N}$  is stored as a key in it. In this study,  $\mathcal{N}$  is simply a list of n-grams, and technically, it can be constructed through many existing resources or automatic methods. Compared to using an off-the-shelf lexicon or the word dictionary from the training data, it is hypothesized that, for the purpose of incorporating wordhood information into the general sequence labeling framework, unsupervised wordhood measures, such as accessor variety (AV) (Feng et al., 2004), pointwise mutual information (PMI) (Sun et al., 1998), and description length gain (DLG) (Kit and Wilks, 1999), would perform better. For example, AV measures the wordhood of an n-gram  $k$  by
$$
A V (k) = \min  \left(L _ {a v} (k), R _ {a v} (k)\right) \tag {2}
$$
where  $L_{av}(k)$  and  $R_{av}(k)$  denote the number of different character types that can precede (left access number) or follow (right access number) the n-gram  $k$ . Normally, the higher the AV score is, the more likely the n-gram forms a word.
2.2 Wordhood Memory Networks
To encode both n-grams and the wordhood information they carry, one requires an appropriate framework to do so for CWS. Compared with other network structures that can exploit n-grams such as the attention mechanism, key-value memory networks are more appropriate to model such pairwise knowledge via transforms between keys and values. In the memory, we map n-grams and their wordhood information to keys and values, respectively. Following Miller et al. (2016), we illustrate how our memory module generates and operates the (keys, values) pair for each  $x_{i}$  in this subsection.
N-gram Addressing For each \( x_{i} \) in a training/test instance, normally there are many n-grams in \( \mathcal{N} \) that contain \( x_{i} \). Therefore, the n-gram addressing step is to generate all n-grams from \( x_{i} \)'s context (including \( x_{i} \)) and keep only the ones that appear in \( \mathcal{N} \), resulting \( \mathcal{K}_{i} = [k_{i,1}, k_{i,2}, \dots, k_{i,j}, \dots, k_{i,m_{i}}] \) that \( x_{i} \) is a part of \( k_{i,j} \). For example, in the input sentence shown in Figure 1, the n-grams that contain the character \( x_{4} = \) "民" (people) form the list \( \mathcal{K}_{4} = ["民"(people),"居民"
Rule	vi,j
xi is the beginning of the key ki,j	VB
xi is inside the key ki,j	VI
xi is the ending of the key ki,j	VE
xi is the single-character key ki,j	VS
Table 1: The rules for assigning different values to  $x_{i}$  according to its position in a key  $k_{i,j}$ .
(resident), "民生" (livel hood), "居民生活" (res- idents' life)], which are highlighted in the dashed boxes illustrated at the bottom part of the figure. Then, the memory module activates the corresponding keys in it, addresses their embeddings (which are denoted as  $\mathbf{e}_{i,j}^{k}$  for each  $k_{i,j}$ ), and computes the probability distribution for them with
$$
p _ {i, j} = \frac {\exp \left(\mathbf {h} _ {i} \cdot \mathbf {e} _ {i , j} ^ {k}\right)}{\sum_ {j = 1} ^ {m _ {i}} \exp \left(\mathbf {h} _ {i} \cdot \mathbf {e} _ {i , j} ^ {k}\right)} \tag {3}
$$
for each key, where  $\mathbf{h}_i$  is the vector for  $x_{i}$  which can be generated from any text encoder.
Wordhood Reading Values in the memory represent the wordhood information for a given  $x_{i}$  and  $k_{i,j}$  pair, which is not a straightforward mapping because  $x_{i}$  may have different roles in each  $k_{i,j}$ . For example,  $k_{i,j}$  delivers different wordhood information when  $x_{i}$  appears at the beginning or the ending of  $k_{i,j}$ . Therefore, we set rules in Table 1 to read a value for a key according to different situations of  $x_{i}$  in  $k_{i,j}$ , where we use a set of values  $\{V_{B}, V_{I}, V_{E}, V_{S}\}$  with embeddings  $\{\mathbf{e}_{V_{B}}, \mathbf{e}_{V_{I}}, \mathbf{e}_{V_{E}}, \mathbf{e}_{V_{S}}\}$  (illustrated in different colors in Figure 1) so that all n-grams should map to one of the values based on  $x_{i}$ 's position in  $k_{i,j}$ . To illustrate that, in the aforementioned example, n-grams in  $\mathcal{K}_4$  for  $x_{4} =$  "民" (people) are mapped to a value list  $\mathcal{V}_4 = [V_S, V_E, V_B, V_I]$  (see Figure 1). As a result, each  $K_{i}$  for  $x_{i}$  has a list of values denoted by  $\mathcal{V}_i = [v_{i,1}, v_{i,2}, \dots, v_{i,j}, \dots, v_{i,m_i}]$ . Then the total wordhood memory for  $x_{i}$  is computed from the weighted sum of all keys and values by
$$
\mathbf {o} _ {i} = \sum_ {j = 1} ^ {m _ {i}} p _ {i, j} \mathbf {e} _ {i, j} ^ {v} \tag {4}
$$
where  $\mathbf{e}_{i,j}^{v}$  is the embedding for  $v_{i,j}$ . Afterwards,  $\mathbf{o}_i$  is summed element-wise with  $\mathbf{h}_i$  and the result is passed through a fully connected layer by
$$
\mathbf {a} _ {i} = \mathbf {W} _ {\mathbf {o}} \cdot \left(\mathbf {h} _ {\mathbf {i}} + \mathbf {o} _ {\mathbf {i}}\right) \tag {5}
$$
2Therefore n-gram and key are equivalent in the memory.
MSR	PKU	AS	CITYU	CTB6
TRAIN	TEST	TRAIN	TEST	TRAIN	TEST	TRAIN	TEST	TRAIN	DEV
CHAR #	4,050K	184K	1,826K	173K	8,368K	198K	2,403K	68K	1,056K	100K
WORD #	2,368K	107K	1,110K	104K	5,500K	123K	1,456K	41K	641K	60K
CHAR TYPE #	5K	3K	5K	3K	6K	4K	5K	3K	4K	3K
WORD TYPE #	88K	13K	55K	13K	141K	19K	69K	9K	42K	10K
OOV RATE	-	2.7	-	5.8	-	4.3	-	7.2	-	5.4
where  $\mathbf{W}_{\mathbf{o}}$  is a trainable parameter and the output  $\mathbf{a}_i \in \mathbb{R}^{|\mathcal{T}|}$  is a weight vector with its each dimension corresponding to a segmentation label.
2.3 Text Encoders and Decoders
To ensure wordhood memory networks functionalize, one requires to generate  $\mathbf{h}_{\mathbf{i}}$  for each  $x_{i}$  by
$$
[ \mathbf {h} _ {1}, \mathbf {h} _ {2}, \dots , \mathbf {h} _ {i}, \dots , \mathbf {h} _ {l} ] = E n c o d e r (\mathcal {X}) \tag {6}
$$
where the Encoder can be different models, e.g., Bi-LSTM and BERT (Devlin et al., 2019), to represent a sequence of Chinese characters into vectors.
Once all  $\mathbf{a_i}$  are generated from the memory for each  $x_{i}$ , a decoder takes them to predict a sequence of segmentation labels  $\widehat{\mathcal{Y}} = \widehat{y}_1\widehat{y}_2\dots \widehat{y}_l$  for  $\mathcal{X}$  by
$$
\widehat {\mathcal {Y}} = D e c o d e r (\mathcal {A}) \tag {7}
$$
where  $\mathcal{A} = \mathbf{a}_1\mathbf{a}_2\dots \mathbf{a}_i\dots \mathbf{a}_l$  is the sequence of output from Eq. 5. The Decoder can be implemented by different algorithms, such as softmax:
$$
\widehat {y} _ {i} = \arg \max  \frac {\exp \left(a _ {i} ^ {t}\right)}{\sum_ {t = 1} ^ {| \mathcal {T} |} \exp \left(a _ {i} ^ {t}\right)} \tag {8}
$$
where  $a_{i}^{t}$  is the value at dimension  $t$  in  $\mathbf{a_i}$ . Or one can use CRF for the Decoder:
$$
\widehat {y} _ {i} = \underset {y _ {i} \in \mathcal {T}} {\arg \max} \frac {e x p (\mathbf {W} _ {c} \cdot \mathbf {a} _ {i} + \mathbf {b} _ {c})}{\sum_ {y _ {i - 1} y _ {i}} e x p (\mathbf {W} _ {c} \cdot \mathbf {a} _ {i}) + \mathbf {b} _ {c}} (9)
$$
where  $\mathbf{W}_c\in \mathbb{R}^{|\mathcal{T}|\times |\mathcal{T}|}$  and  $\mathbf{b}_c\in \mathbb{R}^{|\mathcal{T}|}$  are trainable parameters to model the transition for  $y_{i - 1}$  to  $y_{i}$ .
We employ five benchmark datasets in our experiments: four of them, namely, MSR, PKU, AS, and CITYU, are from SIGHAN 2005 Bakeoff (Emerson, 2005) and the fifth one is CTB6 (Xue et al., 2005). AS and CITYU are in traditional Chinese characters whereas the other three use simplified
Table 2: Statistics of the five benchmark datasets, in terms of the number of character and word tokens and types in each training and test set. Out-of-vocabulary (OOV) rate is the percentage of unseen word tokens in the test set.
BC	BN	MZ	NW	WEB
CHAR #	275K	483K	403K	443K	342K
WORD #	184K	287K	258K	260K	210K
CHAR TYPE #	3K	3K	4K	3K	4K
WORD TYPE #	12K	23K	26K	21K	21K
OOV RATE	3.4	6.0	8.9	5.9	7.1
Table 3: Statistics of CTB7 with respect to five different genres. The OOV rate for each genre is computed based on the vocabulary from all the other four genres.
ones. Following previous studies (Chen et al., 2015, 2017; Qiu et al., 2019), we convert traditional Chinese characters in AS and CITYU into simplified ones. For MSR, AS, PKU, and CITYU, we follow their official training/test data split. For CTB6, we use the same split as that stated in Yang and Xue (2012); Chen et al. (2015); Higashiyama et al. (2019), and only use its test set for the final experiment. Table 2 show the statistics of all datasets in terms of the number of characters and words and the percentage of out-of-vocabulary (OOV) words in the dev/test sets with respect to the training set.
In addition, we also use CTB7 (LDC2010T07) to perform our cross-domain experiments. There are five genres in CTB7, including broadcast conversation (BC), broadcast news (BN), magazine (MZ), newswire (NW), and weblog (WEB). The statistics of all the genres are reported in Table 3, where the OOV rate for each genre is computed according to the union of all other genres. For example, the OOV rate for BC is computed with respect to the union of BN, MZ, NW, and WEB.
3.2 Wordhood Measures
We experiment with three wordhood measures to construct  $\mathcal{N}$ . The main experiment adopts the aforementioned AV as the measure to rank all n-grams, because AV was shown to be the most effective wordhood measure in previous CWS studies (Zhao and Kit, 2008). Since AV is sensitive to
3The conversion scripts are from https://github. com/skydark/nstools/tree/master/zhtools
MSR	PKU	AS	CITYU	CTB6
AV	49K	71K	105K	104K	50K
PMI	18K	16K	22K	21K	16K
DLG	32K	22K	32K	27K	16K
corpus size, in our experiments we use different AV thresholds when building the lexicon for each dataset: the threshold is 2 for PKU, CityU, CTB6 and CTB7, and 5 for MSR and AS.
To test the robustness of WMSEG, we also try two other wordhood measures, i.e., PMI (Sun et al., 1998) and DLG (Kit and Wilks, 1999). PMI measures pointwise mutual information between two Chinese characters,  $x'$  and  $x''$ , via
$$
P M I \left(x ^ {\prime}, x ^ {\prime \prime}\right) = \log \frac {p \left(x ^ {\prime} x ^ {\prime \prime}\right)}{p \left(x ^ {\prime}\right) p \left(x ^ {\prime \prime}\right)} \tag {10}
$$
where  $p$  computes the probability of an n-gram (i.e.,  $x', x''$  and  $x'x''$ ) in a dataset. A high PMI score indicates that the two characters co-occur a lot in the dataset and are likely to form a word. Hence, we use a threshold to determine whether a word boundary delimiter should be inserted between two adjacent characters in the dataset. In our experiments, we set the threshold to 0, PMI score lower than it will result in a segmentation. In other words, for each dataset, we use PMI to perform unsupervised segmentation and collect the segmented words from it to build the n-gram lexicon  $\mathcal{N}$ .
The other measure, DLG, computes wordhood of an n-gram  $s$  according to the change of the description length of a dataset  $D$  with and without treating that n-gram as a segment:
$$
D L G (s) = D L (\mathcal {D}) - D L (\mathcal {D} [ r \rightarrow s ] \oplus s) \tag {11}
$$
where  $\mathcal{D}$  denotes the original dataset and  $\mathcal{D}[r\to s]\oplus s$  represents a new dataset by treating  $s$  as a new segment, replacing all the occurrences of  $s$  with a new symbol  $r$  (which can be seen as an index for newly identified segment  $s$ ), and then appending  $s$  at the end.  $DL(\mathcal{D})$  is the Shannon-Fano code length of a dataset  $\mathcal{D}$ , calculated by
$$
D L (\mathcal {D}) = - \sum_ {x \in \mathcal {V}} c (x) \log \frac {c (x)}{| \mathcal {D} |} \tag {12}
$$
where  $\mathcal{V}$  refers to the vocabulary of  $\mathcal{D}$  and  $c(x)$  the count of segment  $x$ . We set the threshold for DLG to 0 and use the n-grams whose DLG is higher than it to build lexicon  $\mathcal{N}$  for each dataset.
Table 4: The size of lexicon  $\mathcal{N}$  generated from different wordhood measures under our settings.
Bi-LSTM	BERT / ZEN
Word Embedding Size	200	-
Hidden State Size	100	768
Hidden State Layers	1	12
Key Embedding Size	200	768
Value Embedding Size	200	768
Dropout Rate	0.2	0.1
Table 5: The hyper-parameters for our models w.r.t. different encoders, i.e., Bi-LSTM, BERT and ZEN.
All aforementioned measures are conducted on the union of the training and test sets, so that ngrams and their wordhood information are shared in both the learning and prediction phase. We remove all white spaces from the data and use the resulted raw texts to perform these measures. Table 4 shows the sizes of the lexicons created with these wordhood measures on the five datasets.
Following previous studies (Sun and Xu, 2011; Chen et al., 2015, 2017; Ma et al., 2018; Qiu et al., 2019), we use four segmentation labels in our experiments, i.e.,  $\mathcal{T} = \{B, I, E, S\}$ . Among them,  $B, I$ , and  $E$  indicate a character is the beginning, inside, and the ending of a word and  $S$  denotes that the character is a single-character word.
Since text representation plays an important role to facilitate many tasks (Conneau et al., 2017; Song et al., 2017, 2018; Sileo et al., 2019), we try two effective and well-known encoders, i.e., Bi-LSTM and BERT. In addition, we test WMSEG on a pretrained encoder for Chinese language, i.e., ZEN (Diao et al., 2019), which learns n-gram information in its pre-training from large raw corpora and outperforms BERT on many Chinese NLP tasks. Table 5 shows the hyperparameter settings for all the encoders: for the Bi-LSTM encoder, we follow the setting of Chen et al. (2015) and adopt their character embeddings for  $\mathbf{e}_i^x$ , and for BERT and ZEN encoders, we follow the default settings in their papers (Devlin et al., 2019; Dao et al., 2019).
For the decoders, we use softmax and CRF, and set their loss functions as cross-entropy and negative log-likelihood, respectively. The memory module can be initialized by random or pre-trained word embeddings for keys and values. In our experiments, we use random initialization for them.
4We use the Chinese base model from https://s3.amazon.com/models.huggingface.co/.
<sup>5</sup>https://github.com/sinovation/ZEN.
<sup>6</sup>We tried different initialization methods, and they did not show a significant difference in CWS performance.
CONFIG EN-DN	WM	MSR	PKU	AS	CITYU	CTB6
F	\( R_{OOV} \)	F	\( R_{OOV} \)	F	\( R_{OOV} \)	F	\( R_{OOV} \)	F	\( R_{OOV} \)
BL-SM	×√	95.5395.61	62.9663.94	91.8591.97	48.8449.00	94.5294.70	62.2164.18	93.7993.88	67.2669.20	93.5693.70	67.3968.52
BL-CRF	×√	95.8095.98	66.1768.75	92.3592.43	52.0456.80	94.3995.07	61.5968.17	93.9694.20	67.8469.91	93.8494.03	70.8171.88
BT-SM	×√	97.8498.16	86.3286.50	96.2096.47	84.4386.34	96.3396.52	77.8678.67	97.5197.77	86.6986.62	96.9097.13	88.4688.30
BT-CRF	×√	97.9898.28	85.5286.67	96.3296.51	85.0486.76	96.3496.58	77.7578.48	97.6397.80	86.6687.57	96.9897.16	87.4388.00
ZEN-SM	×√	98.3598.36	85.7885.30	96.2796.49	84.5084.95	96.3896.55	77.6278.02	97.7897.86	90.6990.89	97.0897.22	86.2086.83
ZEN-CRF	×√	98.3698.40	86.8284.87	96.3696.53	84.8185.36	96.3996.62	77.8179.64	97.8197.93	91.7890.15	97.1397.25	87.0888.46
Table 6: Experimental results of WMSEG on SIGHAN2005 and CTB6 datasets with different configurations. "EN-DN" stands for the text encoders ("BL" for Bi-LSTM and "BT" for BERT) and decoders ("SM" for softmax and "CRF" for CRF). The "WM" column indicates whether the wordhood memories are used ( $\sqrt{}$ ) or not ( $\times$ ).
4 Results and Analyses
In this section, we firstly report the results of WMSEG with different configurations on five benchmark datasets and its comparison with existing models. Then we explore the effect of using different lexicon  $\mathcal{N}$  and different wordhood measures in WMSEG. We also use a cross-domain experiment to illustrate the effectiveness of WMSEG when more OOVs are in the test set. Lastly, a case study is performed to visualize how the wordhood information used in WMSEG helps CWS.
4.1 Results on Benchmark Datasets
In the main experiment, we illustrate the validity of the proposed memory module by comparing WM-SEG in different configurations, i.e., with and without the memory in integrating with three encoders, i.e., Bi-LSTM, BERT, and ZEN, and two decoders, i.e., softmax and CRF. The experimental results on the aforementioned five benchmark datasets are shown in Table 6, where the overall F-score and the recall of OOV are reported. With five datasets and six encoder-decoder configurations, the table includes results from 30 pairs of experiments, each pair with or without using the memories.