Venue: EMNLP 2017

# Word-Context Character Embeddings for Chinese Word Segmentation

 Hao Zhou1 Nanjing University & Toutiao AI Lab zhouhao.nlp@bytedance.com Zhenting Yu1 Nanjing University yuzt@nlp.nju.edu.cn Yue Zhang Singapore University of Technology and Design yue_zhang@sutd.edu.sg Shujian Huang Nanjing University huangjs@nlp.nju.edu.cn Xinyu Dai Nanjing University daixinyu@nju.edu.cn Jiajun Chen Nanjing University chenjj@nlp.nju.edu.cn

Footnote 1: Equal contributions

###### Abstract

Neural parsers have benefited from automatically labeled data via dependency-context word embeddings. We investigate training character embeddings on a word-based context in a similar way, showing that the simple method significantly improves state-of-the-art neural word segmentation models, beating tri-training baselines for leveraging auto-segmented data.

## 1 Introduction

Neural network Chinese word segmentation (CWS) models [20, 35, 4] appeal for their strong ability of feature representation, employing unigram and bigram character embeddings as input features [26, 8, 9]. They give state-of-the-art performances. We investigate leveraging automatically segmented texts for enhancing their accuracies.

Such semi-supervised methods can be divided into two main categories. The first one is _bootstrapping_, which includes _self-training_ and _tri-training_. The idea is to generate more training instances by automatically labeling large-scale data. Self-training [7, 12, 13, 14] labels additional data by using the base classifier itself, and tri-training [25, 13] uses two extra classifiers, taking the instances with the same labels for additional training data. A second semi-supervised learning method in NLP is _knowledge distillation_, which extracts knowledge from large-scale auto-labeled data as features.

Tri-training has been used in neural parsing, giving considerable improvements for both of dependency [21] and constituent parsing [20, 15]. Knowledge from auto-labeled data has also been used for parsing [2, 13], where word embeddings are trained on automatic dependency tree context. Such knowledge has also been proved effective in conventional discrete CWS models, such as _label distribution information_[23, 20]. However, it has not been investigated for neural CWS.

We propose _word-context character embeddings_ (WCC), using segmentation label information in the pre-training of unigram and bigram character embeddings. The method packs the label distribution information into the embeddings, which could be regarded as a way for knowledge parameterization. Our idea follows levy2014deep, who use _dependency contexts_ to train _word embeddings_. Additionally, motivated by co-training, we propose _multi-view word-context character embeddings_ for cross-domain segmentation, which pre-trains two types of embedding for in-domain and out-of-domain data, respectively. In-domain embeddings are used for solving data sparseness, and out-of-domain embeddings are used for domain adaptation.

Our proposed model is simple, efficient and effective, giving average 1% accuracy improvement on in-domain data and 3.5% on out-of-domain data, respectively, significantly out-performing self-training and tri-training methods for leveraging auto-segmented data.

## 2 Baseline Segmentation Model

Chinese word segmentation can be regarded as a character sequence labeling task, where each char

acter in the sentence is assigned a _segment label_ from left to right, including {_B, M, E, S_}, to indicate the segmentation Xue et al. (2003); Low et al. (2005); Zhao et al. (2006). _B, M, E_ represent the character is the beginning, middle or end of a multi-character word, respectively. \(S\) represents that the current character is a single character word.

Following Chen et al. (2015b), a standard bi-LSTM model Graves (2008) is used to assign segmentation label for each character. As shown in Figure 1, our model consists of a _representation layer_ and a _scoring layer_. The representation layer utilizes a bi-LSTM to capture the context of each character in the sentence. Given a sentence {\(w_{1}\), \(w_{2}\), \(w_{3}\), \(\cdots\), \(w_{N}\) }, where \(w_{i}\) is the \(i_{th}\) character in the sentence, and \(N\) is the sentence length, we have a corresponding embedding \(e_{w_{i}}\) and \(e_{w_{i-1}w_{i}}\) for each character unigram \(w_{i}\) and character bigram \(w_{i-1}w_{i}\), respectively. A forward word representation \(e^{f}_{i}\) is calculated as follows:

\[e^{f}_{i}=\textit{concat}_{1}(e_{w_{i}},e_{w_{i-1}w_{i}}),\] \[=\text{tanh}(W_{1}[e_{w_{i}};e_{w_{i-1}w_{i}}])\]

A backward representation \(e^{b}_{i}\) can be obtained in the same way. Then \(e^{f}_{i}\) and \(e^{b}_{i}\) are fed into forward and backward LSTM units at current position, obtaining the corresponding forward and backward LSTM representations \(r^{lstm-f}_{i}\) and \(r^{lstm-b}_{i}\), respectively.

In the _scoring layer_, we first obtain a linear combination of \(r^{lstm-f}_{i}\) and \(r^{lstm-b}_{i}\), which is the final representation at the \(i_{th}\) position.

\[r_{i}=\textit{concat}_{2}(r^{lstm-f}_{i},r^{lstm-b}_{i})\] \[=\text{tanh}(W_{2}[r^{lstm-f}_{i};r^{lstm-b}_{i}])\]

Given the representation \(r_{i}\), we use a scoring unit to score for each potential segment label. Given \(r_{i}\), the score of segment label \(M\) is:

\[f^{i}_{M}=W_{M}h,\]

where

\[h =\textit{concat}_{3}(r_{i},e_{M}),\] \[=\text{tanh}(W_{3}[r_{i};e_{M}])\]

\(W_{M}\) is the score matrix for label \(M\), and \(e_{M}\) is the label embedding for label \(M\).

## 3 Word-Context Character Embeddings

Our model structure is a derivation from the skipgram model Mikolov et al. (2013), similar to Levy and Goldberg (2014). Given a sentence with length \(n\): {\(w_{1}\), \(w_{2}\), \(w_{3}\), \(\cdots\)\(w_{n}\)} and its corresponding segment labels: {\(l_{1}\), \(l_{2}\), \(l_{3}\), \(\cdots\)\(l_{n}\)}, the pre-training context of current character \(w_{t}\) is the around characters in the windows with size \(c\), together with their corresponding segment labels (Figure 2). Characters \(w_{i}\) and labels \(l_{i}\) in the context are represented by vectors \(e^{c}_{w_{i}}\in\mathbb{R}^{d}\) and \(e^{c}_{l_{i}}\in\mathbb{R}^{d}\), respectively, where \(d\) is the embedding dimensionality.

The word-context embedding of character \(w_{t}\) is represented as \(e_{w_{t}}\in\mathbb{R}^{d}\), which is trained by predicting the surrounding context representations

Figure 1: Baseline model architecture.

 

and \(e_{l_{i}}^{c}\), parameterizing the labeled segmentation information in the embedding parameters. To capture order information Ling et al. (2015), we use different embedding matrices for context embedding in different context positions, training different embeddings for the same word when they reside on different locations as the context word. In particular, our context window size is five. As a result, each word has four different versions of \(e^{c}\), namely \(e_{-1}^{c}\), \(e_{-2}^{c}\), \(e_{+1}^{c}\), and \(e_{+2}^{c}\), each taking a distinct embedding matrix. Given the context window [\(w_{-2}\), \(w_{-1}\), \(w\), \(w_{+1}\), \(w_{+2}\)], \(w_{-1}\) is the left first context word of the focus word \(w\), \(e_{-1,w_{i}}^{c}\) will be selected from embedding matrix \(E_{-1}\), and \(w_{+1}\) is the right first word of w, \(e_{+1,w_{i}}^{c}\) will be selected from embedding matrix \(E_{+1}\).

Note that each character has two types of embeddings, where \(e_{w_{i}}\) is the embedding form of \(w_{i}\) when \(w_{i}\) is the focus word, and \(e_{w_{i}}^{c}\) is the embedding form of \(w_{i}\) when \(w_{i}\) is used as a surrounding context word. We do not have \(e_{l_{i}}\) because \(l_{i}\) only acts as the surrounding context. After pre-training, \(e_{w_{i}}\) will be used as the WCC embeddings.

The objective of our model is to maximize the average log probability of the context:

\[\frac{1}{T}\sum_{t=1}^{T}\sum_{-c\leq j\leq c,j\neq 0}\text{log}\,p(w_{t+j}|w_ {t})+\text{log}\,p(t_{t+j}|w_{t})\]

Negative sampling Mikolov et al. (2013) is used, where \(\text{log}\,p(w_{t+j}|w_{t})\) and \(\text{log}\,p(t_{t+j}|w_{t})\) are computed as:

\[p(w_{t+j}|w_{t})= \text{log}\,\sigma(e_{w_{t+j}}^{c}{}^{\top}e_{w_{t}})\] \[+\sum_{i=1}^{k}\mathbb{E}_{w_{i}\sim P_{n}(w)}[\text{log}\,\sigma (-e_{w_{i}}^{c}{}^{\top}e_{w_{t}})]\]

and

\[p(t_{t+j}|w_{t})= \text{log}\,\sigma(e_{l_{t+j}}^{c}{}^{\top}e_{w_{t}})\] \[+\sum_{i=1}^{k}\mathbb{E}_{l_{i}\sim P_{n}(l)}[\text{log}\,\sigma (-e_{l_{i}}^{c}{}^{\top}e_{w_{t}})],\]

respectively, where \(P_{n}(w)\) and \(P_{n}(l)\) is the noise distributions and \(k\) is the size of negative samples for each data sample.

Bigram embeddings are trained in the same way as unigram character embeddings. For out-of-domain segmentation, we pre-train two embeddings for each token, extracting knowledge from the two domains, respectively.

## 4 Experiments

### Set-up

We perform experiments on three standard datasets for Chinese word segmentation: PKU and MSR from the second SIGHAN bakeoff shared task, and Chinese Treebank 6.0 (CTB6). For PKU and MSR, 10% of the training data are randomly selected as development data. We follow Zhang et al. (2016) to split the CTB6 corpus into training, development and testing sections. For evaluating cross-domain performance, we also experiment on Chinese novel data. Following Zhang et al. (2014), the training set of CTB5 is selected for training, and the manually annotated sentences of free Internet novel 'Zhuxian' (ZX) are selected as the development and test data Liu and Zhang (2012)1.

Footnote 1: [http://zhangmeishan.github.io/](http://zhangmeishan.github.io/) eacl14mszhang.zip

Chinese Gigaword (LDC2011T13, 4M) is used for in-domain unlabeled data. For out-of-domain data, 20K raw sentences of Zhukian is used. We take self-training and tri-training as baselines, which also use large-scale auto-segmented data. For self-training, skip-gram pre-training and word-context character embedding, unlabeled corpus is segmented automatically by our baseline model. For tri-training, we additionally use the ZParZhang and Clark (2007) and ICTCLAS2 as our base classifiers .

Footnote 2: [http://ictclas.nlpir.org/](http://ictclas.nlpir.org/)

We use F1 to evaluate segmentation accuracy. The recalls of in-vocabulary (IV) and out-of-vocabulary (OOV) are also measured.

### Hyper-Parameters

The hyper-parameters used in this work are listed in Table 1. The values are selected according

Figure 2: Word-context for the character ’\(\bot\)’ in two different sentences. The windows size \(c=3\).

 

to the development set of CTB6. Many previous character-based CWS models use a transition matrix to model the tag dependency and CRF for structured inference Pei et al. (2014); Chen et al. (2015a). However, we find that, the greedy model obtains comparable segmentation accuracies across CTB6, PKU and MSR, yet giving much fast speed (Table 2). Hence we adopt the greedy model as our baseline segmentation model.

### Utilizing Varying-Scale Data

The results of self-training and tri-training with varying-scale training data are list in Table 3, where \(+4\)X means adding 4 times the size of supervised training data into the training set. We find that self-training does not work well, and tri-training with 16X gives a 0.5% accuracy improvement. We adopt this setting for our baseline in the remaining experiments3.

Footnote 3: For out-of-domain experiments, we include both the \(+16\)X and the 20K out-of-domain data for self-training and tri-training.

We also try to choose more effective examples for self-training and tri-training, by selecting training instances according to the base segmentation model score. However, the segmentation performances do not get improved. A possible reason is that the training instances with higher confidence are always shorter than the original sampled sentences, which may not be very helpful for semi-supervised segmentation.

### In-Domain Results

As shown in Table 4, pre-training with conventional skip-gram embeddings gives only small improvements, which is consistent as findings of previous work Chen et al. (2015a); Ma and Hinrichs (2015); Cai and Zhao (2016). Segmentation with self-training even shows accuracy drops on PKU and MSR. We speculate that the self-training by the neural CWS baseline is sensitive to the segmentation errors of the auto-labeled data. On average, our method obtains an absolute 1% accuracy improvement over the baseline, outperforming other semi-supervised method significantly4.

Footnote 4: The p-values are below 0.01 using pairwise t-test.

We compare our model with other state-of-the-art segmentation models5, which are grouped into 3 classes, namely traditional segmentation models (_non-nn_), neural segmentation models (_nn_), and the combination of both neural and traditional discrete features (_comb_). Our simple model gives top accuracies compared with related work. Liu et al. (2016), Cai and Zhao (2016) and Zhang et al. (2016) propose to incorporate word embedding features in the neural CWS, pre-training the word embeddings in the large-scale labeled data. Different to them, we employ a simpler character level model containing word information, yet obtaining higher F1 scores.

\begin{table}
\begin{tabular}{l|c} \hline unigram dimension & 50 \\ bigram dimension & 50 \\ label embedding dimention & 32 \\ LSTM hidden size & 100 \\ LSTM input size & 100 \\ learning rate & 0.1 \\ windows size & 5 \\ \hline \end{tabular}
\end{table}
Table 1: Hyper-parameters.

\begin{table}
\begin{tabular}{c|l c|c|c} \hline Type & System & CTB6 & PKU & MSR \\ \hline \multirow{7}{*}{_non-nn_} & Tseng et al. (2005) & - & 95.0 & 96.4 \\  & Sun et al. (2009) & - & 95.2 & 97.3 \\  & Wang et al. (2011) & 95.8 & - & - \\  & Zhang et al. (2013) & - & **96.1** & 97.5 \\ \hline \multirow{7}{*}{_mn_} & Zheng et al. (2013) & - & 92.4 & 93.3 \\  & Pei et al. (2014) & - & 95.2 & 97.2 \\  & Kong et al. (2015) & - & 90.6 & 90.7 \\  & Ma and Hinrichs (2015) & - & 95.1 & 96.6 \\  & Chen et al. (2015c)† & - & 94.8 & 95.6 \\  & Xu and Sun (2016) & 95.8 & **96.1** & 96.3 \\  & Liu et al. (2016) & 95.5 & 95.7 & 97.6 \\  & Zhang et al. (2016) & 95.4 & 95.1 & 97.0 \\  & Cai and Zhao (2016) & - & 95.5 & 96.5 \\ \hline _comb_ & Zhang et al. (2016) & 96.0 & 95.7 & 97.7 \\ \cline{2-4}  & _baseline_ & 94.9 & 95.0 & 97.2 \\ \cline{2-4}  & _+self-training_ & 95.0 & 94.8 & 97.0 \\ \cline{2-4}  & _+tri-training_ & 95.5 & 95.5 & 97.4 \\  & _+skip-gram embeddings_ & 95.3 & 95.5 & 97.4 \\  & _+ WCC embeddings_ & **96.2** & **96.0** & **97.8** \\ \hline \end{tabular}
\end{table}
Table 4: Comparison with other models.

\begin{table}
\begin{tabular}{l|c c c|c} \hline System & CTB6 & PKU & MSR & Speed \\ \hline Greedy & 94.9 & 95.0 & 97.2 & 14.7 \\ CRF & 95.0 & 95.1 & 97.2 & 3.6 \\ \hline \end{tabular}
\end{table}
Table 2: Comparisons between greedy and CRF segmentation. Speed: tokens per millisecond.

\begin{table}
\begin{tabular}{c|l l|c} \hline Type & System & CTB6 & PKU & MSR \\ \hline \multirow{7}{*}{_non-nn_} & Tseng et al. (2005) & - & 95.0 & 96.4 \\  & Sun et al. (2009) & - & 95.2 & 97.3 \\  & Wang et al. (2011) & 95.8 & - & - \\  & Zhang et al. (2013) & - & **96.1** & 97.5 \\ \hline \multirow{7}{*}{_mn_} & Zheng et al. (2013) & - & 92.4 & 93.3 \\  & Pei et al. (2014) & - & 95.2 & 97.2 \\  & Kong et al. (2015) & - & 90.6 & 90.7 \\  & Ma and Hinrichs (2015) & - & 95.1 & 96.6 \\  & Chen et al. (2015c)† & - & 94.8 & 95.6 \\  & Xu and Sun (2016) & 95.8 & **96.1** & 96.3 \\  & Liu et al. (2016) & 95.5 & 95.7 & 97.6 \\  & Zhang et al. (2016) & 95.4 & 95.1 & 97.0 \\  & Cai and Zhao (2016) & - & 95.5 & 96.5 \\ \hline _comb_ & Zhang et al. (2016) & 96.0 & 95.7 & 97.7 \\ \hline \multirow{7}{*}{_Ours_} & _baseline_ & 94.9 & 95.0 & 97.2 \\  & _+self-training_ & 95.0 & 94.8 & 97.0 \\ \cline{1-1}  & _+tri-training_ & 95.5 & 95.5 & 97.4 \\ \cline{1-1}  & _+skip-gram embeddings_ & 95.3 & 95.5 & 97.4 \\ \cline{1-1}  & _+WCC embeddings_ & **96.2** & **96.0** & **97.8** \\ \hline \end{tabular}
\end{table}
Table 3: Results of self-training and tri-training on CTB6 with varying scaled training data.

 



### Out-of-Domain Results

We test out-of-domain performance of our model on the ZX dataset. We also use the multi-view word-context character embeddings (WCC) for cross domain segmentation, which uses two types of embeddings by simple vector concatenation. One type of embeddings is pre-trained on in-domain data, and the other type is pre-trained on out-of-domain data. In such case, the multi-view embeddings includes cross-domain information, which may enhance the cross-domain segmentation performance Mou et al. (2016).

As shown in Table 5, using word-context character (\(WCC\)) embeddings and multi-view word-context character embeddings both give significantly higher accuracy improvements compared with other semi-supervised methods. Additionally, we find that multi-view WCC embeddings give an extra 1% F1 score improvement over WCC embeddings. Our proposed model also significantly improves the OOV recall (ROOV) and IV recall (RIV). By studying the cases of segmented output (Figure 3), we find that our model can recognize OOV words such as '+ '+ '+ '+ '+ '+ '+' and the IV word '+ '+', which are incorrectly labeled by the baseline. This confirms that our proposed model is helpful for the data sparseness problem on closed domain and domain adaptation on across domain.

We also list the results of Zhang et al. (2014) and Liu et al. (2014) on this dataset. Liu et al. (2014) obtains better out-of-domain performance than our model. However, their results cannot be compared directly with ours because they use partial labeled URL link data from Chinese Wikipedia data for training.

## 5 Conclusion

We proposed word-context character embeddings for semi-supervised neural CWS, which makes the segmentation model more accurate on in-domain data, and more robust on the out-of-domain data. Our segmentation model is simple yet effective, achieving state-of-the-art segmentation accuracies on standard benchmarks. It can also be useful for other NLP tasks with small labeled training data, but a large unlabeled data. Our code could be downloaded at [https://github.com/zhouh/WCC-Segmentation](https://github.com/zhouh/WCC-Segmentation).

## Acknowledge

We would like to thank the anonymous reviewers for their insightful comments. We also thank Ji Ma and Meishan Zhang for their helpful discussions. This work was partially founded by the Natural Science Foundation of China (61672277, 71503124) and the China National 973 project 2014CB340301.

## References

* Bansal et al. (2014) Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2014. Tailoring continuous word representations for dependency parsing. In _ACL (2)_, pages 809-815.
* Cai and Zhao (2016) Deng Cai and Hai Zhao. 2016. Neural word segmentation learning for chinese. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 409-420. Association for Computational Linguistics.
* Chen et al. (2013) Wenliang Chen, Min Zhang, and Yue Zhang. 2013. Semi-supervised feature transformation for dependency parsing. In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_, pages 1303-1313. Association for Computational Linguistics.



* Chen et al. (2014) Wenliang Chen, Yue Zhang, and Min Zhang. 2014. Feature embedding for dependency parsing. In _COLING_, pages 816-826.
* Chen et al. (2015a) Xinchi Chen, Xipeng Qiu, Chenxi Zhu, and Xuanjing Huang. 2015a. Gated recursive neural network for chinese word segmentation. In _Proceedings of Annual Meeting of the Association for Computational Linguistics_.
* Chen et al. (2015b) Xinchi Chen, Xipeng Qiu, Chenxi Zhu, Pengfei Liu, and Xuanjing Huang. 2015b. Long short-term memory neural networks for chinese word segmentation. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_.
* Chen et al. (2015c) Xinchi Chen, Xipeng Qiu, Chenxi Zhu, Pengfei Liu, and Xuanjing Huang. 2015c. Long short-term memory neural networks for chinese word segmentation. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_, pages 1197-1206. Association for Computational Linguistics.
* Choe and Charniak (2016) Kook Do Choe and Eugene Charniak. 2016. Parsing as language modeling. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 2331-2336. Association for Computational Linguistics.
* Graves (2008) Alex Graves. 2008. _Supervised Sequence Labelling with Recurrent Neural Networks_. Ph.D. thesis, Technical University Munich.
* Huang et al. (2010) Zhongqiang Huang, Mary Harper, and Slav Petrov. 2010. Self-training with products of latent variable grammars. In _Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing_, pages 12-22. Association for Computational Linguistics.
* Kong et al. (2015) Lingpeng Kong, Chris Dyer, and Noah A Smith. 2015. Segmental recurrent neural networks. _ICLR_.
* Levy and Goldberg (2014) Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In _ACL (2)_, pages 302-308. Citeseer.
* Li et al. (2014) Zhenghua Li, Min Zhang, and Wenliang Chen. 2014. Ambiguity-aware ensemble training for semi-supervised dependency parsing. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 457-467. Association for Computational Linguistics.
* Ling et al. (2015) Wang Ling, Chris Dyer, Alan Black, and Isabel Trancoso. 2015. Two/too simple adaptations of word2vec for syntax problems. In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics.
* Liu and Zhang (2012) Yang Liu and Yue Zhang. 2012. Unsupervised domain adaptation for joint segmentation and pos-tagging. In _COLING (Posters)_, pages 745-754.
* Liu et al. (2016) Yijia Liu, Wanxiang Che, Jiang Guo, Bing Qin, and Ting Liu. 2016. Exploring segment representations for neural segmentation models. In _Proceedings of the 25th International Joint Conference on Artificial Intelligence_.
* Liu et al. (2014) Yijia Liu, Yue Zhang, Wanxiang Che, Ting Liu, and Fan Wu. 2014. Domain adaptation for crf-based chi-nese word segmentation using free annotations. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 864-874. Association for Computational Linguistics.
* Low et al. (2005) Jin Kiat Low, Hwee Tou Ng, and Wenyuan Guo. 2005. A maximum entropy approach to chinese word segmentation. In _Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing_, volume 1612164, pages 448-455.
* Ma and Hinrichs (2015) Jianqiang Ma and Erhard Hinrichs. 2015. Accurate linear-time chinese word segmentation via embedding matching. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1733-1743. Association for Computational Linguistics.
* McClosky et al. (2006) David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In _Proceedings of the main conference on human language technology conference of the North American Chapter of the Association of Computational Linguistics_, pages 152-159. Association for Computational Linguistics.
* Melamud et al. (2016) Oren Melamud, David McClosky, Siddharth Patwardhan, and Mohit Bansal. 2016. The role of context types and dimensionality in learning word embeddings. In _Proceedings of NACCL-HLT_, pages 1030-1040.
* Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. _arXiv preprint arXiv:1301.3781_.
* Mou et al. (2016) Lili Mou, Zhao Meng, Rui Yan, Ge Li, Yan Xu, Lu Zhang, and Zhi Jin. 2016. How transferable are neural networks in nlp applications? In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 479-489, Austin, Texas. Association for Computational Linguistics.
* Pei et al. (2014) Wenzhe Pei, Tao Ge, and Baobao Chang. 2014. Maxmargin tensor neural network for chinese word segmentation. In _ACL (1)_, pages 293-303.
* Sun et al. (2009) Xu Sun, Yaozhong Zhang, Takuya Matsuzaki, Yoshimasa Tsuruoka, and Jun'ichi Tsujii. 2009. A discriminative latent variable chinese segmenter with hybrid word/character information. In _Proceedings ofHuman Language Technologies: The 2009 Annual 

Conference of the North American Chapter of the Association for Computational Linguistics_, pages 56-64. Association for Computational Linguistics.
* Tseng et al. (2005) Huihin Nseng, Pichuan Chang, Galen Andrew, Daniel Jurafsky, and Christopher Manning. 2005. A conditional random field word segmenter for sighan bakeoff 2005. In _Proceedings of the fourth SIGHAN workshop on Chinese language Processing_, volume 171.
* Vinyals et al. (2015) Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey E. Hinton. 2015. Grammar as a foreign language. In _NIPS_.
* Wang et al. (2011) Yiou Wang, Yoshimasa Tsuruoka Jun'ichi Kazama, Yoshimasa Tsuruoka, Wenliang Chen, Yujie Zhang, and Kentaro Torisawa. 2011. Improving chinese word segmentation and pos tagging with semi-supervised methods using large auto-analyzed data. In _IJCNLP_, pages 309-317.
* Weiss et al. (2015) David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured training for neural network transition-based parsing. _arXiv preprint arXiv:1506.06158_.
* Xu and Sun (2016) Jingjing Xu and Xu Sun. 2016. Dependency-based gated recursive neural network for chinese word segmentation. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 567-572. Association for Computational Linguistics.
* Xue (2003) Nianwen Xue. 2003. Chinese word segmentation as character tagging. _Computational Linguistics and Chinese Language Processing_, 8(1):29-48.
* Yarowsky (1995) David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In _Proceedings of the 33rd annual meeting on Association for Computational Linguistics_, pages 189-196. Association for Computational Linguistics.
* Zhang et al. (2013) Longkai Zhang, Houfeng Wang, Xu Sun, and Mairgup Mansur. 2013. Exploring representations from unlabeled data with co-training for chinese word segmentation. In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_, pages 311-321. Association for Computational Linguistics.
* Zhang et al. (2014) Meishan Zhang, Yue Zhang, Wanxiang Che, and Ting Liu. 2014. Type-supervised domain adaptation for joint segmentation and pos-tagging. In _EACL_, pages 588-597.
* Zhang et al. (2016) Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Transition-based neural word segmentation. In _Proceedings of the 54nd Annual Meeting of the Association for Computational Linguistics_.
* Zhang and Clark (2007) Yue Zhang and Stephen Clark. 2007. Chinese segmentation with a word-based perceptron algorithm. In _Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics_, pages 840-847, Prague, Czech Republic. Association for Computational Linguistics.
* Zhao et al. (2006) Hai Zhao, Chang-Ning Huang, and Mu Li. 2006. An improved chinese word segmentation system with conditional random field. In _Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing_, volume 1082117. Sydney: July.
* Zheng et al. (2013) Xiaoqing Zheng, Hanyang Chen, and Tianyu Xu. 2013. Deep learning for chinese word segmentation and pos tagging. In _EMNLP_, pages 647-657.
* Zhou and Li (2005) Zhi-Hua Zhou and Ming Li. 2005. Tri-training: Exploiting unlabeled data using three classifiers. _IEEE Transactions on knowledge and Data Engineering_, 17(11):1529-1541.
* Zhu et al. (2013) Muhua Zhu, Yue Zhang, Wenliang Chen, Min Zhang, and Jingbo Zhu. 2013. Fast and accurate shift-reduce constituent parsing. In _51st Annual Meeting of the Association for Computational Linguistics_.