Supervised Chinese word segmentation has been widely approached as sequence labeling or sequence modeling . Recently, some researchers attempted to treat it as character-level translation  , but there is still a performance gap between the translation-based approach and other methods. In this work, we apply the best practices from low-resource neural machine translation to Chinese word segmentation. We build encoder-decoder models with attention, and examine a series of techniques including regularization, data augmentation, objective weighting, transfer learning and ensembling. When benchmarked on MSR corpus under closed test condition without additional data, our method achieves 97.6\% F1, which is on a par with the state of the art  .
Supervised Chinese word segmentation has entered the deep learning era which reduces the hassle of feature engineering . Recently, some researchers attempted to treat it as character-level translation which further simplified model designing and building , but there is still a performance gap between the translation-based approach and other methods. In this work, we apply the best practices from low-resource neural machine translation to Chinese word segmentation. We build encoder-decoder models with attention, and examine a series of techniques including regularization, data augmentation, objective weighting, transfer learning and ensembling. Our method is generic for word segmentation, without the need for feature engineering or model implementation. In the closed test with constrained data, our method ties with the state of the art on the MSR dataset and is comparable to other methods on the PKU dataset .
R
been widely approached as sequence labeling or sequence modeling
entered the deep learning era which reduces the hassle of feature engineering
A
which further simplified model designing and building
R
When benchmarked on MSR corpus under closed test condition without additional
Our method is generic for word segmentation, without the need for feature engineering or model implementation. In the closed test with constrained
R
achieves 97.6\% F1, which is on a par
ties
A
on the MSR dataset and is comparable to other methods on the PKU dataset