Domain Adaptation of T hai Word Segmentation Models using Stacked Ensemble (2020)

Abstract: Like many Natural Language Processing tasks, Thai word segmentation is domain-dependent. Researchers have been relying on transfer learning to adapt an existing model to a new domain. However, this approach is inapplicable to cases where we can interact with only input and output layers of the models, also known as “black boxes”. We propose a filter-and-refine solution based on the stacked-ensemble learning paradigm to address this black-box limitation. We conducted extensive experimental studies comparing our method against state-of-the-art models and transfer learning. Experimental results show that our proposed solution is an effective domain adaptation method and has a similar performance as the transfer learning method.

## introduction word segmentation (ws) is an essential process for several natural language processing (nlp) tasks such as part-of-speech (pos) tagging and machine translation (mt). the accuracy of ws significantly affects the accuracy of these nlp tasks, as shown in experimental results from @xcite while ws is considered relatively simple in english, it is still an open problem in languages without explicitly defined word delimiters, such as thai, chinese, and japanese. however, unlike chinese and japanese, thai ws did not receive much research attention. there are only six notable publications @xcite @xcite @xcite on thai ws for the past ten years. on the other hand, there are at least eight papers from well-established conferences on chinese and japanese ws @xcite @xcite @xcite @xcite within only the last two years. this investigation focuses on the segmentation of thai words since it is a challenging problem that has an excellent opportunity to improve, especially in the area of domain adaptation. like many nlp tasks, thai ws is domaindependent. for instance, @xcite recorded an accuracy drop from 91% to 81% when their model trained on a generic domain corpus @xcite was tested on a social media one @xcite . results from our analysis (section 3) also conform to these findings. one way to solve the domain dependency problem is through transfer learning (tl), which is a common technique in domain adaptations @xcite . however, tl may not be applicable when working with a commercial api or a model that does not support weight adjustments @xcite @xcite . we call this type of model a black box. in this paper, we propose a stacked-ensemble learning solution to overcome the black-box limitation. instead of making changes to the existing model directly, we build a separate model to improve the accuracy of predictions made by the black box. our solution comprises two parts, domain-generic (dg) and domain-specific (ds). the pretrained black box handles the domain-generic part, and a new model is constructed to handle the domain-specific part. all samples go through domain-generic, which makes initial predictions. we rank all predictions according to uncertainty and send the top-k uncertain predictions to domain-specific for further consideration. we combine the predictions from domain-specific with the remaining from domain-generic to form the final predictive results. we conducted extensive experimental studies to assess our solution's performance against a base-line model and transfer learning solutions. we also applied our stacked-ensemble filter-and-refine (sefr) technique to chinese and japanese. experimental results showed that our proposed solution achieved the accuracy level comparable to those of transfer learning solutions in thai. for chinese and japanese, we showed that model adaptation using the sefr technique could improve the performance of black-box models when used in a cross-domain setting. our contributions are as follows. first, we propose a novel solution for adapting a black-box model to a new domain by formulating the problem as an ensemble learning one. second, we derive a filter-and-refine method to speed up the inference process without sacrificing accuracy in some cases. third, we conducted extensive experimental studies; experimental results validate the effectiveness of our solution. fourth, we make our code available at: github.com/mrpeerat/sefr_cut 2 stacked-ensemble method ## performance evaluation we evaluated our sefr solution against state-ofthe-art models on nine benchmark corpora from three languages. specifically we studied the effect of our sefr method and report the performance by adapting a black-box model to a new domain by formulating the problem as an ensemble learning. ## conclusion we proposed a novel solution for adapting a blackbox model to a new domain by formulating it as an ensemble learning problem. we conducted extensive experimental studies using nine benchmark corpora from three languages. for thai word segmentation, the results showed that our method is an effective domain adaptation method and has similar performance as the transfer learning method. the results from japanese and chinese word segmentation experiments showed that our method could improve the performance of japanese and chinese black-box models.