In this paper , we propose a method for learning a classifier which combines outputs of more than one Japanese named entity extractors . The proposed combination method belongs to the family of stacked generalizers , which is in principle a technique of combining outputs of several classifiers at the first stage by learning a second stage classifier to combine those outputs at the first stage . Individual models to be combined are based on maximum entropy models , one of which always considers surrounding contexts of a fixed length , while the other considers those of variable lengths according to the number of constituent morphemes of named entities . As an algorithm for learning the second stage classifier , we employ a decision list learning method . Experimental evaluation shows that the proposed method achieves improvement over the best known results with Japanese named entity extractors based on maximum entropy models . In the recent corpus-based NLP research , system combination techniques have been successfully applied to several tasks such as parts-of-speech tagging ( van Halteren et al . , 1998 ) , base noun phrase chunking ( Tjong Kim Sang , 2000 ) , and parsing ( Henderson and Brill , 1999 ; Henderson and Brill , 2000 ) . The aim of system combination is to combine portions of the individual systems ' outputs which are partial but can be regarded as highly accurate . The process of system combination can be decomposed into the following two sub-processes : 1 . Collect systems which behave as differently as possible : it would help a lot if at least the collected systems tend to make errors of different types , because simple voting technique can identify correct outputs . Previously studied techniques for collecting such systems include : i ) using several existing real systems ( van Halteren et al . , 1998 ; Brill and Wu , 1998 ; Henderson and Brill , 1999 ; Tjong Kim Sang , 2000 ) , ii ) bagging / boosting techniques ( Henderson and Brill , 1999 ; Henderson and Brill , 2000 ) , and iii ) switching the data expression and obtaining several models ( Tjong Kim Sang , 2000 ) . 2 . Combine the outputs of the several systems : previously studied techniques include : i ) voting techniques ( van Halteren et al . , 1998 ; Tjong Kim Sang , 2000 ; Henderson and Brill , 1999 ; Henderson and Brill , 2000 ) , ii ) switching among several systems according to confidence values they provide ( Henderson and Brill , 1999 ) , iii ) stacking techniques ( Wolpert , 1992 ) which train a second stage classifier for combining outputs of classifiers at the first stage ( van Halteren et al . , 1998 ; Brill and Wu , 1998 ; Tjong Kim Sang , 2000 ) . In this paper , we propose a method for combining outputs of ( Japanese ) named entity chunkers , which belongs to the family of stacking techniques . In the sub-process 1 , we focus on models which differ in the lengths of preceding / subsequent contexts to be incorporated in the models . As the base model for supervised learning of Japanese named entity chunking , we employ a model based on the maximum entropy model ( Uchimoto et al . , 2000 ) , which performed the best in IREX ( Information Retrieval and Extraction Exercise ) Workshop ( IREX Committee , 1999 ) among those based on machine learning techniques . Uchimoto et al . ( 2000 ) reported that the optimal number of preceding / subsequent contexts to be incorporated in the model is two morphemes to both left and right from the current position . In this paper , we train several maximum entropy models which differ in the lengths of preceding / subsequent contexts , and then combine their outputs . As the sub-process 2 , we propose to apply a stacking technique which learns a classifier for combining outputs of several named entity chunkers . This second stage classifier learns rules for accepting / rejecting outputs of several individual named entity chunkers . The proposed method can be applied to the cases where the number of constituent systems is quite small ( e.g. , two ) . Actually , in the experimental evaluation , we show that the results of combining the best performing model of Uchimoto et al . ( 2000 ) with the one which performs poorly but extracts named entities quite different from those of the best performing model can help improve the performance of the best model . This paper proposed a method for learning a classifier to combine outputs of more than one Japanese named entity chunkers . Experimental evaluation showed that the proposed method achieved improvement in F-measure over the best known results with an ME model ( Uchimoto et al . , 2000 ) , when a complementary model extracted named entities quite differently from the best performing model .

Challenge: System combination techniques have been shown successful in corpus-based research which aims to combine portions of individual systems' outputs that are partially but highly accurate.
Approach: They propose a decision list-based classifier which learns to combine outputs from multiple maximum entropy-based Japanese named entity extraction models with different context lengths.
Outcome: They show that combining the existing best performing model with worse-performing models improves the performance of the original best model.