This paper describes an experimental comparison of seven different learning algorithms on the problem of learning to disambiguate the meaning of a word from context . The algorithms tested include statistical , neural-network , decision-tree , rule-based , and case-based classification techniques . The specific problem tested involves disambiguating six senses of the word " line " using the words in the current and proceeding sentence as context . The statistical and neural-network methods perform the best on this particular problem and we discuss a potential reason for this observed difference . We also discuss the role of bias in machine learning and its importance in explaining performance differences observed on specific problems . Recent research in empirical ( corpus-based ) natural language processing has explored a number of different methods for learning from data . Three general approaches are statistical , neural-network , and symbolic machine learning and numerous specific methods have been developed under each of these paradigms ( Wermter , Riloff , & Scheler , 1996 ; Charniak , 1993 ; Reilly & Sharkey , 1992 ) . An important question is whether some methods perform significantly better than others on particular types of problems . Unfortunately , there have been very few direct comparisons of alternative methods on identical test data . A somewhat indirect comparison of applying stochastic context-free grammars ( Periera & Shabes , 1992 ) , a transformation-based method ( Brill , 1993 ) , and inductive logic programming ( Zelle & Mooney , 1994 ) to parsing the ATIS ( Airline Travel Information Service ) corpus from the Penn Treebank ( Marcus , Santorini , & Marcinkiewicz , 1993 ) indicates fairly similar performance for these three very different methods . Also , comparisons of Bayesian , information-retrieval , neural-network , and case-based methods on word-sense disambiguation have also demonstrated similar performance ( Leacock , Towell , & Voorhees , 1993b ; Lehman , 1994 ) . However , in a comparison of neural-network and decision-tree methods on learning to generate the past tense of an English verb , decision trees performed significantly better ( Ling & Marinov , 1993 ; Ling , 1994 ) . Subsequent experiments on this problem have demonstrated that an inductive logic programming method produces even better results than decision trees ( Mooney & Califf , 1995 ) . In this paper , we present direct comparisons of a fairly wide range of general learning algorithms on the problem of discriminating six senses of the word " line " from context , using data assembled by Leacock et al . ( 1993b ) . We compare a naive Bayesian classifier ( Duda & Hart , 1973 ) , a perceptron ( Rosenblatt , 1962 ) , a decision-tree learner ( Quinlan , 1993 ) , a k nearest-neighbor classifier ( Cover & Hart , 1967 ) , logic-based DNF ( disjunctive normal form ) and CNF ( conjunctive normal form ) learners ( Mooney , 1995 ) and a decisionlist learner ( Rivest , 1987 ) . Tests on all methods used identical training and test sets , and ten separate random trials were run in order to measure average performance and allow statistical testing of the significance of any observed differences . On this particular task , we found that the Bayesian and perceptron methods perform significantly better than the remaining methods and discuss a potential reason for this observed difference . We also discuss the role of bias in machine learning and its importance in explaining the observed differences in the performance of alternative methods on specific problems . This paper has presented fairly comprehensive experiments comparing seven quite different empirical methods on learning to disambiguate words in context . Methods that employ a weighted combination of a large set of features , such as simple Bayesian and neural-network methods , were shown to perform better than alternative methods such as decision-tree , rule-based , and instancebased techniques on the problem of disambiguating the word " line " into one of six possible senses given the words that appear in the current and previous sentence as context . Although different learning algorithms can frequently perform quite similarly , they all have specific biases in their representation of concepts and therefore can illustrate both strengths and weaknesses in particular applications . Only rigorous experimental comparisons together with a qualitative analysis and explanation of their results can help determine the appropriate methods for particular problems in natural language processing .

Challenge: Statistical, neural-network, and symbolic machine learning and numerous specific methods have been proposed but only a few works compare them with identical test data.
Approach: They compare statistical, neural-network, decision-tree, rule-based, and case-based classification techniques on the disambiguation of meanings of words from contexts using identical training and test sets.
Outcome: They show that the methods that employ a weighted combination of a large set of features such as statistical and neural-network methods perform the best.