Rule extraction from support vector machines: A review
Over the last decade, support vector machine classifiers (SVMs) have demonstrated superior generalization performance to many other classification techniques in a variety of application areas. However, SVMs have an inability to provide an explanation, or comprehensible justification, for the solutions they reach. It has been shown that the black-box nature of techniques like artificial neural networks (ANNs) is one of the main obstacles impeding their practical application. Therefore, techniques for rule extraction from ANNs, and recently from SVMs, were introduced to ameliorate this problem and aid in the explanation of their classification decisions. In this paper, we conduct a formal review of the area of rule extraction from SVMs. The review provides a historical perspective for this area of research and conceptually groups and analyzes the various techniques. In particular, we propose two alternative groupings; the first is based on the SVM (model) components utilized for rule extraction, while the second is based on the rule extraction approach. The aim is to provide a better understanding of the topic in addition to summarizing the main features of individual algorithms. The analysis is then followed by a comparative evaluation of the algorithms salient features and relative performance as measured by a number of metrics. It is concluded that there is no one algorithm that can be favored in general. However, methods that are kernel independent, produce the most comprehensible rule set and have the highest fidelity to the SVM should be preferred. In addition, a specific method can be preferred if the context of the requirements of a specific application, so that appropriate tradeoffs may be made. The paper concludes by highlighting potential research directions such as the need for rule extraction methods in the case of SVM incremental and active learning and other application domains, where special types of SVMs are utilized.
svms machine learning data mining knowledge discovery information extraction pattern recognition applications