Using support vector machines to determine linear separability

Most theories on categorization agree on a main principle: category learning involves creating a category representation, and categorizing an item involves comparing that item to the representations of different categories. The theories, however, disagree on the nature of these category representations. There are two main competing lines of thought on category representations: exemplar-based theories and prototype-based theories (Valentine, 1991). Prototype-based theories argue that objects are stored based on how similar they are to a central prototype (Rosch, 1973). In contrast, exemplar-based theories reason that objects are encoded in their absolute structure, defined by their own properties only and unrelated to any abstract summary representation (Medin & Schaffer, 1978). Years of research on the nature of categorization has resulted in mixed results, with evidence for both approaches. One example of an issue on which exemplarand prototype based theories make different predictions is linear separability. Two categories are considered linearly separable if a linear function of attributes exists that perfectly separates their exemplars (Ruts, Storms, & Hampton, 2004). According to prototype-based models, for any pair of linearly separable categories represented in a geometrical space, that space is divided into two half spaces by a linear function that defines the points which are equidistant towards both prototypes. An item is then categorized in the category with the closest prototype (in that geometrical space). Thus, category membership can be determined simply by looking at the distance to the prototype, making linearly separable categories relatively easy to learn. On the other hand, categories that are not linearly separable would take considerably longer to master, as this simple strategy of deciding on the closest prototype would not be sufficient to determine category membership. According to exemplar-based models, on the other hand, proximity to the center of the category plays no role of any kind (Ruts, Storms, et al., 2004). Thus, exemplarbased models predict that, all other factors kept constant, linearly separable categories are not easier to master than other categories. As prototype-based and exemplar-based models make different predictions regarding linearly separable categories, we can use those categories to shed light on the mechanisms that underlie categorization. In order to do so, however, we have to determine which categories are linearly separable, and which are not. There has been surprisingly little research into this issue. Studies that do investigate this tend to assess linear separability by first obtaining a geometric representation of the exemplars using multidimensional scaling, and then analyzing this representation with visual inspection (Malt, Sloman, Gennari, Shi, and Wang, 1999) or log linear analysis (Ruts, Storms, et al., 2004). The current research intends to expand on these previous studies in investigating which semantic categories are linearly separable and which are not. Compared to these existing studies, however, we will use a fundamentally different technique to assess linear separability: linear support vector machines (LSVMs). A support vector machine (SVM) is a mathematical concept used for supervised pattern learning (Vapnik, 1982; Cortes & Vapnik, 1995). Presented with a set of input data and their corresponding classes, an SVM learns which data correspond to which class. Once trained, the machine can be used for classification; for any given input, it predicts the corresponding class. SVMs transform the input vectors into a (usually) high dimensional feature space with the help of a kernel