BLOSSOM: Best Path Length on a Semantic Self-Organizing Map

BLOSSOM: Best path Length on a Semantic Self-Organizing Map Robert V. Lindsey (lindsr@rpi.edu) Michael J. Stipicevic (stipim@rpi.edu) Vladislav D. Veksler (vekslv@rpi.edu) Wayne D. Gray (grayw@rpi.edu) Rensselaer Polytechnic Institute Cognitive Science Department, 110 8th Street Troy, NY 12180 USA unlimited vocabularies. These measures are not as computationally intensive as vector-based MSRs, but suffer from a number of deficits. One issue is that statistical MSRs cannot measure the relatedness of large multi-word terms (e.g. documents). Another issue relates to second-order word co-occurrence (words that do not occur together, but are often found in similar contexts). There is evidence in the literature that vector-based measures provide better performance over statistical measures due to context-based word processing (Budiu, Royer, & Pirolli, 2007; Landauer, et al., 2007). In this paper, we describe VGEM (Veksler, Govostes, & Gray, 2008), a technique combining the benefits of statistical MSRs with the power of vector-based MSRs. We then propose BLOSSOM, an MSR that utilizes a Self- Organizing Map to reduce noise in vector-space semantic models. The psychological validity of the proposed technique is then evaluated using human word-association norms. BLOSSOM is shown to exhibit consistent improvement over VGEM. Finally, we describe the unique benefits of using BLOSSOM over other MSRs, including intuitive visualizations and concept-path formations. Abstract We describe Vector Generation from Explicitly-defined Multidimensional semantic Space (VGEM), a method for converting a measure of semantic relatedness (MSR) into vector form. We also describe Best path Length on a Semantic Self-Organizing Map (BLOSSOM), a semantic relatedness technique employing VGEM and a connectionist, nonlinear dimensionality reduction technique. The psychological validity of BLOSSOM is evaluated using test cases from a large free-association norms dataset; we find that BLOSSOM consistently shows improvement over VGEM. BLOSSOM matches the performance of its base-MSR using a 21 dimensional vector-space and shows promise to outperform its base-MSR with a more rigorous exploration of the parameter space. In addition, BLOSSOM provides benefits such as document relatedness, concept-path formation, intuitive visualizations, and unsupervised text clustering. Keywords: Measures of Semantic Relatedness, Self- Organizing Maps, nonlinear dimensionality reduction, computational linguistics, natural language processing, VGEM, BLOSSOM, Dijkstra’s algorithm, SOM traversal Introduction VGEM Measures of Semantic Relatedness (MSRs) are a class of computational methods that calculate association strengths between terms in order to quantify the meaning of text. MSRs have a wide variety of applications derived from their mathematical modeling of text meanings. These applications range from the computational modeling of human text comprehension (Lemaire, Denhiere, Bellissens, Jhean- Larose, 2006) to automated essay-grading algorithms (Landauer & Dumais, 1997). Most MSRs employ either statistical or vector-based techniques. Vector-based MSRs such as Latent Semantic Analysis (LSA; Landauer & Dumais, 1997) and Generalized Latent Semantic Analysis (GLSA; Matveeva, Levow, Farahat, & Royer, 2005) generally model term-to-term association strengths well, but require significant computational resources to manipulate vector-space models of text. For example, LSA involves the difficult task of solving a “large, sparse symmetric eigenproblem” (Landauer, McNamara, Dennis, & Kintsch, 2007, p. 45). Statistical MSRs such as Pointwise Mutual Information (PMI; Turney, 2001) and Normalised Google Distance (NGD; Cilibrasi & Vitanyi, 2007) are often based on search engine technology (e.g. Google search) and have virtually To convert a statistical MSR M into vector form, we use Vector Generation from Explicitly-defined Multi- dimensional semantic space (VGEM). VGEM's semantic space is explicitly defined by a set of dimensions d = {d 1 , d 2 , . , d n }, where each word defines a single dimension. To compute the vector representation of a word in this semantic space, VGEM uses its base-MSR M to calculate the semantic relatedness between the target word w and each dimension in d as follows: v(M,w,d) = [ M(w,d 1 ), M(w,d 2 ), . , M(w,d n ) ] For example, if d = { animal , friend } and the word in question is dog , then the vector for dog would be [M( dog , animal ), M( dog , friend )]. If M( dog , animal ) is 0.81 and M( dog , friend ) is 0.84, then the vector is [0.81, 0.84] as demonstrated in Table 1 and Figure 1. This example uses only two dimensions, though typically many more are used. Like other vector-based measures (e.g. LSA, GLSA), VGEM defines similarity between two words to be the cosine of the angle between the vectors that represent those words. As the angle becomes smaller and the cosine approaches 1.0, the words are considered more related. A