NEURO-FRACTAL COMPOSITION OF MEANING : TOWARD A COLLAGE THEOREM FOR LANGUAGE

This paper presents languages and images as sharing the fundamental property of self-similarity. The self-similarity of images, especially those of objects in the natural world (leaves, clouds, galaxies), has been described by mathematicians like Mandelbrot, and has been used as the basis for fractal image compression algorithms by Barnsley and others. Self-similarity in language appears in the guise of stories within stories, or sentences within sentences (”I know what I know”), and has been represented in the form of recursive grammar rules by Chomsky and his followers. Having observed this common property of language and images, we present a formal mathematical model for putting together words and phrases, based on the iterated function system (IFS) method used in fractal image compression. Building (literally) on vector-space representations of word meaning from contemporary cognitive science research, we show how the meaning of phrases and sentences can likewise be represented as points in a vector space of arbitrary dimension. As in fractal image compression, the key is to find a set of (linear or non-linear) transforms that map the vector space into itself in a useful way. We conclude by describing some advantages of such continuous-valued representations of meaning, and potential implications. NOMENCLATURE Self-similarity, fractals, language, grammars, iterated function systems, recurrent neural networks. INTRODUCTION: SELF-SIMILARITY Self-similarity is a property by which an object contains smaller copies of itself at arbitrary scales. As noted by Mandelbrot (1988), this property is ubiquitous in the natural world, appearing in objects as diverse as leaves, mountain ranges, galaxies, and clouds. Figure 1 shows a mathematical abstraction of this concept, in which the original image is arrived at through a sequence of zooming operations.2 Self-similarity is also a hallmark of human language. The following verse, from a poem by Wallace Stevens (Stevens, Kermode, and Richardson 1997), meanders through a sequence Supported by a grant from the Keck Foundation. Images reproduced with permission from http://www.chanceandchoice. com Figure 1: The Mandelbrot Set at various scales. of relative clauses, arriving ultimately at self-description: I know noble accents And lucid, inescapable rhythms; But I know, too, That the blackbird is involved In what I know. THE TWO CULTURES The author and scientist C.P. Snow famously described the modern rift between the humanities and sciences as representing “two cultures”, each having its own distinct vocabulary and methodology, and frequently hostile to the other. (Snow 1964) A similar, if less dramatic, divide exists between the mathematical approaches used to understand cognition and language on the one hand, and most natural phenomena on the other. Cognitive science, linguistics, artificial intelligence, and formal logic have traditionally relied on the use of atomic symbols and the graph structures, grammars, and discrete calculi that operate on them. Electrical engineering and dynamical systems theory (among many other fields) have made use of metric spaces, continuous vectors, and continuous transforms to describe a broad variety of natural phenomena, from sounds and images to population dynamics and the formation of galaxies. Not surprisingly, the question of how the brain “does” language has been the focus of attention from both of these two camps. Neuroscientists, biologists, physicists, and engineers tend to see the brain as a massively connected signal-processing device operating on continuous-valued quantities (electrical, chemical, and acoustic) in a probabilistic manner. Linguists and cognitive scientists, on the other hand, have typically rejected this level of description as hopelessly inadequate or “low-level” for the phenomena that interest them, most notably the systematic composition of meaning that is the hallmark of human (versus animal) language (Chomsky 1956). THE CONNECTIONIST ALTERNATIVE Research in the field connectionist (“neural”) networks – the hallmark brain-inspired cognitive systems approach – has made significant inroads into providing a unified computational framework for addressing this issue. Strong criticism of feed-forward network models in the late 1980’s (Fodor and Pylyshyn 1988), (Pinker and Prince 1988) has helped fuel the drive for connectionist models that deal adequately with the issues of systematicity (roughly, grammar) and compositionality (roughly, structure) in a principled way. The Simple Recurrent Net (SRN) of Elman has shown impressive abilities in both inducing semantic structure (Elman 1990) and predicting long-distance grammatical dependencies (Elman 1991). For the task of representing explicit compositional structure, the Holographic Reduced Representation (HRR) model of Plate (Plate 2003) has yielded significant insights. The present work is likewise concerned with addressing the issues of linguistic systematicity and compositionality within a connectionist framework. Unlike the approaches taken by Elman and by Plate, however, our approach is built on a a view of language as a fundamentally self-similar object. 3 To flesh out this approach, we will therefore need at least the three following components: (1) a mathematical model of self-similarity (2) a way of representing primitive linguistic units (words) as primitives of the model (3) a way of representing the composition of these units in terms of the operations provided by the model. Beginning in the next section, we treat each of these requirements in turn. ITERATED FUNCTION SYSTEMS Iterated Function Systems provide perhaps the simplest mathematical apparatus for describing self-similar objects. Figure 2 shows a rather lifelike image of a fern plant, where the overall shape of the plant is replicated at various scales (stem, branch, leaf); i.e., the image shows a high degree of self-similarity. This image was generated by the Iterated Function System (IFS) method: starting with an arbitrary set of points in twodimensional space, the following set of equations was applied We do not mean to imply that SRNs or HRRs are inadequate for dealing with self-similar language structures; rather, we wish to highlight the distinguishing feature of our approach. −3 −2 −1 0 1 2 3 0 1 2 3 4 5 6 7 8 9 10 Figure 2: A fern generated by an Iterated Function System. Transforms were applied with the following probabilities: "! # $ % iteratively to its own output, until the image no longer changed: '& () * ,+ . /10 +2() 023 + 40 " &5()5* + 6 7 68 90 +2() 0 3 +  /:0 ;&5()5* + < 7 < / < = >< <? 0 +2() 0 3 +  /:0 "! & ( ) * + 7 . @ A ><B ><B/ ><C6 0 + ( ) 0 3 + 6 6 0 More generally, a given transform can be represented using the formula D & ( )5* +2E DGF@F E D.F H E D.HIF E D.HIH40 + ( ) 0 3 +2E DGF E D.H40 where the E are weights or coefficients representing scaling, rotation, and translation of the input. From a practical standpoint, an IFS represents a powerful means of compressing or encoding the digital representation of an image: instead of representing the entire set of points comprising the image – the so-called attractor of the IFS – we can store and transmit only the small number of IFS coefficients contained in the four IFS equations, or transforms. The actual image can then be reconstructed using the iterative method described above. In principle, this method can be used to encode images of arbitrary dimension – a fact that we will find