Linguistic Universals : Language-independent semantic fingerprints

Semantic processing1 is central to our understanding of natural languages, ensuring accuracy in monolingual communications, and minimizing losses in cross-lingual translations. The mechanism of semantics is a less-charted territory, unlike phonology 2–4, morphology5–8, syntax8–12, among other aspects 13 of human languages. Data-hungry algorithms in machine learning achieve impressive success in some tasks of document comprehension14,15, through high-dimensional numerical representations of words and phrases16,17. Such computationally taxing algorithms are far from the efficient mechanism by which we humans understand texts and acquire knowledge18,19. Here we advance a cost-effective model that assigns language-independent semantic fingerprints to words in a particular document, without consulting external knowledge-base 16 or thesaurus20. Our universal semantic fingerprints quantify local meaning of words in 14 representative languages across 5 major language families. Instead of embedding words into very high dimensional spaces, our method represents each concept by a few dozen parameters, interpretable as algebraic invariants in succinct statistical operations. Concise and transparent, our semantic fingerprints numerically characterise connectivity and association of individual concepts, even with scant input of data. These semantic representations enable a robot reader to both understand short texts in a given language (automated question-answering) and match medium-length texts across different languages (automated word translation). Quantitative semantic models for human languages have broad implications, in both linguistics and artificial intelligence. Such models will set the stage for efficient algorithms to clarify the denotations/connotations of words in context, which are vital to comprehension and translation. Previous quantitative studies of phonology2–4, morphology 5–8 and syntax 8–12 offered mechanistic explanations for the typological diversity of human languages in these aspects. Our current work presents a quantitative mechanism for semantics that is independent of typological features in specific languages. The universality of our semantic model draws on the separation of length scales in languages. On short scales such as syllables, words, and phrases, human languages do not exhibit a common pattern related to semantics. Except for a few onomatopoeias, the sounds of words do not affect their meaning21. Neither do morphological parameters22 (say, singular/plural, present/past) or syntactic rôles 11 (say, subject/object, active/passive). In short, there are no universal semantic mechanisms at the phonological, lexical or syntactical levels 23. Grammatical “rules and principles” 11,22 play no definitive rôle in determining the inherent meaning of a word. Motivated by the observations above, we will build our quantitative semantic model on long-range and language-independent textual features. Specifically, we will measure the lengths of text fragments flanked by word patterns of interest (Fig. 1a). Here, a word pattern is a collection of content words that are identical up to morphological parameters and syntactic rôles. A content word signifies definitive concepts (like apple, eat, red), instead of serving purely grammatical or logical functions (like but, of, the). Fragment length statistics will tell us how tightly/loosely one concept is connected to another. This in turn, will provide us with quantitative criteria for inclusion/exclusion of different concepts within the same (computationally constructed) semantic field. Such statistical semantic mining will then pave the way for machine comprehension and machine translation. Usually, one can assume that a reader processes texts at roughly uniform speed (Supplementary Information, section 1.1). So, up to a constant scaling factor, the recurrence times for a word pattern Wi are approximately distributed as nii samples of the effective fragment lengths Lii (Fig. 1a). Here, while counting as in Fig. 1a, we ignore contacts between short-range neighbours, which may involve language-dependent redundancies (Supplementary Information, section 1.2). As a working definition, we consider a word pattern Wi non-topical if its nii counts of effective fragment lengths Lii are exponentially distributed, within 95% margins of error (formula (1) in Methods section ‘Recurrence and topicality’). This definition hearkens back to the exponentially distributed recurrence times in a randomly reshuffled text 24, or a memoryless (hence banal) Poisson process (Extended Data Fig. 1b). In contrast, we consider a word pattern Wi topical if its diagonal statistics nii,Lii constitute significant departure from the Poissonian line (blue line in Fig. 1b). Notably, in Fig. 1b, most data points for topics in Jane Austen’s Pride and Prejudice mark systematic downward departures from the Poissonian line. This suggests that the topical recurrence times follow weighted mixtures of exponential distributions (formulae (2) and (3) in Methods section ‘Recurrence and topicality’; Extended Data Fig. 1c,c′). The diagonal statistics nii,Lii (Fig. 1a) have enabled us to extract topics automatically (Fig. 1b). The off-diagonal statistics ni j,Li j (Fig. 1a) will allow us to determine how strongly one word pattern Wi binds to another word pattern W j, through empirical Markov matrices P = (pi j) (Fig. 1c). This matrix formalism consistently reproduces word count statistics (Extended Data Fig. 2a) through Markov equilibrium. Moreover, the spectrum σ(P) (collection of eigenvalues) is approximately invariant against translations of texts (Fig. 1d), which can be explained by a matrix similarity transformation (equation (5) in Methods section ‘Markov text model’). Later on, specializing such spectral invariance to individual topical patterns, we will be able to generate semantic fingerprints through a list of topic-specific and language-independent eigenvalues. Here, we will be particularly interested in recurrence eigenvalues of individual topical patterns, which correspond to multiple decay rates in the weighted mixtures of exponential distributions (equation (2) in Methods section ‘Recurrence and topicality’).