Number Theory Meets Linguistics: Modelling Noun Pluralisation Across 1497 Languages Using 2-adic Metrics (2022)

Abstract: A simple machine learning model of pluralisation as a linear regression problem minimising a p-adic metric substantially outperforms even the most robust of Euclidean-space regressors on languages in the Indo-European, Austronesian, Trans New-Guinea, Sino-Tibetan, Nilo-Saharan, Oto-Meanguean and Atlantic-Congo language families. There is insufficient evidence to support modelling distinct noun declensions as a p-adic neighbourhood even in Indo-European languages.

## introduction in this paper, we study whether p-adic metrics are a useful addition to the toolkit of computational linguistics. it has been known in the mathematical community since 1897 --although only clearly since @xcite -that there is an unusual and unexpected family of distance metrics based on prime numbers which can be used instead of euclidean metrics, which have infinitesimals (to support calculus), the triangle inequality (to support geometry), and other useful properties all the while maintaining mathematical consistency. they are known as the p-adic metrics. @xcite provides a valuable and readable introduction to p-adic analysis. given a prime number p it is possible to define a 1-dimensional distance function d as: if p (r -q) 1 p d p r p , q p otherwise (where x y means "x does not divide y") for example, if @xmath0, 4) = 1 3 and d 3 (2, 83) = 1 81 . in particular, if @xmath1, the authors have found that the 2-adic distance is a surprisingly useful measure for grammar morphology tasks. in many of the languages in this study we found that identifying the grammar rules for pluralisation turned into a problem of finding a linear regressor which minimised a p-adic metric. ## pluralisation as linear regression in this paper we use a simple and naive approach for converting vocabulary words into vectors: use whatever the unicode bit sequence for the word would be; this bit sequence can also be viewed as an integer vector with one element. this is of course extremely arbitrary and subject to the whims of the unicode consortium, but it is the most common way to represent text from any human language on a computer. note that in this naive encoding scheme words like "sky", "fry" and "butterfly" are very close using a 2-adic metric -the last 32 bits are the same, meaning that the distance between them is less than or equal to than 2 -32 . using a euclidean metric "butterfly" is at least 2 32 6 = 2 192 apart from the other two words. a little exploration will observe that noun declensions in many languages -especially ones in the indo-european family -have this property that they consist of words that form tight 2-adic clusters. this odd correspondence between 2-adic geometry and grammar morphology extends to declension rules for case and number where they exist. consider that the first two rules in figure 2 have the property that in the naive utf-32 encoding they 1. if the singular form ends in "y", replace the "y" with "ies". figure 2 : a simplified and incomplete set of rules for forming plurals in english can all be accurately modelled using a linear regression performed on points in the local 2-adic neighbourhood. the fourth rule is illustrated in figure 1 , with singulars and plurals of "cat", "dog" and "eye" plotted. they lie on the straight line @xmath2. ## experiment the aim of this research is to identify whether or not using a p-adic metric space is likely to generate improvements on computational linguistics tasks. a linear model will obviously not be able to capture irregular nouns. the 2-adic neighbourhood will not capture nouns that belong to different noun declensions but share the same ending. comparing a linear regression model (even if it is operating over an unusual space) to a million-parameter neural network 5 where such subtleties can be captured is going to be uninformative in telling us about the usefulness of p-adic metrics. as a result we are comparing p-adic linear regression against methods that are clearly not the state-of-the-art, but are methods which can be legitimately compared. the choice of the siegel regressor @xcite as the representative for euclidean regression was forced by the need for robustness to a large number of outliers. the leaftop data set is known to be only 72% accurate and any irregular nouns will also be outliers. huber 1964, theil-sen 1950 and ordinary least squares regression are all ruled out by these criteria. the siegel and p-adic regressors were run in "global" mode (learn from as many examples as possible) and "local" mode (learning from a small number of nearby words). to identify the impact of the p-adic neighbourhood vs the impact of the p-adic linear regressor, local siegel was run twice, once with a p-adic (a "hybrid" of a euclidean regressor and a p-adic neighbourhood) and once with a euclidean neighbourhood (labeled "local siegel"). the complete set of algorithms and their configurations is listed in the only metric that can be used for this comparison is l0 -accuracy -since any other metric (e.g. l1 or l2 norms) will bias the results towards the metric space that they operate in. a leave-oneout cross validation was done for each algorithm for each language. ## results a plot of results by algorithm is in figure 3 . summary statistics for each language family and algorithm combination are shown in in all language families (and overall across all languages), p-adic approaches outperformed euclidean ones, however the results were not all statistically significant. the differences in performance between algorithms on a language do not follow a normal distribution. since the research question is simply "which is better?" the magnitude of the effect is unimportant, and a wilcoxon signed-rank test can be used. the pratt method was used for handling situations where the scores were identical and no sign can be calculated. the probability is there are 80 statistical tests required to perform to confirm validity. there are 17 languages families in the ethnologue and glottolog plus another 3 pseudo-families from the leaftop labelling (unclassifed, unrecorded and all). for each of these 20 families, there are 4 tests: global p-adic vs global siegel; local p-adic vs local siegel; local p-adic vs siegel using a p-adic neighbourhood; siegel with a euclidean neighbourhood vs a p-adic neighbourhood. the correction to apply to the raw statistical test results is therefore @xmath3. it is this latter (corrected) number foot_5 that is reported in there is strong evidence that noun pluralisation in languages in the indo-european, austronesian, trans new guinea, sino-tibetan, niger-congo, nilo-saharan, oto-meanguean and atlantic-congo families can be modelled better with p-adic linear regression than with euclidean. this is also true for the unclassified languages in the leaftop dataset. moreover, the data in ## related work murtagh (e.g. his overview paper @xcite and bradley (e.g. @xcite have written the most on p-adic metrics in machine learning, having explored clustering and support vector machines in some depth. @xcite provides an algorithm for training a neural network. an extensive literature search has failed to find any other p-adic adaptions of traditional machine learning algorithms. this paper is the first to discuss p-adic linear regression. expanding the literature search more broadly, we find that there have been very few side-by-side comparisons of euclidean metrics versus strongly mathematically-formulated non-euclidean metrics for tasks in computational linguistics. @xcite , @xcite ) and @xcite performed their learning of word embeddings on a non-euclidean metric, choosing a poincaré hyperbolic space. calculating derivatives and finding minima of a function in a poincaré space is substantially more complex both mathematically and computationally than for a euclidean space. p-adics are simpler in both regards, but give rise to a space with similar hyperbolic properties. we believe that this may be a fruitful area of future research. ## conclusion we demonstrated superiority over euclidean methods on languages in the indo-european, austronesian, trans new-guinea, sino-tibetan, nilo-saharan and oto-meanguean and atlantic-congo based on this, we expect that substituting p-adic metrics for euclidean metrics in other computational linguistics tasks and machine learning methods may be an exciting area of research. 1. ∀i, j, @xmath4, y i -y j x i -x @xmath5. it contains the origin (x 0 , y 0 ) = (0, 0) and one of the optimal lines of best fit passes through the origin, and can therefore be written as @xmath6 a proof that the p-adic line of best fit passes through at least two points in the dataset the proof is in three sections: the phrase "optimal line" will be used to mean "one of the set of lines whose p-adic residual sum is equal to the minimum residual sum of any line through that data set". the notation res p ({(x i , y i )}, @xmath7, y i )}.