Comparative Concepts or Descriptive Categories: a UD Case study (2025)

Abstract: In this paper, we present a series of methods used to quantify the soundness of using the same names to annotate cases in different languages. We follow the idea described by Martin Haspelmath that descriptive categories and comparative concepts are different objects and we look at the necessary simplification taken by the Universal Dependencies project. We thus compare cases in closely related languages as belonging to commensurable descriptive categories. Then we look at the corresponding underlying comparative concepts. We finally looked at the possibility of assigning cases to adpositions.

## introduction there is a fundamental distinction between language-particular categories of languages (which descriptive linguists must describe by descriptive categories of their descriptions) and comparative concepts (which comparative linguists may use to compare languages). martin haspelmath in @xcite language description and language comparison are two intertwined yet distinct endeavours. language description is often done in a language different from the one being described (many grammars have been written in english, french, russian, spanish and portuguese for example) and often uses a conventionalised descriptive meta-language associated with a given descriptive school. language comparison relies on the previous step of language description as it main data source but also needs a common metalanguage to name the various phenomena under study. in his paper, @xcite warns us against the confusion of the different metalanguages (the descriptive languages used in each individual description and the common comparative meta-language). he advocates for a careful choice of terms when describing similar categories across multiple languages, even when the similarities compel us to use the same term. that is, one should avoid using a single term to describe two categories from two different languages. even more so, when this term is also used as a comparative concept which then further increases the risk of cross-meta-language confusion. with all its qualities, the universal dependencies (ud) project @xcite puts itself exactly in this somewhat uncomfortable situation. one of the main aims of the project is to foster linguistic typological research, and thus it proposes a common annotation scheme for creating treebanks for all natural languages @xcite . figure 1 depicts the dependency tree of a turkish sentence as an example. while the scheme has means to accommodate language specific phenomena, its core is language agnostic and treebank creators are compelled to reuse previously defined language specific extensions when annotating similar structures in new languages as a mean to increase the overall consistency and comparability of the corpora. in the dependency tree, the labels of the edge going out of a node is called its dependency relation and the target of the edge it the governor of the node. however, the annotation also needs to be sound from the point of view of each annotated language (see points 1 and 2 of the presentation page at https://universaldependencies. org/introduction.html ). each individual treebank can thus be seen as a kind of description of its language. indeed, that is exactly what @xcite do in their work, where they use sparse representation methods to try to extract a grammar sketch for a language from its annotated treebank. in ud, the same terms are thus used both as comparative concepts and as descriptive categories for all the languages that express that category. in this study, we investigate the descriptivecomparative confusion arising from ud's annotation scheme at the morphosyntactic level. we especially focused on the category of case and its different realisations across several languages with the following question in mind: do cases sharing their name have the same value across different languages? the main reason to focus on the case category, is that it has both strongly syntactic and strongly semantic values. for example, in languages with a case marking the subject of both transitive and intransitive verbs, this case is usually called nominative foot_0 based on its syntactic properties. if the same language has another case marking the "together with" relation, it will usually be called comitative on semantic ground. this study should provide insight on the extent to which one can transfer information about a feature from a language to another simply by reusing the same name (using the same descriptive category). in the end, it could help improve crosslingual learning scenarios where we want to use as much information from other languages as we can, even at the morphological and syntactic levels. this paper is organised as follows. section 2 gives an overview of ud's guidelines on case annotation and how these are realised in practice. section 3 describes how we assign representations to cases. section 4 looks at the similarity between cases from different languages as if they were descriptive categories. section 5 then turns to looking at cases as comparative concepts applied to each individual treebank. section 6 takes an in between look directly at the cases from all the treebanks. section 7 investigates the possibility of assigning cases directly to adpositions. eventually, section 8 concludes this paper. ## the case feature across treebanks while realising this study, we stumbled upon a number of incongruities in the way the different corpus use the case feature. there are essentially three ways the feature case is used in the ud treebanks. the first and by far the most common use is to annotate inflected forms of nouns, pronouns and proper nouns in languages where these words inflect according to their role in a clause, as well as determiners, adjectives and participles in languages where they inflect to match the case of their governor. the second use that is documented in ud's guidelines foot_1 , is to annotate adpositions with the case they give to their nominal phrase, especially so in languages without over case marking on nouns. this annotation principle indicates that ud leans more toward the application of comparative concepts to individual languages. indeed, if a language does not use the case category, then the "case" represented by an adposition can only be inferred either by comparing its distribution to the distribution of actual cases in languages that possess that category, or by applying formal comparative definitions. however, this is not always how this feature is used, as in czech cltt treebank @xcite for example, adpositions are annotated with the case feature and their value always match that of their governing noun. this is all the es ¸eklerin sırtlarına yüklenmis ¸sepetlerle tas ¸ınırdı üzümler . noun noun verb noun verb noun punct @xmath0:poss obl acl obl nsubj punct figure 1: representation of the dependency graph of the turkish sentence "es ¸eklerin sırtlarına yüklenmis şepetlerle tas ¸ınırdı üzümler." from ud's turkish boun corpus, meaning "grapes were carried in baskets loaded on donkeys' backs." more surprising that czech adpositions are invariable and can license several case values. this indeed points to another problem with case annotation on adpositions. like languages exhibiting case syncretism foot_2 , adpositions can in principle also be used to mark different syntactic and semantic roles. it becomes then even less clear how one should proceed in assigning cases to adposition. the third and most divergent use of the case feature can be seen in the persian seraji treebank. in this treebank, we only find three case values : @xmath1, @xmath2. the first two values are exclusively used to annotate adverbs of place and adverbs of time respectively. the third value is used to annotate an interjection used to create vocative noun phrase. ## case representation in order to compare cases from different languages, we need to find a shared representation that should be as language agnostic as possible. we decided to use the syntactic profile of a case defined as the probability distribution foot_3 over the dependency relations to its governors. this choice is both theoretical since core cases are usually defined in terms of syntactic relations to the other constituents of a sentence, and practical since ud treebanks are annotated with dependency labels. in order to make the representations even more language agnostic, we decided to ignore rela-tion sub-types since they are not consistently used across languages and corpora. so, both flat:foreign and flat:name are counted as flat. we give two representations to each case in a language. the first is the empirical probability distribution of the relation of a word displaying that case to its governor. however, there are several mechanisms underlying case assignment, and not all are as informative. for example, when determiners inflect for case, they usually inherit their value from their head noun, which therefore does not teach us much about that case since a determiner can in principle take any case that way. similarly, it would artificially separate cases from languages with articles (a high proportion of det relations) from those of languages without. furthermore, as mentioned in the previous section, ud also allows annotation of the case feature on adpositions, which is quite different from the way cases are generally assigned to nouns. for all these reasons, we thus decided to have a partof-speech based representation too. the second representation is thus the syntactic profile of the nouns (noun) which bear the said case. this gets rid of less informative dependency relations such as case, amod or det and we further decided to ignore the conj relation for similar reasons. the relation distributions are computed from the concatenation of the three parts (train, dev and test) of each treebank from ud version 2.14 @xcite , except when precised otherwise. ## sharing descriptive categories with our case representations, we first look at cases used in different treebanks as representing the values of a descriptive category. we want to know how relevant is to apply the same name to values of a similar category in different languages. first, we compare case labels from two closely related languages, namely czech and russian foot_4 . to do so we compute the euclidean distance between each case in the first language and each case in the second language. then, we generate a 1nearest neighbour graph assuming the neighbours of a node must come from the other language. this gives us an idea of the way cases could be mapped in a transfer learning setting for example. figure 2 represents the 1-nearest neighbour graph of czech and russian cases when representations are computed over all the words marked for case. we see that the czech and russian nomina-tives are each other's nearest neighbour and such is the case for the two genitives. however, for the other cases, the picture is less clear. this is likely due to the fact that when we compute the representations using all the parts-of-speech at once, we confuse the different types of case assignment. figure 3 which represents the 1-nearest neighbour graph of czech and russian cases when representations are computed only on nouns, is clearer. on top of the nominatives and geni-tives, the accusatives and instrumentals are also each other's nearest neighbours. only the datives, locatives and russian partitive are still entangled. looking directly at the data, we realize that the iobj relation is never used in the czech cllt corpus. the increased probability of seeing a noun in the dative descending from an obl relation makes the czech dative more distinct from the russian dative and the czech locative is. the distance matrices for these two graphs can be found in the appendix, along with distance matrices for czech -turkish. in the latter, we might for example see that the equative behaves erratically on nouns, but that simply comes from the fact that only one noun is annotated with equative in the turkish boun corpus. note that not all pairs of languages are as well behaved as czech and russian, as we shall see in section 6. ## applying comparative concepts in the previous section we have compared cases from two languages as if they were from a commensurable descriptive category. in this section, we take the other view that universal dependencies defines comparative concepts and that the various treebanks are annotated with these concepts. this means that each case has a language agnostic definition and that it is then applied to each language accordingly. here, the data used is only from the dev part of the treebanks, for computational time reasons. since we do not have language agnostic mathematical representations of the various grammatical cases used in ud's annotations, we need to extract them from the available annotated corpora. since a case profile depends not only on the choice of a language, but also on the sentences in the corpus (replacing a few sentences will generally slightly affect the frequency distribution), we model each comparative case with a random variable taking values from the probability distributions (or normalized frequency distributions) over the set of dependency relations to a word's governor. formally, let c be a case, d a dependency relation and t a treebank. we note f t (c, d) the frequency at which a word inflected in case c is attached to its governor via a relation of type we know each of these random variables through a number of realisations: the vector representations of the considered case across all corpora where it is present (which are exactly the probability distributions representations π t (c, •) for each corpus t ). that is, this random variable maps a language/corpus to a probability distribution over the dependency relations reaching words marked with that case. then, to compute the profile of the comparative cases, we compute the expected value of the random variables associated to each case. since the values of the random variables are distributions we also compute the barycenter of the realisations of each variable for the wasserstein 1-distance (or earth mover's distance). we will denote the latter by wasserstein barycenter. another interesting fact is that the dative's main role is not that of indirect object but rather of oblique. this comes from the strong limitations that ud imposes on the use of the iobj relation. but still, dative is virtually the only case to assume that role. however, while this representation allows us to distinguish many cases syntactically, it doesn't allow to distinguish all cases. more specifically, some cases work in the same syntactic constructions and thus are mostly distinguished through their semantic properties. for example, the finnish ellative and illative are used to signify that a movement respectively comes from a place or into a place. in the sentence "i went into his house", house would be in illative in finnish, while in "i come back from his house", house would be in ellative. to check the representativeness of a comparative case p of its realisations across treebanks, we compute also compute its energy e. the energies associated to the two barycenters are of the same magnitude, with the wasserstein barycenter being more exacerbated as can be seen in figure 4 for the accusative case. here, the ρ i are ℓ 1 -normalized vectors representing cases, and d is the metric used to define the geometry of the space (here, we use the ℓ 2 -metric and the wasserstein 1-distance). the x-axis represents the different dependency relations leading to nouns in the accusative, the exact list is given in the appendix for convenience. it represents in red the uniform mean of distributions (the expectancy of the variable), in yellow the barycenter of the distributions associated to the wasserstein 1-distance and in purple the (unnormalized for graphical purposes) apparition frequency. we can notably see that for uniform mean some relations are represented because very present in a few languages while this is not the case for the wasserstein barycenter, which is more centered on the dependency relations present in a lot of languages. ## case clustering in this section we apply data visualisation techniques as a mean to look at the general landscape of case across languages. this is a way to explore similarity between cases for many languages at once and without assuming a prototypical representation for each case. from a practical annotation perspective, this is interesting since it is more likely too capture the underlying structure of ud's annotations. indeed, ud's guidelines are sometimes underspecified, which is expected from an annotation scheme whose aim is to be applicable to as many languages as possible. not all use cases and language specific phenomena will have been thought of during the creation of the guidelines. therefore, when annotators stumble upon a new structure that does not lend itself to a straightforward analysis, they will both turn to the guidelines and to other treebanks in order to see how similar phenomena might have been annotated in other languages. we first used a t-sne analysis (van der @xcite with the hope of seeing well defined clusters. however, plotting all the cases at once proved unmanageable and so we resorted to visualising only a pair of cases each time. the algorithm consists in looking at the probability distribution generated by the high dimensional vectors 7 representing each instance of the cases and generating a distribution over pairs of those vectors in a way that pairs of close vectors are assigned higher probabilities. then t-sne defines a probability distribution on pairs of 2d points that minimizes the kullback-leibler divergence between the two distributions. figures 5 and 6 represents the t-sne applied to all the nominatives and genitives using either the profiles computed on all the words, or just on nouns. it seems that the two cases make for clusters, in the sense they can be grouped along distinct directions. while this is not enough for us to have a classification algorithm, it hints towards 7 here the vectors are normalized for the ℓ 1 -norm, but we do not consider them as probability distributions -30 -20 -10 0 10 20 30 40 component 1 -30 -20 -10 0 10 20 30 component 2 2d t-sne analysis on nouns for gen, nom gen nom 1 figure 6: representation of 2d t-sne analysis of genitive and nominative profiles gathered only on the nouns inflected for these cases. to confirm this hunch we tried to use tomato @xcite , a persistence based clustering algorithm, which uses sub-level sets of a function to design a persistence diagram and derive clusters. the implementation that was used comes from @xcite . the idea behind tomato is to compute the density at each point in the representation space and to cluster points using geodesics: every point above a certain elevation and inside the same geodesic belongs in the same cluster (the same hill) and every point below is ignored. by repeating the process for different elevations foot_6 we can see clusters appear and merge. when two clusters merge, the one with the highest elevation absorbs the other and we say that the lowest one dies. one can then represent on a diagram the birth and death time of each cluster. this is depicted in figure 7 for genitives and nom-inatives. the closer a cluster is to the diagonal the shorter its life and therefore the more likely it is to represent random noise rather than an actual cluster. in figure 7 the algorithm proposes multiple clusters, which could not be combined to form better defined clusters. this suggests, as already suggested by figures 5 and 6 that the possible clusters are not well defined and might overlap with each other. to try and measure the overlap of the clusters, we computed a confusion matrix by the method of the k-nearest neighbours. ## adposition annotation as discussed in section 2, some corpora in ud make use of the case feature on adpositions and it is recommended by ud's guidelines. given the postulate according to which all natural languages are equally expressive, one could indeed see case marking and the use of adpositions as two means of achieving the same linguistic goals. two means that are by no mean exclusive since languages that use case tend to have a rather limited inventory and use adpositions to express a broader range of meanings and relations. following @xcite , we have applied the methods described above to represent certain adpositions and to give them a syntactically equivalent case representation. this could partially prove the postulate, as well as help justifying the way some corpora annotate adpositions for case. to do so, we counted the dependency relations leading to the governors of each adposition. this gave us a distribution on syntactic usage of adpositions similar to a profile, and allowed us to compare adpositions to cases. ## conclusion in this paper, we have investigated the comparative-descriptive confusion that haspelmath warned us about using universal dependency data. we have compared cases between different languages as is it was a commensurable descriptive category and seen that at least for some closely related languages the alignment stands at least for core cases. we then tried to represent archetypal cases as if case was a comparative concept applied onto each treebank, and saw that core cases mostly align with our expectations. however, this asks for a more principled analysis of the use of the term nominative for the default case especially so when the nominative-accusative distinction does not exist or when it does not simply mark a syntactic role but also definiteness for example.