Meaning-Text Theory within Abstract Categorial Grammars: Toward Paraphrase and Lexical Function Modeling for Text Generation (2023)

Abstract: The meaning-text theory is a linguistic theory aiming to describe the correspondence between the meaning and the surface form of an utterance with a formal device simulating the linguistic activity of a native speaker. We implement a version of a model of this theory with abstract categorial grammars, a grammatical formalism based on lambda-calculus. This implementation covers the syntax-semantic interface of the meaning-text theory, i.e., not only the three semantic, deep-syntactic and surface-syntactic representation levels of the theory, but also their interface (i.e., the transformation from one level to another). This implementation hinges upon abstract categorial grammars composition in order to encode level interfaces as transduction operate.

## introduction we present in this article our implementation of a model of the meaning-text theory (mtt, @xcite with abstract categorial grammars @xcite , focusing on the meaning to text direction, i.e., generation. mtt is a linguistic theory that has already been used in a generation context, while acgs are a grammatical framework known to encode a range of various grammatical formalisms. mtt aims to describe the link between the meaning and the textual representation of an utterance. this description is made possible thanks to a formal device, a meaning-text model (mtm), that simulates the linguistic activity of a native speaker. it uses, among others, a dependency syntax and the key concepts of paraphrase and lexical functions (lf) (see section 2.2). the latter enables a text to be more natural: the syntagmatic lfs for instance encode collocations or support verbs. they play an important role, especially for surface representations, and when producing text. some formalisations and implementations focusing on text generation already exist, such as gust @xcite or marquis @xcite . acgs are a grammatical framework based on λ-calculus. they enable the implementation of other grammatical formalisms, and have many advantages. acgs are reversible. we can therefore use them in generation or analysis @xcite . their capacities to encode other formalisms, such as tree adjoining grammars @xcite ), and to be used in generation were employed to generalize the g-tag model @xcite . they are also currently used in an industrial context by yseop. we aim in this article to use these properties with another linguistic theory that was already proven usefull for generation systems: mtt. we may now wonder if acgs are a relevant tool to implement a linguistic theory based on a dependency syntax by assessing it on a text generation task. this article presents the feasibility of such an implementation, based on a restricted number of examples which illustrate several specificities of mtt. as a grammatical framework, acgs can provide the grammatical formalisms they encode with their generic algorithms, making it unnecessary to develop and implement specific information. they also offer a reversible encoding so that, for instance, we get here both synthesis and analysis for mtt. because we wish to have a fined-grained control over the generated text, we choose to focus on text generation with formal methods. the same motivations can be found in grammatical framework @xcite , that use a type-theoretic system too. the encoding and links to other formalisms in acgs have been well studied. indeed, ## abstract categorial grammars acgs (de @xcite , whose definitions we use here) are a grammatical formalism based on λcalculus. an acg is composed of two languages, linked together by a lexicon. the first langage is called the abstract language and is the set of abstract grammatical structures, such as analysis trees. the second one, called the object language, is the set of the surface representations generated by the abstract language, such as strings or logical representations in the form of a graph. each one of these languages is a set of λ-terms obtained by induction over a signature. definition 1 let a be a set of atomic types. t (a) is the set of linear implicative types, obtained inductively over a: definition 2 let σ be a higher order signature. σ is of the form σ = ⟨ a, c, τ ⟩, where: we express with ⊢ σ 1 t : s that the type of a λterm t is s in the signature σ (or t : s if there is no ambiguity). we express λ(σ) the set of λ-terms obtained using the constants of c, the variables, the abstractions and the applications. definition 3 let σ 1 and σ 2 be two signatures. a lexicon l 12 from σ 1 to σ 2 is a pair of morphisms ⟨f, g⟩ such that f : τ (a 1 ) -→ τ (a 2 ) and g : λ(σ 1 ) -→ λ(σ 2 ). we write l 12 (t) = γ to express that γ is the interpretation of t by l 12 (or t := γ if there is no ambiguity on the used lexicon). the signatures (like σ dsynt tree , see figure 2 ) we describe here use almost linear λ-terms. we will not explain this notion, for it is not of great interest for what we say (the used variables being neither discarded nor duplicated in the lexicons). nevertheless we use the notation λ o and λ for the linear and non-linear abstractions respectively. moreover, an interesting property of acgs is that when they are second order almost linear, the morphisms inversions (see below) are decidable in a polynomial time @xcite , and when they are not almost linear, they remain decidable, even though the complexity is not polynomial anymore @xcite . we therefore were careful to use as much as possible second order almost linear acgs in this implementation. we may thereby define σ dsynt tree in figure 2 that corresponds to the dsyntr level. indeed, the constants of this signature enable to build the deepsyntactic trees of dsynts, like the ones in figure 3 . constant type c dt john : l c dt be : l c dt restless : l c dt upset : l c dt calm : l a1 : rel a2 : rel attr : rel in due to space considerations, we will not define a and c in the following paragraphs and sections anymore, they can be deduced from the table representing τ . we may now define the notions of acg, abstract and object languages: definition 4 an abstract categorial grammar is a tuple @xmath0, σ 2 , l 12 , s⟩ where: definition 5 the abstract language a and the object language o of an acg @xmath1, σ 2 , l 12 , s⟩ are: in this article we use βη-equivalence as equality between λ-terms. σ dsynt tree illustrated above gives the needed constants to build the dependency trees of but, in order to do so with an acg, we need another signature as well as a lexicon: we now introduce the abstract signature σ deep syntactic (see we define the notion of order and complexity of an acg as well. they are used in definition 6 @xcite the order of an acg is the maximum of the order of its abstract constants. the order of an abstract constant is the order of its type τ . the order of a type τ ∈ t (a) is inductively defined: the complexity of an acg is the maximum of the orders of its atomic types realizations. an acg of order γ and of complexity η is written acg (γ,η) . in the following paragraphs and sections, we use the notation c x l for the constant of σ x encoding the lu l. if there are two different constants representing the same lu l in σ x , we write c x l1 and c x l2 to distinguish them. we also use γ x i for the complex λ-term of σ x indexed by i. these complex λ-terms being the encoding of possible representations for an expression, the index i indicate this expression. therefore, we will use au for "john is not not upset" (cf. figure 3a ), ar for "john is not not restless" (cf. figure 3b ), and c1, c2 for "john is not calm". we use dt and d0f instead of dsynt tree and dsynt 0 f l. that being said, we define the complex λ-terms: moreover, we can produce a lot of structures inside a signature. but, they do not all have an antecedent in the abstract signature. indeed, when parsing a structure of an object signature of an acg, if it has an antecedent, it will be found (for parsing is decidable, see above). if it does not have one, then nothing happens. that means that, when applying transduction between two object signa-tures (or two representation levels, in the case of our implementation), if one structure should not have a correspondance in the next object signature, it will not have one: no new structure will be produced. ## overview of our implementation in order to represent the mtm, at least from semr to ssyntr, we implemented signatures and lexicons (see figure 5 ) and experimented our encoding with acgtk @xcite , a piece of software allowing for defining grammars and using the associated parsing and interpretation operation. nevertheless, for simplification purpose we did not implement the other substructures than the main one for each representation level, like the communicatives structures (see section 2). this implementation uses transduction (see figure 4 ) which is the heart of our implementation, for it is used to perform all transformations (see figure 5 , where we can guess the use of transduction). we can see that the third area (see figure 5 ) looks like a detour. that is due to the fact that, on one hand, mtt does not modify a structure when going from a representation level to another (which transduction also does), and, on the other hand, during some steps of the generation process, we want to keep the former structures as well as the newly obtained ones. that is the case of the deep-syntactic paraphrasing, for we want to keep all possible dependency trees that would lead to a sentence, so all the possible paraphrases. in other words, mtt makes other structures appear inside of one representation level, and we want them all to reach the next representation level, not only the last one. for this is not the goal of transduction, deep-syntactic paraphrasing is hard to perform in the current state of our implementation, and is performed by this third area, although it is problematic, as we will explain later. our implementation encodes the different levels of representation of a mtm (see figure 5 ): σ sem corresponds to semr, σ dsynt tree to dsyntr, and σ ssynt tree to ssyntr. as we can see, we use the transduction between: it was tested on a sample of example sentences. this sample is short, but covers many lexical phenomena, like collocations, the use and realization of lfs, semantic or syntactic equivalences, as well as obligatory arguments optionally expressible. as stated at the end of section 3, inside a representation level (or a signature), the structures that should not have a correspondance in the next representation level (because they are incorrect for example) do not have one: they will not find an antecedent by reversing the lexicon during transduction ( the code and the examples are available at https://inria.hal.science/hal-04104453 . on top of that, this implementation also deals with adverbial groups, that have a specific treatment inspired by the work on tag of @xcite . their treatment is indeed not the same depending on the signature we look at. in semr, i.e., in λ(σ semantic ), the arc between the adverbial group and the verb it modifies is directed toward the verb, while in dsyntr, so λ(σ dsynt tree ), it is directed toward the adverbial group. this is because adverbial groups are modifiers @xcite . we used the same approach as @xcite for the dependency inversions between derived trees and derivation trees in tag (see figure 6). the manipulation of obligatory arguments of a semr that are optionally expressible is also possible, and was inspired by @xcite . it is done thanks to some constants in σ deep syntactic . these last two points will not be detailed here, but you can find further detail in @xcite . nevertheless, the section 5 will explain the last step of the paraphrase illustrated on figure 2 , that is the realization of lfs. it is a good example to show the different kinds of possible equivalence inside an acg, as well as how lfs are used in this modeling. expression linked to the semantic graph "charlie kills taylor intentionally" "john is calm" semr (σsemantic) 1 1 dsyntr before deep-syntactic paraphrasing (σ dsynt-tree ) 2 1 of which will be accepted at the next stage 2 1 dsyntr after deep-syntactic paraphrasing (σ dsynt-tree ) 2 6 foot_0 of which will be accepted at the next stage 2 3 dsyntr after fls realization (σ dsynt-0-fl ) 2 5 of which will be accepted at the next stage 2 3 ssyntr (σssynt-tree) 2 3 ## example this section gives a detailed example on how the transduction works and how we realize lfs in our implementation. we consider here the signatures σ dsynt tree (see we consider the example of the paraphrase between expressions (6a) and (6b). we may remember that anti(calm)={upset, restless}, so we also have anti(upset) = @xmath2, but we are interested in calm here). after the deep-syntactic paraphrasing of sentence (6a), before realizing lfs, we have a dependency tree such as figure 3a . but, (6a) is a paraphrase of (6b), and so is (6c) (illustrated in figure3b). they both have the same paraphrase (6b), so these two sentences (6a) and ( 6c ) are paraphrases of each other themselves (depending on the context, but that point will be explained in the conclusion). therefore, we want for our implementation to allow this link, this equivalence between the expressions (6c) and (6a). thus, we want to obtain an equivalence such as (7): indeed, we may remember that γ dt au represents the dsynts of expression (6a), and γ dt ar the dsynts of expression (6c). because both expressions are paraphrases of (6b) and because two different images of a morphism cannot have the same antecedent, we will have to use transduction here. the equivalence between (6a) (or (6c)) and (6b) will take two steps, i.e., parsing and application. we want, if we write t for the transduction relation, our implementation to allow equations such as (8): hence, we want to use two acgs sharing the same abstract signature, to have the following equations ( 9 ), ( 10 ) and ( 11 ) (see figure 4 that illustrates it) in order to have equations ( 8 ) and ( 7 ): tables 2, 6a, 6b and 6c define the constants of the signatures and lexicon we use in this section in order to do so. they are simplified and show only relevant information for this example. the constants such as lex i and a i (see section 3, we implement the realization of lfs with the transduction and the properties of λ-calculus, like β-reduction. indeed, one expression may have different representations in σ dsynt tree (see the different levels of equivalencies are the following (see figure 4 ): thus transduction between signatures σ dsynt tree and σ dsynt 0 fl allows to realize lfs, and to perform the third type of paraphrase (see section 2). ## conclusion and future work we have shown a possible implementation of a mtm with acgs. this implementation models the semr to ssyntr levels of mtm. even though this implementation uses only the main structures of the representation levels of mtt and not the other substructures (like the communicatives ones), when tested over a sample of example sentences, their ssynts are correctly obtained (see the implemented model enables the semantic paraphrase to take place, as well as the transitions between the representation levels thanks to transduction, the realization of lfs thanks to transduction too, the handling of obligatory semantic arguments optionally expressible, and the handling of adverbial groups. however, this implementation has some limitations. indeed, the deep-syntactic paraphrasing is not optimal. it is actually made possible by the detour of area 3 in figure 5 , but we need to manually iterate the process. we need to save the previous structures, perform the paraphrasing loop, and apply again the process for the newly generated structures until no new correct structure is obtained. transduction is showing some limitations here: this mechanism is not well suited for the deep-syntactic paraphrasing because rewritten structures still need to be processed as well as resulting structures. furthermore, we have not exploited all the possible types of paraphrasing @xcite in our implementation yet: we also want to continue in this direction to implement all of them. moreover, we want to include, for each level, other substructures, such as the communicative structures, to have more information about the theme, the rheme, and the speaker intentions, in order to have an implementation nearer to mtt than what it currently is.