The role of morphosyntactic similarity in generating related sentences (2024)

Abstract: In this paper we describe our work on Task~2: Creation of Educational Materials. We tried three approaches, but only the third approach yielded improvement over the baseline system. The first system was a fairly generic transformer model. The second system was our own implementation of the edit tree approach from the baseline system. Our final attempt was a version of the baseline system where if no transformation succeeded, we applied transformations from similar morphosyntactic relations. We describe all three here, but, in the end, we only submitted the third system.

## introduction the nature of the task was to transform one sentence into another in three languages based on a specification of the morphosyntactic differences between the input and output sentences @xcite . the three languages are bribri, guarani, and maya. we give sample data in glosses or translations were not provided. in addition, we do not know what morphosyntactic features might be appropriate for the input sentence. we were provided with definitions for the different tags. in the first example in we tried three different approaches: a simple transformer, our own implementation of edit trees in terms of a single regular transduction, and using related morphosyntactic tags when possible. in the following sections, we describe these three attempts. ## transformer the first model we tried was a simple transformer @xcite . 1 we first concatenate the input and morphological tags to serve as input. these are fed into the encoder which first creates embeddings, applies drop-out, and feeds these to a gru layer @xcite . the output and hidden weights are then fed to an attention-based decoder with two layers of grus and a simple linear layer. attention was bahdanau @xcite . batch size varied, but was typically around 32. the dimensions for all hidden layers was either 512 or 1024 for different runs. dropout for the encoder was set at 0.1. loss was negative log likelihood and the adam optimizer was used. we tried a variety of different configurations, but best performance was at 600 epochs with 512 hidden nodes at all layers. see the data are extremely limited and this surely impaired performance. our sense is that simply concatenating the input and morphosyntactic tags was also not the best choice. ## edit trees as a single transduction the baseline system for the task is based on the notion of edit trees. the basic idea is to build a tree representation of changes that the input must undergo to be converted to the output @xcite . chrupała gives the edit tree in figure 1 for the polish word pair najtrudniejszy 'hardest' and trudny 'hard'. the basic logic is that we identify the largest shared span, in this case characters 3 through 6. there is then a function split which maps from a string and indices to three strings (σ * × n × n) → (σ * × σ * × σ * ), taking a string w 1...n and indices i and j and returning the triple an edit tree is then either a replace node with two strings or it is a split node with two indices and two daughter nodes that are themselves edit trees. an edit tree is built with the function et which is defined as follows with respect to strings w and w ′ . if there is no lcs span, then the tree is simply replace(w, w ′ ). otherwise it is defined as: in our implementation, these operations are mapped to a single regular transduction with backreferences. for convenience, we use python for this. first, split nodes are represented as a list of three elements: the left daughter, the two indices, and the right daughter. replace nodes are represented as a pair of strings. the tree in figure 1 would be represented as: [(naj , ϵ), (3, 6), [(iejsz , ϵ), (5, 0), (ϵ, ϵ)]] (2) we traverse the tree from left to right converting each node into a pair of strings. all string pairs map to themselves. pairs of indices i, j are translated into maps from .{1,k}, where @xmath0, to the next available backreference. in the case of the tree in figure 1 , we have the translation naj.{1,3}iejsz.{1,1} mapping to \1\2. this mapping is executed in our code using the python re.sub function. this approach approximates the baseline system, but does not perform as well. see ## morphosyntactic similarity our final model, and the one we submitted, was an addition to the baseline system. the baseline system records the edit trees that are associated with specific morphosyntactic tag combinations along with the relative frequency of each tree. at inference stage, one selects the edit trees associated with the tags for the test item, sorts these by how frequently they're used in training, and try them one by one starting from the most frequent. if a rule succeeds, the output of that rule is returned. if no rule succeeds or the specific tag combination is not in the training data, then the input is returned as the output. our adaptation here was to add additional options to the list. if the procedure above produced no distinct output form, we then applied additional rules. these additional rules were generated from the full list of rules, sorted by how similar the tag sequence is to the test item tag sequence. if the above procedure results in no change, we then turn to these rules, going through them one by one. this procedure is terminated the same way as the baseline system: when a rule produces a change, that is the final output and further rules are not considered. if no rule produces a distinct output, the input itself is returned. this change results in a modest overall improvement in the baseline as seen in ## conclusion to summarize, we tried three different techniques: transformer, edit trees as transductions, and exploiting morphosyntactic similarity in selecting edit trees. as implemented, the transformer performed the worst. systems built on edit operations seem to perform much better in these character-to-character mapping domains, so this is not really a surprise. translating edit trees into regular transductions did not reach the level of the baseline, but is not an unreasonable approach to pursue further. edit operations are clearly useful. the question is what is the scope of those operations. are they separate operations as in edit trees as originally developed or can some amalgam of those operations be more successful. finally, using morphosyntactic similarity is successful and this is thus clearly an approach worth pursuing further. one very obvious way to go further is to build a model of the morphosyntactic structure of the input. we do not know what the words mean, but perhaps we can get some mileage toward identifying parts of speech from the meanings of the tags. with this in hand, we could exploit that part of speech information in our edit trees. this last approach is purely speculative, but it seems like a fairly obvious way to go (in hindsight!) given the nature of the task.