Discourse Context Predictability Effects in H indi Word Order (2022)

Abstract: We test the hypothesis that discourse predictability influences Hindi syntactic choice. While prior work has shown that a number of factors (e.g., information status, dependency length, and syntactic surprisal) influence Hindi word order preferences, the role of discourse predictability is underexplored in the literature. Inspired by prior work on syntactic priming, we investigate how the words and syntactic structures in a sentence influence the word order of the following sentences. Specifically, we extract sentences from the Hindi-Urdu Treebank corpus (HUTB), permute the preverbal constituents of those sentences, and build a classifier to predict which sentences actually occurred in the corpus against artificially generated distractors. The classifier uses a number of discourse-based features and cognitive features to make its predictions, including dependency length, surprisal, and information status. We find that information status and LSTM-based discourse predictability influence word order choices, especially for non-canonical object-fronted orders. We conclude by situating our results within the broader syntactic priming literature.

## introduction grammars of natural languages have evolved over time to factor in cognitive pressures related to production @xcite and comprehension @xcite , learnability @xcite and communicative efficiency @xcite . in this work, we test the hypothesis that maximization of discourse predictability (quantified using lexical repetition surprisal and adaptive lstm surprisal) is a significant predictor of hindi syntactic choice, when controlling for information status, dependency length, and surprisal measures estimated from n-gram, lstm and incremental constituency parsing models. our hypothesis is inspired by a solid body of evidence from studies based on dependency treebanks of typologically diverse languages which show that grammars of languages tend to order words by minimizing dependency length @xcite and maximizing their trigram predictability @xcite . parallel to this line of work on sentence-level word order, another strand of work has focused on discourse-level estimates of entropy starting from the constant entropy rate hypothesis (cer; @xcite . to overcome the major difficulty of estimating sentence probabilities conditioned on the previous discourse context, @xcite approximated discourse-level entropy using lexical cues from the previous context. in contrast, we leverage modern computational psycholinguistic neural techniques to obtain word and sentencelevel estimates of inter-sentential discourse predictability and study the impact of these measures on hindi word order choices. we conclude that discourse-level priming influences hindi word order decisions and interpret our findings in the light of the factors outlined by @xcite . hindi (indo-aryan language; indo-european language family) has a rich case-marking system and flexible word order, though it mainly follows sov word order @xcite as exemplified below. ## data and models our dataset comprises 1996 reference sentences containing well-defined subject and object constituents from the hutb foot_0 corpus of dependency trees @xcite . the hutb corpus, which belongs to the newswire domain and contains written text in a natural discourse context, is a human-annotated, multi-representational, and multi-layered treebank. the dependency trees here assumes panini's grammatical model where each sentence is represented as a series of modifiermodified elements @xcite . each tree in the hutb corpus denotes words in the sentence with nodes such that head words (modified) are linked to the dependent words (modifier) via labelled links denoting the grammatical relationship between word pairs. for each reference sentence in the hutb corpus, we created artificial variants by permuting the preverbal constituents whose heads were linked to the root node in the dependency tree. inspired by grammar rules proposed in the nlg literature (rajkumar and white, 2014), ungrammatical variants were automatically filtered out by detecting dependency relation sequences not attested in the original hutb corpus. after filtering, we had 72833 variant sentences for our classification task. figure 1 in appendix a displays the dependency tree for example sentence 1a and explains our variant generation procedure in more detail. to determine whether the original word order (i.e. the reference sentence) is preferred to the permuted word orders (i.e. the variant sentences), we conducted a targeted human evaluation via forcedchoice rating task and collected sentence judgments from 12 hindi native speakers for 167 randomly selected reference-variant pairs in our data set. participants were first shown the preceding sentence, and then they were asked to select the best continuation between either the reference or the variant. we found that 89.92% of the reference sentences which originally appeared in the hutb corpus were also preferred by native speakers compared to the artificially generated grammatical variants expressing similar meaning (further details are provided in appendix g). therefore, in our analyses we treat the hutb reference sentences as human-preferred gold orderings compared with other possible automatically-generated constituent orderings. ## experiments and results we tested the hypothesis that discourse predictability (estimated from adaptive lstm and lexical repetition surprisal) influences constituent ordering in hindi over other baseline cognitive controls, including dependency length, information status and trigram and non-adaptive surprisal. the adaptive lstm surprisal had a high correlation with all other surprisal features and a low correlation with ## variance inflation factor in this section, we evaluate our regression models for multicollinearity in terms of variance inflation factor (vif) score. as figure 2 in appendix c denotes, the adaptive lstm surprisal measure has a high correlation with all other surprisal predictors, which raises some suspicion that estimates of effects of the variables in our regression model might be unreliable. ## discussion our main findings suggest that in written hindi, people choose word orders that maximize discourse predictability. the actual psychological mechanisms are conceivably lexical and structural priming. our results indicate that lexical priming is most influential in canonical sentence contexts, but syntactic priming does influence ordering preferences in non-canonical contexts. below, we discuss the implications of our findings in terms of the 4 factors affecting syntactic priming discussed in detail by @xcite : inverse frequency interaction, decay, lexical boost, and cumulativity. the io-fronted construction is very rare (0.76% of our data) compared to do-fronted non-canonical sentences (1% of our data) in the hutb corpus of 13274 sentences. we find strong priming effects in io-fronted constructions but weak priming in the case of do-fronted constructions, providing evidence for an inverse frequency interaction @xcite . our finding that priming is not aided by longterm contexts indicates a decay effect in priming, which supports the residual activation (short-lived) hypothesis of priming in comprehension @xcite . nevertheless, there has been evidence for implicit learning effects in comprehension as well @xcite . more recently, @xcite using a similar setup as our current work argued for the existence of both the accounts viz., residual activation and implicit learning, and demonstrated the role of dual mechanism priming effects @xcite in hindi word order. previous work suggests that lexical overlap between prime and target sentences enhances syntactic priming @xcite . the repeated lexical items become cues during sentence planning and bias the speaker to produce similar structures that those repeated lex-ical items tend to occur in. overall, we find that lexical repetition influences hindi syntactic choice; however, syntactic priming is observed over and above lexical repetition in non-canonical constructions. it's interesting to note that comparable results have also been reported for english dialogue corpora @xcite . we plan to conduct a systematic investigation on hindi spoken data as a part of future work. finally, with regards to the cumulativity of priming, @xcite showed in their corpus study of production of passives and thatinsertion/omission that the effect of priming increases with the number of primes preceding it. our work does not investigate this specifically, and more controlled experiments would be required. the success of lstm-based surprisal estimates over and above dependency length can also be interpreted in light of @xcite point about the limitation of surprisal theory with respect to word order. futrell modified surprisal theory by positing that the per-word processing difficulty is proportional to its surprisal given a lossy memory representation of the preceding context. moreover, @xcite proposed the information locality hypothesis (ilh) which states that all pairs of words with high mutual information (not merely syntactically related words) tend to be located close to one another. the long window offered by lstm surprisal thus models relationships between words at varying distances (over and above conventional trigram models). the success of these surprisal estimates for the task of reference sentence prediction provides some preliminary evidence for ilh in the case of word order. future work needs to tease apart priming effects of both vanilla lstm and adaptive lstm surprisal in the light of recent works. in this work, sentences are treated as independent while estimating their surprisal using vanilla lstm lm, so there is no way vanilla lstm can exhibit syntactic priming given the independent sentences. however, @xcite demonstrated that bert exhibits "priming effect". the bert lm was able to predict a word with greater probability when the context included a related word than an unrelated word. however, the effect decreased as the amount of information provided by the context increases. in other words, the related prime under high contex-tual constraint started acting as distractor-actively demoting the target word in the probability distribution; thus exhibiting "mispriming effect" @xcite . this could be due to stylistic avoidance of repeated structures/words in the adjacent sentences. future work also needs to investigate whether word-order preferences can be jointly optimized using multiple factors @xcite . in particular, the relationship between the drive to minimize surprisal (as found in this work) and the tendency to make information profiles uniform @xcite needs to be explored more thoroughly in the light of recent findings @xcite . overall, our results demonstrate that hindi word order preferences are influenced by discourse predictability maximization considerations. the actual mechanisms of discourse effects are plausibly lexical and syntactic priming. ## limitations the 'levels' problem discussed in @xcite which posits 2 levels of linguistic optimisation is germane while evaluating our work. our results are restricted to the level of syntactic choices made by individual speakers or users of a given language over a lifetime (and not at the level of entire grammars and evolutionary timescales). our experiments conducted on written text need to be performed on spoken data in order to make claims about priming in language production. generate the artificial variants (3b and 3c). the ungrammatical variants were automatically filtered out using dependency relation sequences (denoting grammar rules) attested in the gold standard corpus of hutb trees. in the dependency tree 1a, "k4-k1", "k7t-k1", "k3-k7t", and "pof-k3" are dependency relation sequences. in cases where the total number of variants exceeded 100 (a random cutoff), 8 we chose 99 non-reference variants randomly along with the reference sentence.