Detecting Age-Related Linguistic Patterns in Dialogue: Toward Adaptive Conversational Systems
Authors: Lennert Jansen, Arabella Sinclair, Margot J. van der Goot, Raquel Fernández, Sandro Pezzelle (University of Amsterdam, 2021)
Topic: Chatbots and Dialogue Systems

Abstract
This work explores an important dimension of variation in the language used by dialogue participants: their age. While previous work showed differences at various linguistic levels between age groups when experimenting with written discourse data (e.g., blog posts), previous work on dialogue has largely been limited to acoustic information related to voice and prosody. Detecting fine-grained linguistic properties of human dialogues is of crucial importance for developing AI-based conversational systems which are able to adapt to their human interlocutors. We therefore investigate whether, and to what extent, current text-based NLP models can detect such linguistic differences, and what the features driving their predictions are. We show that models achieve a fairly good performance on age-group prediction, though the task appears to be more challenging compared to discourse. Through in-depth analysis of the best models' errors and the most predictive cues, we show that, in dialogue, differences among age groups mostly concern stylistic and lexical choices. We believe these findings can inform future work on developing controlled generation models for adaptive conversational systems.

Introduction
Research on developing conversational agents has experienced impressive progress, particularly in recent years <cit.>. However, artificial systems that can tune their language to that of a particular individual or group of users continue to pose more of a challenge. Recent examples of this line of research include adaptation at style level <cit.>, persona-specific traits <cit.>, or other traits such as sentiment <cit.>. Personalised interaction is of crucial importance to obtain systems that can be trusted by users and perceived as natural <cit.>, but most of all to be accessible to varying user profiles, rather than targeted at one particular user group <cit.>. We conjecture that stylistic and lexical differences between age groups can be detected. Here, we experiment at the level of the utterance. In this work, we focus on one particular aspect that may influence conversational agent success: user age profile. We investigate whether the linguistic behaviour of conversational participants differs across age groups using state-of-the-art NLP models on purely textual data, without considering vocal cues. We aim to detect age from characteristics of language use  and adapt to this signal, rather than work from ground-truth metadata about user demographics. This is in the interest of preserving privacy, and from the perspective that while age and language use may have a relationship, this will not be linear <cit.> and there are individual differences.  Previous work on age detection in dialogue has focused on speech features, which are known to systematically vary across age groups. For example, Wolters et al. wolters2009age learn logistic regression age classifiers from a small dialogue dataset  using different acoustic cues supplemented with a small set of hand-crafted lexical features, while Li et al. li2013automatic develop SVM classifiers using acoustic and prosodic features extracted from scripted utterances spoken by participants interacting with an artificial system. In contrast to this line of work, we investigate whether different age groups can be detected from textual linguistic information rather than voice-related cues. We explore whether, and to what extent, various state-of-the-art NLP models are able to capture such differences in dialogue data as a preliminary step to age-group adaptation by conversational agents.  We build on the work of Schler et al. schler2006effects, who focus on age detection in written discourse using a corpus of blog posts. The authors learn a Multi-Class Real Winnow classifier leveraging a set of pre-determined style- and content-based features, including part-of-speech categories, function words, and the 1000 unigrams with the highest information gain in the training set. They find that content features (lexical unigrams) yield higher accuracy (74%) than style features (72%), while their best results (76.2%) are obtained with their combination. We extend this investigation in several key ways: (1) we leverage state-of-the-art NLP models that allow us to learn representations end-to-end, without the need to specify concrete features in advance; (2) we apply this approach to dialogue data, using a large-scale dataset of transcribed, spontaneous open-domain dialogues, and also use this approach to replicate the experiments of Schler et al. schler2006effects on disccourse; (3) we show that text-based models can indeed detect age-related differences, even in the case of very sparse signal at the level of dialogue utterances; and finally (4) we carry out an in-depth analysis of the models' predictions to gain insight on which elements of language use are most informative.[Code and data available at: <https://github.com/lennertjansen/detecting-age-in-dialogue>]  Our work can be considered a first step toward the modeling of age-related linguistic adaptation by AI conversational systems. In particular, our results can inform future work on controlled text generation for dialogue agents <cit.>.

Conclusion
We investigated whether, and to what extent, NLP models can detect age-related linguistic features in dialogue data. We showed that, in line with what we observed for discourse, state-of-the-art models are capable of doing so with a reasonable accuracy, in particular when the dialogue fragment is long enough to contain discriminative signal. At the same time,we found thatmuch simplermodels based on n-grams achieve comparable performance, which suggests that, in dialogue, `local' features can be indicative of the language of speakers from different age groups. We showed this to be the case, with both lexical and stylistic cues being informative to thesemodels in this task.While weperformed the classification task at the level ofsingle dialogue utterances, future work maytake into accountlarger dialogue fragments, such as the entire dialogue or a fixed number of turns.This wouldmake the setup more comparable to discourse, but wouldrequire making experimental choices and dealing with extra computationalchallenges.Moreover,it could be testedwhether the language used by a speaker is equally discriminative when talking to a same-age(this work)or a different-age interlocutor.Finally, we believe our findings could inform future work on developing adaptive conversational systems. Since consistent language style differences were found between age groups (for example, at the level of exclamatives and acknowledgments), systems whose language generation capabilities aim to be consistent with a given age group should therefore reproduce these patterns. This could be achieved, for example, by embedding one or more discriminative modules thatcontrolthe generation of a system's output, which could lead to better, more natural interactions between human speakers and a conversational system.