Lexical variation in E nglish language podcasts, editorial media, and social media (2022)

Abstract: The study presented in this paper demonstrates how transcribed podcast material differs with respect to lexical content from other collections of English language data: editorial text, social media, both long form and microblogs, dialogue from movie scripts, and transcribed phone conversations. Most of the recorded differences are as might be expected, reflecting known or assumed difference between spoken and written language, between dialogue and soliloquy, and between scripted formal and unscripted informal language use. Most notably, podcast material, compared to the hitherto typical training sets from editorial media, is characterised by being in the present tense, and with a much higher incidence of pronouns, interjections, and negations. These characteristics are, unsurprisingly, largely shared with social media texts. Where podcast material differs from social media material is in its attitudinal content, with many more amplifiers and much less negative attitude than in blog texts. This variation, besides being of philological interest, has ramifications for computational work. Information access for material which is not primarily topical should be designed to be sensitive to such variation that defines the data set itself and discriminates items within it. In general, training sets for language models are a non-trivial parameter which are likely to show effects both expected and unexpected when applied to data from other sources and the characteristics and provenance of data used to train a model should be listed on the label as a minimal form of downstream consumer protection.

## genres and podcast transcripts the way human language is used varies across channels and styles, and we have for the longest while made a clear distinction between spoken and wri en language as two major distinctive modes of communication @xcite @xcite . the differences between writing and reading can to a large extent be related to situational differences: where speech has been used in transient situations in which interlocutors are present, writing has typically been used in asynchronous communication with participants at a remove from each other. this distinction has through the introduction of communication technologies become less and less clear-cut. wri en language is used for momentary and fleeting conversations with li le planning or editorial oversight; spoken language material is created, published and distributed in ways which are more formal and more permanent and archival than before. podcasts are a new medium and a new format for spoken language. the styles of language use in podcasts are as yet unformed and have not yet coalesced into stable functional and generally accepted genres: podcast material will require us to recalibrate many of the assumptions we make about how language is used. recently, a collection of over 100,000 podcast episodes, including automatically generated transcripts, has been released for the purposes of retrieval and summarisation experimentation. the companion paper released with the podcast material set gives some indicative differences between the transcripts and written language as represented by the brown corpus @xcite ) and shows i.a. that the frequency of amplifiers and personal pronouns is greater than in the various genres represented in the brown corpus @xcite . this paper demonstrates how some such differences across text collections of different types are indicative of genre differences, some of which can be expected to depend on how spoken genres continue to evolve with changing technology and evolving situations of usage. this examines differences in anchoring, subjective language, and discourse handling, which can all be expected to be dimensions in which podcast language will differ from wri en genres. podcasts are a rapidly evolving medium. the variation and volatility is great and we can expect that only a few years from now there will be new formats of language use not represented in the present collection. these measurements are intended to inspire the systematic exploration of such differences as they occur, to make possible documentation of current and future changes in the medium, to make explicit differences that may have effects on the applicability of language models trained on one type of material on another, and to ensure that application to classification, retrieval, or large scale extraction of information is informed and sensitive to those systematic differences that might impact results. ## data overview seven data sets were used for these experiments. these data sets are of varying age and collected with various methods, but have all been used in research and benchmarking projects and are selected by virtue of being accessible for experimentation and further study. the representativeness of the corpora may vary: movie scripts change over time as the cra of writing and acting evolves; conventions in phone conversations change as new technologies cater to new use cases; social media platforms, with various conventions and various technological affordances, go in and out of fashion; editorial media shi their focus and their offerings according to the shi ing constitution and preferences of their audiences. the editorial media data set is the one most clearly governed by conventions and constraints imposed by audience expectations for the genre and is likely to be the data set with least change over time. these changes are all likely to affect the stylistic statistics on reported below in various ways; the differences found between genres are robust enough to majorise the within genre differences over time. ## dimensions of variation the measures examined in this study focus on readily inspectable aspects of language use where spoken language and informal channels traditionally are assumed to show difference to wri en formal genres. spoken language due to its immediacy and synchronous nature frequently has more overt markers for interpersonal functions, and utilises different textual functions to organise the discourse. since wri en genres more frequently are used for abstract and complex topical ma er, it is to be expected that those ideational functions that concern argumentation and logical structure are rendered differently. biber and colleagues, in their studies on register variation across several languages @xcite , posit a number of variational dimensions using factorial analyses and then formulate a low dimensional space of functional bases in which they position the genre samples such as lectures, faceto-face conversations, broadcasts, private le ers, academic prose, offical documents, and many more. this study uses a subset of the variables examined by biber and colleagues (the variables used by biber are variously accessible for automated analysis). the addition of podcast material to the data used by biber are likely to extend the variational dimensions posited by his original study, since podcast material cuts across many of the suggested dimensions such as "involved vs informational" which separates e.g. speeches from e.g. academic prose; "narrative vs non-narrative", which separates fiction text from e.g. face-to-face interaction; "textual vs situational reference", which separates e.g. phone conversations from official documents and so forth. podcasts incorporate material with the situatedness of personal conversations to the abstraction of formal lectures, and material with the immediacy and interactive online planning of live dialogue to the editorially oriented production qualities of broadcast news. we can expect that many of the variational dimensions are relevant for podcasts even as new conventions and new genres gradually develop. spoken unscripted language is characterised by explicit features related to the organisation of discourse which involve turn-taking, interruptions, dysfluencies, and repair. these are somewhat challenging to study with the given collections, especially as transcription o entimes removes and normalises much of the signal. notably, in the present collections, while the phone conversation transcripts render turn-taking in detail, the podcast transcriptions leave out overlapping speech. this study focusses on features of language use which is situated, where the participants are synchronously present during the communicative situation as opposed to communication where the author or speaker is separated from the audience, and personal and subjective, where the a itude and stance of the author or speaker is clearly expressed and modulated to capture the a ention and fit the reactions of the intended audience, in contrast to language framed to be formal and couched in objective terms and expressions, abstracted from the present situation. the surface features to be expected are more a itudinal and overtly subjective language, with intensifiers, first and second person pronouns, more present tense and narratives, more questions and affirmations than in scripted and planned language use. ## a itude and affect in language subjective language is of interest for many reasons, but not least for its potential applications in information retrieval and text categorisation. since the introduction of computational sentiment analysis as a research topic @xcite various efforts to extend or typologise the field have been explored, @xcite @xcite and many mostly lexical approaches were implemented for commercial application. now, with computational methods that allow full scope over an entire u erance without relying on single items, some of the lexical approaches are less immediately impressive than before, but for reasons of transparency, many are still in use in practical applications and they correlate well with findings from non-lexical approaches. for the present experiments, a standard lexicon of polar items has been used to represent the manifold expressions of human emotion found in text @xcite , and the incidence of items from the lexicon are shown in the table gives counts both per word, i.e. how many of the tokens of the collection sample were polar evaluative lexical items (le half of the table), and per sentence, i.e. how many sentences of the 100 000 sample contained a polar evaluative lexical item (right half of the table). the results show that podcast transcripts have a noticeably higher incidence of positive polarity items and lower incidence of negative polarity items than written genres and that popular lectures exhibit much the same distribution. news stories exhibit more negative polarity than positive polarity items, which is likely to 2017 2019 before 2010 early 1990s 2017 have to do with editorial considerations: negative news drive reporting. this probably explains a similar imbalance for the microblog posts, which to a large extent are commentary to current news stories. the two more traditional spoken genres have much lower counts of both polarities, which may mean that the lexicon used here is not optimised for spoken material or that spoken language demonstrates polarity more o en in constructional items rather than purely lexical ones ("this put me off."). ## amplification amplifiers are linguistic items that serve to increase the perceived strength of an evaluative expression. they are typically constructed as adverbials, as shown in example (1) @xcite and in this study such items are used, and other amplifying constructions are le aside. amplifiers can be subcategorised in several ways, and here a three-way distinction is made. gradation amplifiers increase the intensity of a gradal expression: (very, immensely, substantially, fucking); affirmation amplifiers emphasise the commitment of the speaker to the sentiment (truly, really); and surprise amplifiers communicate that the qualities under consideration are unexpected or anomalous (amazingly, surprisingly, unusually). these distinctions are of course not independent of each other. the amplifiers used in this study are given in appendix a. the incidence of amplifiers in the seven collections are given in ## negation negation is a foundational semantic operator whose exact semantic function on the meaning of an u erance can be discussed and modelled at length (von @xcite . negation can affect an entire clause ("i didn't eat the cookies.") or more locally, a constituent of a clause ("i will eat no more cookies."). in english, clausal negation most o en is formed through the negative verbal affix "n't", which in wri en or more formal registers, or when emphasised, o en is rendered as the separate lexical item "not". local negation is formed through prefixing the negated component with "no" or "not", or by using more elaborate construction such as "neither … nor", "nobody", "none", or "never" @xcite ). negation has an obvious relation to polarity and antonymy which has motivated great interest in research on methods for the practical handling of negation in sentiment analysis and related experiments and applications @xcite @xcite @xcite . some examples of negation and its effect on polarity are given in example (2). in this study, negation is included as an example of an accessible semantic operator useful for modulation and modification of a itudinal expressions. the list of negations used in this study, compiled from irk et al. @xcite and @xcite is given in appendix b and the incidence of negations is given in ## interrogatives the incidence of interrogative u erances, defined as sentences that end with a "?", differs across the collections as shown in ## situatedness personal pronouns are used when the author or speaker and the audience have a shared understanding of the context they are in. first and second person pronouns are less prevalent in formal discourse and more prevalent in face-to-face conversation than in other situations; narrative discourse will show a higher proportion of third person pronouns than non-narrative discourse. the expected differences are found in the data, as shown in shared across all genres except the news material is the preponderance of present tense in comparison with past tense as shown in ## concluding observations this initial study demonstrates some clear differences in lexical content between transcribed podcast material and other collections of language data: editorial text, social media, both long form and microblogs, dialogue from movie scripts, transcribed phone conversations, and popular lectures. most of the recorded differences are as might be expected, reflecting known or assumed difference between spoken and wri en language, between dialogue and soliloquy, and between scripted formal and unscripted informal language use. most notably, podcast material, compared to the hitherto typical training sets from editorial media, is characterised by being in the present tense, and with a much higher incidence of pronouns and negations. these characteristics are, unsurprisingly, largely shared with social media texts. where podcast material differs from social media material is in its a itudinal content, with many more amplifiers and much less negative a itude than in blog texts. there is a solid base to explain these differences in the studies by biber referred to above, and in the more general notion of metafunctions of language which are utilised with various relative strength across communicative situations. it is to be expected that the results presented in this study will age rapidly with respect to their details: the podcast medium will evolve and new genres and stylistic conventions will emerge an coalesce in the near future as podcasts gain a broader audience, more creators, and further situations of use. the popular lec northern european journal of language technology tures, an offshoot from classical academic lectures, but with their form modified by new transmission channels and by influence from other staged presentations, shows one direction of development which is clearly related to podcasts; we should expect some podcasts to adhere to this genre, while others will be more like drama and scripted speech, and some continue to exhibit similarities to more unscripted and informal conversation. across all genres, the difference between he and she in their various forms was dramatic -this is something that may change over time. these observations have some direct ramifications for computational work. firstly, any useful approach to information access for material which is not primarily topical should be designed to be sensitive to such variation that defines the data set itself and discriminates items within it. more generally, training sets for language models are a non-trivial parameter which are likely to show effects both expected and unexpected when applied to data from other sources. the characteristics and provenance of data used to train a model should be listed on the label as a minimal form of downstream consumer protection. what these counts specifically demonstrate is that filtering the a data set through application of "stoplists" or other feature reduction methods or assessing the quality of language models using gold standards built on referential semantics based on nouns (cf. @xcite ) will reduce the richness of expression more in pronoun-rich and verbrich genres than in those with less pronouns and verbs. the variation demonstrated by the lexical tables given here is of obvious philological interest, casting light on how human communicative behaviour is modulated by the channel over which it proceeds. these reported statistics are but a scratch on the surface: more sophisticated and hypothesis-driven methods will be able to present more unified underlying variables and models with more explanatory power.