Text vs. Transcription: A Study of Differences Between the Writing and Speeches of U . S . Presidents (2024)

Abstract: Even after many years of research, answering the question of the differences between spoken and written text remains open. This paper aims to study syntactic features that can serve as distinguishing factors. To do so, we focus on the transcribed speeches and written books of United States presidents. We conducted two experiments to analyze high-level syntactic features. In the first experiment, we examine these features while controlling for the effect of sentence length. In the second experiment, we compare the high-level syntactic features with low-level ones. The results indicate that adding high-level syntactic features enhances model performance, particularly in longer sentences. Moreover, the importance of the prepositional phrases in a sentence increases with sentence length. We also find that these longer sentences with more prepositional phrases are more likely to appear in speeches than in written books by U.S. presidents.

## introduction scholars across various fields have sought to answer what makes writing different from speaking. the answers range from the notion that there is no fundamental difference to the belief that they are entirely distinct domains. these investigations lead to in-depth explorations with different approaches and perspectives, depending on the population or the system under study. for example, some scholars look for answers to support non-native speakers during the language acquisition process, while others attempt to measure the cognitive load through the sound or word production process. likewise, some try to enhance our ability to program machines and unlock new insight into the differences between spoken and written text. rajaei @xcite investigated the difference between speaking and writing, focusing on morphological, lexical, and syntactic features at both sentence and chunk levels. they showed the superiority of bert @xcite as well as the importance of sentence length, percentage of nouns, percentage of verbs, and depth of the parse tree. in this paper, we expand the corpus and focus on high-level syntactic features to examine their effectiveness in distinguishing the transcriptions of speeches and written books by united states presidents. we analyze linguistically inspired features instead of simply counting categories. throughout the paper, the text is analyzed in different sentence lengths, categorized as short, medium, and long. we use corenlp @xcite as a state-of-the-art to parse the sentences. @xcite provide a comparison of several nlp toolkits, including nltk, opennlp, and stanford corenlp on both formal and informal texts and conclude that depending on the task and text types, the toolkits perform differently. regarding text similarity measurement, wang and dong (2020) recommend using a combination of techniques and models for higher accuracy, concluding that no single method works best for all similarity measurement tasks in nlp. for this investigation, we use support vector machines (svm), decision tree (dt), random forest (rf), and bert as our machine-learning classification models to answer the following questions: rq1: how do syntactic features impact detection performance in sentences with different lengths? rq2: which syntactic features improve the model performance in distinguishing written sentences from transcribed spoken sentences? in the literature review, we will explore how other scholars have approached these questions. the methodology will be detailed in the procedure section. following that, we will present the re-sults and discussion, draw conclusions, outline the limitations, and finally offer future research. the artifacts of this work are published online. 1 ## related work in this section, we explore related findings in linguistics, cognitive science, and computer science. ## procedure in this section, we describe the dataset and the extracted features. ## results and discussion the results of our first experiment, shown in figure 1 , indicate that accuracy across all models is lower for short sentences compared to longer ones. this suggests two key points. first, the extracted syntactic features are more informative in longer sentences, and their rare occurrence in short sentences leads to lower performance across all models. second, despite having more short samples in the dataset, the selected features performed better on longer sentences. this implies that there is likely to be higher accuracy in a larger dataset containing more long sentences. it should also be noted that the similarity between the results for short sentences and the overall dataset is due to the large number of short sentences, which biases the models' performance. furthermore, the greater similarity between the results for medium and short sentences, compared to medium and long sentences, is due to the closer boundary numbers for short and medium sentences. although bert does not have explicit access to the linguistic features, we note that it performs better than any of the other models. as shown in looking at rq2, notably, as sentences become longer, the performance of our syntactic features improves, probably because longer sentences provide more information, which enables the models to more accurately distinguish between speech and writing. figure 3 shows the feature importance ranking of the merged set of features. note that high-level features like conjunction phrases (conjp) and imperatives show less influence. this is possibly due to the fact that these features rarely appear in sentences in our dataset. for instance, the conjp feature appears in only about 2 percent of all sentences. as shown in ## conclusions in this study, we analyze low-level and high-level syntactic features to identify the differences between the speeches and written books of presidents of the united states. we conducted two experiments to achieve these goals. in the first experiment, sentences were divided into three categories: short, medium, and long. we found that, despite having fewer samples for long increasing the sentence length also raised the importance ranking of prepositional phrases. furthermore, the most significant features identified are sentence length, verb percentages, noun percentages, and prepositional phrases. in the second experiment, we added a new set of syntactic features to morphological, lexical, and other syntactic features. the results showed that combining both groups of features improves model performance. furthermore, sentence length and prepositional phrases emerged as the two important features in distinguishing the textual data of u.s. presidents. based on our analysis, u.s. presidents are more likely to use prepositional phrases and longer sentences in their speeches than in their books.