Putting Natural in Natural Language Processing (2023)

Abstract: Human language is firstly spoken and only secondarily written. Text, however, is a very convenient and efficient representation of language, and modern civilization has made it ubiquitous. Thus the field of NLP has overwhelmingly focused on processing written rather than spoken language. Work on spoken language, on the other hand, has been siloed off within the largely separate speech processing community which has been inordinately preoccupied with transcribing speech into text. Recent advances in deep learning have led to a fortuitous convergence in methods between speech processing and mainstream NLP. Arguably, the time is ripe for a unification of these two fields, and for starting to take spoken language seriously as the primary mode of human communication. Truly natural language processing could lead to better integration with the rest of language science and could lead to systems which are more data-efficient and more human-like, and which can communicate beyond the textual modality.

## introduction the acl 2023 theme track urges the community to check the reality of the progress in nlp. this position paper adopts an expansive interpretation of this question. it is definitely worth inquiring into the apparent advances of current nlp in their own terms. here, however, i question these terms and argue that our field has focused on only a limited subset of human language which happens to be convenient to work with, and thus misses major aspects of human communication. ## nlp is written language processing to anyone with experience reading, reviewing and publishing papers in nlp conferences and journals (such the acl conferences and tacl) it is evident that the field is very strongly focused on processing written language. while this is evident to practitioners, it is also largely tacit and implicit. ## spoken language is richer while the primacy of the spoken modality as means of communication is the consensus view in linguistics, section 2.1 identifies unstated assumptions among nlp practitioners which amount to the opposite view. here i outline why these assumptions contradicting the scientific view are not only incorrect but also detrimental to progress on understanding and processing real human language. ## unifying speech processing and nlp as evident from the examples highlighted above, spoken language is in some ways quite different from written language and presents a distinct set of challenges and potentials. in order to understand how much progress the fields of speech and nlp are making in understanding and implementing human language, we need to take speech seriously qua language, not just a cumbersome modality, and measure our progress accordingly. ## limitations the main limitation of this paper is the one applying to any opinion piece: it is subjective and personal, as the views of the authors are inherently limited by their expertise and experience. more specifically, this paper argues for an increased interaction between the speech and nlp communities, but the author is more strongly embedded in the latter, and thus addresses this audience primarily. additionally, the short paper format imposes significant constraints on the amount of nuance, detail and discussion of relevant literature, and thus readers may find some of the claims to be less strongly supported and less hedged than would be ideal, or proper in a longer treatment of this topic.