An Inclusive Notion of Text (2023)

Abstract: Natural language processing (NLP) researchers develop models of grammar, meaning and communication based on written text. Due to task and data differences, what is considered text can vary substantially across studies. A conceptual framework for systematically capturing these differences is lacking. We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. Towards that goal, we propose common terminology to discuss the production and transformation of textual data, and introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling. We apply this taxonomy to survey existing work that extends the notion of text beyond the conservative language-centered view. We outline key desiderata and challenges of the emerging inclusive approach to text in NLP, and suggest community-level reporting as a crucial next step to consolidate the discussion.

## terminology textual data available to nlp is a result of multiple processes that determine the composition and properties of texts. to support our discussion, we outline the data journey a typical text undergoes, and introduce common terminology. figure 2 illustrates our proposed model, and the remainder of this section provides more details. text production. every text has been produced by a human or an algorithm with a certain communicative purpose. raw text is rarely exchanged; to avoid ambiguity, we use the term document for a unit of information exchange 1 . documents consist of text along with additional structural and multimodal elements, serialized in a certain format and accompanied by metadata. in our definition, tex-tual documents cover a broad spectrum ranging from blog posts, wikipedia articles and tweets to dialogue turns and search queries. a few widely used formats are plain text, markdown, pdf. document space. all textual documents ever produced make up the abstract document space. document space incorporates both persistent textual documents that are stored (e.g. wikipedia articles), and transient textual documents that only exist temporarily (e.g. search queries). despite the apparent abundance of textual documents on the web, a large (if not major) part of the document space is not openly available, or is protected from research use by the copyright, privacy and technical constraints. sampling and sources since capturing the entire document space is not feasible, a sample from the subspace of interest is used. document space can be segmented in a variety of ways, including language, domain or variety (plank, 2016), creation time, etc. one common way to sample textual documents is based on source: documents from the same source often share key characteristics like language variety, text production environment, format and licensing. some widely used data sources in nlp are wikipedia, arxiv etc. @xcite . ## prior efforts our proposal draws inspiration from recent efforts in documenting other common properties of machine learning and nlp artifacts. model cards @xcite capture core information about machine learning models including technical characteristics, intended and out-of-scope use and preprocessing details. data sheets @xcite focus on dataset composition, details of the data collection process, preprocessing, distribution and maintenance. in nlp, data statements (bender and friedman, 2018) focus on bias mitigation, detailing key aspects of nlp artifact production such as curation strategy, language variety, demographics of speakers and annotators, speech situation, topic and genre. @xcite propose a formalised checklist documenting risks related to copyright, bias, privacy and confidentiality. formal proposals are mirrored by community efforts on data repositories like huggingface datasets @xcite ; editorial guidelines foot_1 encourage the authors to report key parameters of nlp artifacts. related metadata collection initiatives propose schemata for capturing core information about language resources like language, type, license and provenance @xcite . while existing approaches to nlp artifact documentation cover a lot of ground, the requirements for documenting the assumed notion of text remain under-specified. our work is thus complementary to the prior efforts. our reporting schema (section 6) can be seen as specification of the speech situation and text characteristics sections of the data statements @xcite , and our taxonomy incorporates some previously proposed documentation dimensions like text creation environment @xcite and granularity @xcite . unlike most prior approaches, we deem it desirable to document the assumed notion of text at each step of the nlp data journey, including text production tools, document space samples, as well as nlp models and datasets, with a special focus on widely reused reference corpora and pre-trained language models. 4 taxonomy of text extensions ## reporting an inclusive approach to text is an emerging trend in nlp that demands systematic study. while preparing this work, it became evident that the lack of systematic reporting limits the meta-analysis of text use in nlp. in line with related documentation efforts, here we propose a simple, semi-structured mechanism for reporting text use. in the short term, such reporting would make it easier to gauge the capabilities of data sources and nlp artifacts, increase community awareness on what aspects of text are represented and used, and allow aggregation of text use information from different studies. in the long term, it would help the community develop standards for applying the inclusive approach to text and formally documenting text use, and allow informed development of general data models and formats @xcite to facilitate interoperability between nlp artifacts that adopt an inclusive approach to text. ## conclusion text plays the central role in nlp as a discipline. but what belongs to text? the rise in applications of nlp to non-linguistic tasks motivates an inclusive approach to text beyond written language. yet, the progress so far has been limited to isolated research efforts. as nlp ventures into new application areas and tackles new tasks, we deem it crucial to document the notion of text assumed by data sources and nlp artifacts. to this end, we have proposed common terminology and a two-tier taxonomy of inclusive approaches to text, complemented by a widely applicable reporting schema. we hope that our contributions and discussion help the community systematically approach the change of nlp scope towards more accurate modeling of text-based communication and interaction.