Learning Through Transcription (2022)

Abstract: Transcribing speech for primarily oral, local languages is often a joint effort involving speakers and outsiders. It is commonly motivated by externally-defined scientific goals, alongside local motivations such as language acquisition and access to heritage materials. We explore the task of ‘learning through transcription’ through the design of a system for collaborative speech annotation. We have developed a prototype to support local and remote learner-speaker interactions in remote Aboriginal communities in northern Australia. We show that situated systems design for inclusive non-expert practice is a promising new direction for working with speakers of local languages.

## introduction speech transcription is typically motivated by the desire for lasting accessible records of language. however transcription can also be a method for linguistic inquiry and language acquisition @xcite . in this case it is a form of note-taking as the transcriber strives to make sense of what they hear. this practice is well established in documentary and descriptive linguistics, and it leads to detailed transcriptions that include metalinguistic detail. computational linguists rely upon these annotated datasets to train language models. non-specialist outsiders may find their alphabetic decoding skills useful for learning the local languages that are spoken in the places where they live and work, as evidenced by the number of learning resources that depend on having a written representation for these primarily oral languages. we propose computational support for an activity we call learning through transcription. the form we propose is that of a system that supports transcription as a series of learning interactions. the focus of the computation is shifted from automation to computer supported cooperative work. we describe a design and engineering effort to address this need in remote aboriginal communities in northern australia. here, local people speak one or more local languages, along with various degrees of proficiency in english. some non-indigenous australians seek competency in aboriginal languages to carry out cultural projects with local people. some locals need to develop literacy in one of the local languages to support knowledge work in art centres, ranger programs, health clinics, schools, tourism operations, and so on. accordingly, speech transcription is a practice that supports language acquisition and literacy development. we report on a system for iterative word-level transcription modelled on 'sparse transcription' @xcite . developed through a course of research-through-design (cf. rtd in @xcite , 'sparzan' is a vehicle for investigating methods for amplifying human effort in transcribing speech in primarily oral, local languages. we cover data models for learning through transcription, technologies and user interfaces for interactive transcription, and systems engineering. a topic foregrounded by the covid-19 pandemic is the tyranny of distance when outsiders seek to conduct fieldwork with people in remote linguistic communities @xcite . we investigate remote collaboration through a novel video messaging appliance which serves as a vector for learner-speaker interactions embedded in transcription work. this paper is organised as follows. in section 2 we describe the role of speech transcription for oral languages, including learning through transcription, interactive transcription, and situated systems design. in section 3 we describe the sparzan system, including the transcription client, the lingobox appliance, and an example application. in section 4 we reflect on the approach and draw lessons for future work. ## background many local languages, including endangered and indigenous languages, are purely oral, and there is no naturally-occurring context for deploying text technologies @xcite . oral languages are not just languages that lack a writing system; the existence of an oral culture unlimited by writing leads to an entirely different situation @xcite . local matters of concern include caring for the country, transmitting ecological knowledge to the next generation, and managing intercultural workplaces. this may mean that there is a need to record and transcribe ancestral knowledge about key places and practices, to compile vocabularies of local flora, fauna, and material culture, and for two-way language learning between the local vernacular and a language of wider communication. apparently simple tasks like accessing an archive of historical recordings become more complex when one considers that we lack a standardised orthography for a community-agreed reference dialect supported by robust speech recognition. thus, there is both a need for transcription, and a need to innovate when it comes to making transcriptions, through the design of novel processes and interfaces. ## project sparzan in this section we describe a system for collaborative computationally-assisted speech transcription. it is a synthesis of prior work in speech transcription methodology, interactive transcription workflows and remote interaction through tangible technology. the system's working title sparzan derives from sparse transcription, the fundamental transcription model we adopt here. two use cases are supported: collaborative lexicon building and individual language acquisition. in both cases, speech is transcribed by non-native speakers as a way to expand their individual and collective understanding, which is why we call this learning through transcription. the sparzan architecture is given in figure 1 , comprising a web application including the transcription activity, backed by server 'stack' responsible for the data model logic, data storage and computational agents dispatched as asynchronous worker processes. remote language consultation is achieved by integrating the external lingobox service, hosted on a commercial backend-as-a-service (baas). in the following sections we elaborate on key components of the sparzan system. ## discussion many people have noted the pressing need to bootstrap data collection for primarily oral, local languages. this is largely a human effort, but it can be scaled up with the support of efficient workflows and assistive technologies. for example, having a searchable lexicon facilitates updating individual entries. however, piecemeal solutions, where we improve the efficiency of individual tasks, only deliver incremental improvements. fully integrated solutions allow us to explore broader questions and to exploit fortuitous opportunities. in the usual pattern of language teaching, learning resources are compiled by 'experts'. learners with their repetitive mistakes and frequent errors have no role to play in crafting lessons. however for local languages there may be few learning resources of the type expected by western learners, and minimal capacity for creating such resources. nevertheless, there are still learning resources to be found. in particular, word recognition errorsobstacles for transcription -are plausible learner errors. these are not useless mistakes to be discarded, but potential prompts for learners to consider and correct. a word that one person has learned and systematically corrected in the course of transcription may become a prompt for another learner. the system remains a prototype and we have not yet not been able to test it on the ground because the communities remain closed. this work has some other limitations. foremost is that the design has been conducted in a university lab, rather than in a co-design process with our partners in the community. a second shortcoming is that the learning potential of collaborative transcription has yet to be explored. using the resulting data to create learning content for dedicated learning apps is promising direction for future work. a third shortcoming is the architectural reliance on low-latency networks. solutions of this type require server infrastructure, but 'bush internet' network conditions rule out the simple convenience of the 'cloud'. one solution is to deploy compact, low-cost server architecture in the field, as we have previously explored with bushpi, a ruggedised batterypowered server to support local use of collaborative language apps @xcite . thus, there is an ongoing need for on-country design and engineering to devise practical, communitybased solutions, building on previous attempts in this space @xcite . despite these shortcomings, we believe this work amounts to a novel and effective design pattern for remote asynchronous collaboration on meticulous language work, serving a variety of documentary and pedagogical goals. the potential for computer supported cooperative language work remains relatively unexplored, and faces an egregious challenge: how do we go from minimally-viable research prototypes to robust, supported, and sustainable solutions (cf. finlayson, 2016, p27)? we believe there remains a clear need for foundational research on technology for working with primarily oral, local languages, supporting a broad range of stakeholders, for the benefit of community goals in sustaining linguistic diversity. ## conclusion we have described a system for cooperative language work, including speech transcription and language learning. it was developed during a period where aboriginal community interactions were severely limited due to the covid-19 pandemic, but where cooperative work and the underlying relationships needed to be sustained. we developed assistive technology to support (and even encourage) language acquisition in the course of transcription and the associated learner-speaker interactions. we set aside expert-defined practice, and instead designed for inclusive participation of learners and speakers, regardless of their technical competencies. in the process, we have demonstrated that the effort of systems engineering for specific sociolinguistic contexts has direct relevance for language data collection and for local language technologies in general.