Dialogue Act Classification for Augmentative and Alternative Communication (2021)

Abstract: Augmentative and Alternative Communication (AAC) devices and applications are intended to make it easier for individuals with complex communication needs to participate in conversations. However, these devices have low adoption and retention rates. We review prior work with text recommendation systems that have not been successful in mitigating these problems. To address these gaps, we propose applying Dialogue Act classification to AAC conversations. We evaluated the performance of a state of the art model on a limited AAC dataset that was trained on both AAC and non-AAC datasets. The one trained on AAC (accuracy = 38.6%) achieved better performance than that trained on a non-AAC corpus (accuracy = 34.1%). These results reflect the need to incorporate representative datasets in later experiments. We discuss the need to collect more labeled AAC datasets and propose areas of future work.

## introduction dialogue act classification takes a conversation transcript as input and identifies the appropriate intent for each turn in a conversation. for example, the sentence "how are you?" might be classified as an open ended question. the exact tags that are used to label sentences depend on the context. the switchboard damsl tag set @xcite is frequently used as a standard initial classification model which has forty-two distinct classes. once labeled conversational data is available, it can be used to create generative statistical systems that take a sentence and a prior dialogue act as input and provide the next most like dialogue act for the conversation. prior research has used this information to analyze both human-human conversations and better facilitate human-machine conversations @xcite . however, research in dialogue act classification has not included conversations with individuals who do not rely solely on verbal speech to communicate. as of the 2010 united states census, approximately 15.7 million adults were listed as having a communicative disability @xcite . the communicative disability domain includes individuals who identify as having either a visual, hearing, or speech impairment or some combination of the three. many of these individuals communicate through non-verbal methods including augmentative and alternative communication (aac) technology; we will refer to this population as aac communicators. aac communicators leverage a broad set of tools to supplement their verbal speech or to replace it entirely. speech language pathologists may recommend aac as part of a treatment plan for an individual in order to maximize their ability to effectively communicate in their environment. in addition to providing more communication methods, research has proven that aac technology can actually improve language development skills in children @xcite . these systems vary in technical sophistication from picture boards corresponding to concepts to tablets or application based speech-generating devices @xcite . figure 1 depicts two such devices, both of which include touch-based text displays. there are standalone or dedicated aac devices available that only provide a communication interface whereas application-based solutions may run on a personal tablet or mobile device. in addition to touch access, devices can also incorporate eye-gaze, switch, or brainwave input. some devices will allow individuals to switch between different access modes to account for fatigue levels they may experience at different times. @xcite the exact system used is tailored to the individual based on their cognitive, communicative, and figure 1 : two examples of aac devices, a) is a dedicated aac device using touch-input and b) is an aac application running on a non-dedicated device. @xcite physical profile. an ideal system will maximize the individual's ability to express themselves while minimizing the cognitive and physical demands of using the system. speech pathologists will perform an initial aac evaluation to match a patient with the appropriate device to fit their immediate needs and long term communication goals and define an aac intervention plan to track progress on these goals. over the course of intervention, the system may be adjusted in order to better suit the needs of the individual whether it be physical changes to accommodate improved or worsening motor functionality or word selection adjustments to introduce more complicated vocabulary. over the last few decades, aac devices have improved significantly, but satisfaction and retention rates for them remain low @xcite . there has been some effort to improve on-screen word prediction, but it has yet to provide sufficiently relevant suggestions during conversations or improve communication rates for aac communicators. as we will discuss in the related work section, the application of nlp to aac technology has been primarily limited to word prediction, despite the expansion of the field to a multitude of other tasks. we hypothesize that incorporating dialogue act information into aac technology will improve the ease of use of these devices and in turn positively impact the ability of aac communicators to participate in conversations. in this paper, we address the potential benefits of applying dialogue act classification to conversations that include a participant communicating via an aac device. we will start by presenting the previous nlp applications that have been used to enhance aac software as well as currently available representative datasets. then, we evaluate the performance of an existing state-of-the-art model on a small dataset of transcribed conversations between an aac communicator and one of their daily communication partners. finally, we present the challenges that inhibit work in this context. ultimately, we hope that future researchers will recognize the value of applying language models to conversations with aac communicators in order to improve their ability to independently participate in educational, social, and career settings. ## implications of dialogue act analysis for aac applying dialogue act classification to conversations including communicators reliant on aac has the potential to improve their ability to communicate as well as enhance the aac intervention and evaluation processes. ## experiments to explore the potential of dialogue act classification for conversations including individuals using aac, we will evaluate the accuracy of a state of the art model on a small representative dataset. ## results the results in the low accuracy scores could also be reflective of the need to use better annotation tags. the aac conversational data used for future experiments could benefit from aac context specific dialogue act tags. although the damsl tags work well for comparison to results on existing datasets, it would be more beneficial to use a refined tag set that is specific to the aac data that is being analyzed. ideally, the tags should be modified depending on the context in which they are being used. a speech pathologist may choose to use a specific set of classes for coding sentences that are closely related to the goals of the individual's intervention. these tags are different from a set that would be used to provide real-time suggestions on a device during a typical conversation. once a set of context appropriate tags has been established, the aac data should be annotated by a group of annotators who are familiar with interpreting aac output, such as speech pathologists in order to establish more accurate standard tags and inter-annotator agreement. ## conclusion in this paper, we introduced the possibility of applying dialogue act classification to conversations that include one or more individuals communicating via aac devices. we have discussed the potential benefits for applying this nlp technique to aac applications as well as speech pathology transcriptions. in the experiments, we trained a dialogue act classification model on a small subset of aac data and determined that a model trained on the switchboard corpus does not perform as well on aac conversational data as one that has been trained on a representative corpus. however, with the current aac corpus available, the accu-racy for the dialogue act classification model is far from the current benchmarks for these models on the switchboard corpus. this indicates the need for more experiments to improve dialogue act classification accuracy in this context. future experiments will require labeled aac conversational data for model training. the data collection task is a daunting proposition due to the fact that aac devices are inherently personalized. each aac intervention session is tailored to the patient, the devices chosen, and goals set are meant to be the best choice for that individual. a representative dataset would need to be sufficiently large to incorporate individuals with different communication impairments, different degrees of impairment, and those from different age groups. the data must also be collected in a manner to preserve the privacy of the individuals' included. furthermore, the current limitations of aac devices may bias the data that could be collected. if a person is currently limited in their communicative ability due to poor user design with the device, the speech that they output will not be reflective of their full communicative desires. to mitigate this risk, further research needs to be done with this population to better incorporate their conversational goals into a representative dataset. additionally, any model proposed should have a personalizable component. this component would allow for supplemental training data that is based on the individual's recent conversation history or a bootstrapping period. the inclusion of a personalization training period would provide enhanced conversational agency as well as incorporating an individual's communication style into their aac device. once a sufficiently accurate dialogue act classification model is available, it can be embedded into a standalone aac device or aac application. this prototype should be designed alongside individuals who currently communicate via aac as well as their aac team. through a collaborative research and design process, we hope to see improvements in usage rates for aac devices and higher satisfaction rates by aac communicators. more importantly, we anticipate that as aac devices improve, aac communicators will find themselves able to participate in more professional, educational, and social conversational opportunities.