Securely Capturing People’s Interactions with Voice Assistants at Home: A Bespoke Tool for Ethical Data Collection (2022)

Abstract: Speech production is nuanced and unique to every individual, but today’s Spoken Dialogue Systems (SDSs) are trained to use general speech patterns to successfully improve performance on various evaluation metrics. However, these patterns do not apply to certain user groups - often the very people that can benefit the most from SDSs. For example, people with dementia produce more disfluent speech than the general population. In order to evaluate systems with specific user groups in mind, and to guide the design of such systems to deliver maximum benefit to these users, data must be collected securely. In this short paper we present CVR-SI, a bespoke tool for ethical data collection. Designed for the healthcare domain, we argue that it should also be used in more general settings. We detail how off-the-shelf solutions fail to ensure that sensitive data remains secure and private. We then describe the ethical design and security features of our device, with a full guide on how to build both the hardware and software components of CVR-SI. Our design ensures inclusivity to all researchers in this field, particularly those who are not hardware experts. This guarantees everyone can collect appropriate data for human evaluation ethically, securely, and in a timely manner.

## introduction data collection is vital if we are to create more natural and more accessible spoken dialogue systems (sdss) embedded within voice assistants and social robots @xcite @xcite . as these technologies are applied with admirable goals in the healthcare domain, general voice datasets lose the ability to accurately reflect the end-user. for example, speech production changes as cognition declines; people use more prepositions, slow their speech rate, pause more frequently mid-sentence, and pause for longer durations as dementia progresses @xcite @xcite @xcite . we can refine evaluation metrics endlessly, but a system's practical benefit to the end-user remains unknown without data representing that specific user group. it is critical that this data is collected ethically and securely as vulnerable user groups are particularly common in the healthcare domain. issues around consent have been explored as individuals develop cognitive impairments, but identifiable information will still be captured and this is a concern @xcite . data privacy does not just affect people with cognitive impairments however, people affected by sight loss can unwittingly reveal sensitive information @xcite , as will individuals conversing during a gp consultation @xcite . off-the-shelf devices are not secure. if used, all sensitive data that is captured will be fully accessible to anyone if the device is lost or stolen. very few audio recorders even exist with this capability due to copyrighting of encrypted audio codecs @xcite , and the ones that do exist are expensive and not applicable or adaptable for ethical data collection (see ## previous work a data capture device, called cvr, was used to collect similar data in a less-sensitive domain @xcite . this device was used to collect family interactions with amazon alexa devices within participants homes over a period of one-month. while we would argue that the cvr would have certainly captured personally identifiable information, this risk is heightened in our desired features dpm cvr cusco cvr-si captures audio ✓ ✓ ✓ ✓ clearly indicates when 'on' to user ✗ ✓ ✗ ✓ clearly indicates when 'recording' to user ✗ ✓ ✗ ✓ user can easily stop the device listening ✗ ✓ ✗ ✓ data is securely stored ✓ ✗ ✓ ✓ data is encrypted in real-time ✓ ✗ ✓ ✓ recording uses wake-word detection ✗ ✓ ✗ ✓ adequate storage capacity ✗ ✓ ✓ ✓ a security-focused data capture device, called cusco (addlesee and albert, 2020), was created for sensitive in-person data collections like medical conversations. participants would interact or complete a task with the researchers in attendance at all times. therefore, this device does not face the same challenges as a long-term device that cannot be monitored or controlled mid-study. cusco does implement real-time data encryption however, a critical feature that ensures no data can be accessed even if the device is stolen during recording. with advice from the creators of the cvr (conditional voice recorder), we used their work as a starting point. hence our device's name: cvr-si (conditional voice recorder for sensitive information). we then adapted the data security features of cusco and integrated them to create cvr-si. in cvr-si has been ethically approved for use by heriot-watt university's ethics committee and has been successfully used within vulnerable participant's homes. in the following sections we will describe the device's software, explain the security features, detail exactly how to construct the cvr-si, and highlight components that tackle ethical issues foot_0 . the final cvr-si can be seen in figure 1 . ## device hardware we created several prototypes of the cvr-si device, and then built this device at scale (20 units) as seen in figure 2 . various hardware design decisions were made to mitigate ethical concerns foot_2 . ## findings from use in practice in this short paper we have detailed both the software and hardware of cvr-si, a data capture device with both data security and ethics at the core of its design. the cvr-si has been ethically approved and used to capture interactions between people with dementia and alexa voice assistants in their own home. we have already learned a great deal from real-world deployment, for example the microsd storage upgrade described in section 4.1. participants have reported using the device's button to stop the cvr-si capturing audio when family or health-workers are visiting, indicating that this feature is desired for privacy and that the leds are clear. one participant noted that they used the button at times they felt "big brother was listening". this is an understandable feeling that is generally felt with smart speakers @xcite , indicating again that the leds are clear and the button is a necessary device feature for participant comfort. although data analysis is yet to be complete, initial observations have revealed instances in which the alexa does not activate when the user says the wake-word. the buffer has therefore proven to be a useful feature, driving the need for live encryption. recordings have been clear and do not skip, demonstrating the sufficient capabilities of both the microphone and raspberry pi. finally, participants have described how they have used the alexa device in their day-to-day lives. people with dementia have been able to reawaken their love for music, set reminders to take medication or walk their dogs, get help with their crosswords, and even find new recipes to help get involved with family mealtimes. voice assistants can clearly have a positive impact, so we hope our work will accelerate voice accessibility research.