Synthesising Audio Adversarial Examples for Automatic Speech Recognition

1 INTRODUCTION . Deep neural network ( DNN ) based models have documented many success stories in various domains , such as reinforcement learning ( Silver et al. , 2017 ) , image classification ( Deng et al. , 2009 ) , and automatic speech recognition ( ASR ) ( Chan et al. , 2016 ) . However , DNN models are found to be vulnerable to adversarial attacks ( Goodfellow et al. , 2015 ) , viz. , the slightly perturbed input would cause severe errors or performance drops of well trained DNN models . This paper mainly focuses on the ASR domain , where many voice assistant systems could be hijacked or controlled by the audio adversarial examples constructed by an attacker . To investigate the threat of audio adversarial examples , many different approaches ( Carlini & Wagner , 2018 ; Yuan et al. , 2018 ; Yang et al. , 2019 ; Qin et al. , 2019 ) have been developed . In general , existing approaches assume that the semantics ( for human beings ) of adversarial audio can be preserved as long as the distance between the adversarial audio and the benign audio is restricted properly . In particular , a shared design principle is to add constrained adversarial perturbations ( i.e. , as imperceptible as possible ) on benign audios yet with the goal of fooling ASR models significantly . For instance , Qin et al . ( 2019 ) introduced a psychoacoustic rule of auditory masking to only add perturbation on a benign audio where the noise is hard to be heard . Therefore , existing approaches can be deemed as audio dependent attack ( ADA ) , viz. , the audio adversarial examples have to be constructed depending on some benign audios as shown in Figure 1 ( a ) . However , in real cases , the human speaker and/or the benign audio may not be available or accessible . Moreover , ADA relies on an imperceptible perturbation principle , viz. , the added perturbation must be restricted enough to avoid being perceived by human beings . In contrast , this paper proposes the audio independent attack ( AIA ) as shown in Figure 1 ( b ) that sheds light on a more general principle of adversarial attacks , viz. , any audio that deceives ASR models yet fails to deceive human beings would cause security issues in speech recognition . Alternatively stated , an adversarial audio does not have to be constructed based on an existing benign audio as ADA does . Our AIA thus enables a novel threat model that constructs audio adversarial examples completely from scratch instead of adding perturbations on existing benign audios . Particularly , mounted on recent advances in neural speech synthesis ( Tan et al. , 2021 ) , we can directly synthesise the adversarial audio that conserves the desired semantic content but deceives the ASR model to predict incorrect or even targeted transcriptions . With this goal in mind , we propose the Speech Synthesising based Attack ( SSA ) as shown in Figure 2 , where a conditional variational autoencoder CVAE is incorporated to synthesise audio waveform x that is connected to an ASR model thereafter . The basic philosophy of SSA is : the audio style vector z in CVAE controls the pitches and rhythms of synthesised waveforms that may cause the ASR model to transcribe incorrectly or even as a target ; thus some particular synthesised waveforms would become adversarial examples . In particular , following ( Kim et al. , 2021 ) , we first train CVAE by adopting a variational inference augmented with normalizing flows and an adversarial training process , so as to synthesise natural sounding audios . Once the CVAE is well trained , given a sequence of conditional text that represents the ground truth audio semantics , we can optimize z to get a particular z∗ that can deceive an advanced ASR model through a reg- ularized connectionist temporal classification loss . In this regard , we formulate the adversarial example synthesising task as a gradient sign based optimization problem . More importantly , to solve the optimization efficiently , we design an adaptive learning rate decay based on the annealing mechanism , which can automatically adjust the step size during optimization . Our contributions are summarized as : • For the first time , we propose an audio independent adversarial attack as a novel threat model in ASR , which constructs adversarial example completely from scratch without depending on any existing speaker/audio . This sheds light on a more general principle of adversarial attacks , viz. , any audio that deceives ASR models yet fails to deceive human beings would cause security issues in speech recognition . • We develop SSA that adopts CVAE as the speech synthesiser . To efficiently synthesise audio adversarial examples , we establish an adaptive sign gradient descent algorithm via designing an annealing mechanism inspired learning rate decay . • Extensive experiments across three datasets ( i.e. , Audio Mnist ( Becker et al. , 2018 ) , Common Voice ( Ardila et al. , 2019 ) , and Librispeech ( Panayotov et al. , 2015 ) ) based on DeepSpeech model ( Amodei et al. , 2016 ) show that our SSA can effectively generate audio adversarial examples . 2 RELATED WORK . In ASR adversarial attacks , Carlini & Wagner ( 2018 ) were among the first ones to showcase that slight perturbations on audio can easily fool a start-of-the-art ASR model to transcribe the perturbed audio into any target sentence . Later on , Yuan et al . ( 2018 ) demonstrated that such adversarial audio perturbation can be embedded into a song . Yang et al . ( 2019 ) analyzed that the temporal dependency could promote the discriminative power against adversarial examples in ASR . Moreover , Khare et al . ( 2019 ) and Taori et al . ( 2019 ) found that similar audio adversarial perturbations could be generated using black-box optimization algorithms ( e.g. , genetic algorithm ( Mitchell , 1998 ) ) . To generate the audio adversarial examples faster and improve the attack efficiency , Liu et al . ( 2020 ) proposed to adaptively rectify the weights of audio perturbations on different positions . To make the audio perturbation more imperceptible , Qin et al . ( 2019 ) introduced a psychoacoustic principle of auditory masking to smartly add perturbations on frames where noises are hard to be perceived . Xie et al . ( 2021 ) utilized generative adversarial network ( GAN ) to generate adversarial examples on speech domain , where GAN is to learn the distribution of predefined adversarial perturbations . Thus the adversarial examples are still generated depending on the benign audios . In summary , previous adversarial attacks on ASR are audio-dependent , viz. , the audio adversarial examples must be generated based on corresponding benign audios . Although there are studies , e.g. , ( Song et al. , 2018 ; Wang et al. , 2019 ) , on generating adversarial examples from scratch , but most of them focus on the image domain . Carlini & Wagner ( 2018 ) studied an audio attack that starts from non-speech ( e.g. , a piece of classic music ) , while existing audios are stilled required to mount perturbations on . Roy et al . ( 2018 ) and Zhang et al . ( 2017 ) proposed to modulate voice commands on ultrasonic carriers ( e.g. , frequency > 20 kHz ) to achieve inaudibility , where the scenario is an adversary that stands on the road and silently controls the voice command assistant systems . This therefore is different from our scope of generating natural sounding adversarial audios . To the best our knowledge , our work is the first attempt to generate audio adversarial examples from scratch without utilizing benign audios in ASR . 3 BACKGROUND . Automatic Speech Recognition ( ASR ) . In this paper , we focus on the speech-to-text tasks that are based on neural ASR models . Following previous studies ( Esmaeilpour et al. , 2021 ; Liu et al. , 2020 ; Yakura & Sakuma , 2019 ; Carlini & Wagner , 2018 ) , we stick our attacks on the DeepSpeech ( Amodei et al. , 2016 ) , a state-of-the-art ASR model based on Connectionist Temporal Classification ( CTC ) method ( Graves et al. , 2006 ) . DeepSpeech uses CTC as the input aduios and the corresponding transcriptions are unaligned , viz. , the exact position of each word in the audio sample is unknown . To enable efficient supervised training , a transcription is first enumerated to obtain all alignments . Hence , the CTC loss is minimized to maximize the probabilities over all alignments . CTC Loss . Let X be the audio input domain and Y be the text output domain with dimension |Y| . The ASR model is donoted as F : XN → YN ·|Y| , which takes a N frames x ∈ X as input and outputs a probability distribution F ( x ) over the output domain . Given y as a phrase ( i.e. , a sequence of characters ) , we define a token sequence π being reducible to y if the two operations , namely , removing sequentially duplicated tokens , and deleting all blank tokens on π could produce y . We further denote π as an alignment of y if π can be reduced to y and the length of π equals to the length of prediction F ( x ) . Let Φ ( y ) be the alignment set obtained from targeted transcription y using dynamic programming ( Graves et al. , 2006 ) . Accordingly , the CTC loss can be formulated as , Lctc ( x , y ) = −log [ ∑ π∈Φ ( y ) ∏N i F ( x ) iπi ] , ( 1 ) whereF ( x ) iπi is the probability of the token πi on the ith frame . To get the transcription in inference , a decoder D ( e.g. , greedy decoding or beam search decoding ) is required . Thereby , if an ASR model is well trained , the transcription would satisfy y = D ( F ( x ) ) = f ( x ) , where f ( · ) is a merged denotation of F and D. White-Box Assumption . Following most of previous studies , we use the similar white-box assumption , viz. , the parameter and structure of ASR model is known . Investigating the black-box attack is another direction . Moreover , there are many ways ( Oh et al. , 2019 ; Zanella-Beguelin et al. , 2021 ; Qin et al. , 2019 ) to convert a black-box model to a white-box one . Targeted Attack . Compared with the untargeted attack that only maximizes the word error rate ( WER ) , the targeted attack is a more challenging task since it requires not only the audio perturbation imperceptible , but also the adversarial audio being transcribed to a specified target phrase . Our main focus is on the targeted scenario . Without specification , the adversarial example in following sections refers to the targeted one . Speech Synthesise ( End-To-End Text-To-Speech ) . Text-to-speech ( TTS ) model synthesises waveforms given text phrases as semantic contents . For efficiently synthesising adversarial audios in our SSA , we turn to the end-to-end TTS model that could easily utilize modern parallel processors for faster synthesis speed . In particular , we utilize a conditional variational autoencoder ( Kim et al. , 2021 ) based TTS model to generate natural sounding audios , which is explained in Section 4.2 .

Summary: This paper describes a white-box attack on ASR systems using TTS.  The main technique is (somewhat) standard adversarial training with the constraint that the CVAE component of a TTS model is the only component that is updated.  Since this component is responsible for prosodic and other non-segmental qualities of speech synthesis, the expectation is that the naturalness of the synthesis will not be impacted, but the ASR recognition will be modified.