Synthesising Audio Adversarial Examples for Automatic Speech Recognition

1 INTRODUCTION . Deep neural network ( DNN ) based models have documented many success stories in various domains , such as reinforcement learning ( Silver et al. , 2017 ) , image classification ( Deng et al. , 2009 ) , and automatic speech recognition ( ASR ) ( Chan et al. , 2016 ) . However , DNN models are found to be vulnerable to adversarial attacks ( Goodfellow et al. , 2015 ) , viz. , the slightly perturbed input would cause severe errors or performance drops of well trained DNN models . This paper mainly focuses on the ASR domain , where many voice assistant systems could be hijacked or controlled by the audio adversarial examples constructed by an attacker . To investigate the threat of audio adversarial examples , many different approaches ( Carlini & Wagner , 2018 ; Yuan et al. , 2018 ; Yang et al. , 2019 ; Qin et al. , 2019 ) have been developed . In general , existing approaches assume that the semantics ( for human beings ) of adversarial audio can be preserved as long as the distance between the adversarial audio and the benign audio is restricted properly . In particular , a shared design principle is to add constrained adversarial perturbations ( i.e. , as imperceptible as possible ) on benign audios yet with the goal of fooling ASR models significantly . For instance , Qin et al . ( 2019 ) introduced a psychoacoustic rule of auditory masking to only add perturbation on a benign audio where the noise is hard to be heard . Therefore , existing approaches can be deemed as audio dependent attack ( ADA ) , viz. , the audio adversarial examples have to be constructed depending on some benign audios as shown in Figure 1 ( a ) . However , in real cases , the human speaker and/or the benign audio may not be available or accessible . Moreover , ADA relies on an imperceptible perturbation principle , viz. , the added perturbation must be restricted enough to avoid being perceived by human beings . In contrast , this paper proposes the audio independent attack ( AIA ) as shown in Figure 1 ( b ) that sheds light on a more general principle of adversarial attacks , viz. , any audio that deceives ASR models yet fails to deceive human beings would cause security issues in speech recognition . Alternatively stated , an adversarial audio does not have to be constructed based on an existing benign audio as ADA does . Our AIA thus enables a novel threat model that constructs audio adversarial examples completely from scratch instead of adding perturbations on existing benign audios . Particularly , mounted on recent advances in neural speech synthesis ( Tan et al. , 2021 ) , we can directly synthesise the adversarial audio that conserves the desired semantic content but deceives the ASR model to predict incorrect or even targeted transcriptions . With this goal in mind , we propose the Speech Synthesising based Attack ( SSA ) as shown in Figure 2 , where a conditional variational autoencoder CVAE is incorporated to synthesise audio waveform x that is connected to an ASR model thereafter . The basic philosophy of SSA is : the audio style vector z in CVAE controls the pitches and rhythms of synthesised waveforms that may cause the ASR model to transcribe incorrectly or even as a target ; thus some particular synthesised waveforms would become adversarial examples . In particular , following ( Kim et al. , 2021 ) , we first train CVAE by adopting a variational inference augmented with normalizing flows and an adversarial training process , so as to synthesise natural sounding audios . Once the CVAE is well trained , given a sequence of conditional text that represents the ground truth audio semantics , we can optimize z to get a particular z∗ that can deceive an advanced ASR model through a reg- ularized connectionist temporal classification loss . In this regard , we formulate the adversarial example synthesising task as a gradient sign based optimization problem . More importantly , to solve the optimization efficiently , we design an adaptive learning rate decay based on the annealing mechanism , which can automatically adjust the step size during optimization . Our contributions are summarized as : • For the first time , we propose an audio independent adversarial attack as a novel threat model in ASR , which constructs adversarial example completely from scratch without depending on any existing speaker/audio . This sheds light on a more general principle of adversarial attacks , viz. , any audio that deceives ASR models yet fails to deceive human beings would cause security issues in speech recognition . • We develop SSA that adopts CVAE as the speech synthesiser . To efficiently synthesise audio adversarial examples , we establish an adaptive sign gradient descent algorithm via designing an annealing mechanism inspired learning rate decay . • Extensive experiments across three datasets ( i.e. , Audio Mnist ( Becker et al. , 2018 ) , Common Voice ( Ardila et al. , 2019 ) , and Librispeech ( Panayotov et al. , 2015 ) ) based on DeepSpeech model ( Amodei et al. , 2016 ) show that our SSA can effectively generate audio adversarial examples . 2 RELATED WORK . In ASR adversarial attacks , Carlini & Wagner ( 2018 ) were among the first ones to showcase that slight perturbations on audio can easily fool a start-of-the-art ASR model to transcribe the perturbed audio into any target sentence . Later on , Yuan et al . ( 2018 ) demonstrated that such adversarial audio perturbation can be embedded into a song . Yang et al . ( 2019 ) analyzed that the temporal dependency could promote the discriminative power against adversarial examples in ASR . Moreover , Khare et al . ( 2019 ) and Taori et al . ( 2019 ) found that similar audio adversarial perturbations could be generated using black-box optimization algorithms ( e.g. , genetic algorithm ( Mitchell , 1998 ) ) . To generate the audio adversarial examples faster and improve the attack efficiency , Liu et al . ( 2020 ) proposed to adaptively rectify the weights of audio perturbations on different positions . To make the audio perturbation more imperceptible , Qin et al . ( 2019 ) introduced a psychoacoustic principle of auditory masking to smartly add perturbations on frames where noises are hard to be perceived . Xie et al . ( 2021 ) utilized generative adversarial network ( GAN ) to generate adversarial examples on speech domain , where GAN is to learn the distribution of predefined adversarial perturbations . Thus the adversarial examples are still generated depending on the benign audios . In summary , previous adversarial attacks on ASR are audio-dependent , viz. , the audio adversarial examples must be generated based on corresponding benign audios . Although there are studies , e.g. , ( Song et al. , 2018 ; Wang et al. , 2019 ) , on generating adversarial examples from scratch , but most of them focus on the image domain . Carlini & Wagner ( 2018 ) studied an audio attack that starts from non-speech ( e.g. , a piece of classic music ) , while existing audios are stilled required to mount perturbations on . Roy et al . ( 2018 ) and Zhang et al . ( 2017 ) proposed to modulate voice commands on ultrasonic carriers ( e.g. , frequency > 20 kHz ) to achieve inaudibility , where the scenario is an adversary that stands on the road and silently controls the voice command assistant systems . This therefore is different from our scope of generating natural sounding adversarial audios . To the best our knowledge , our work is the first attempt to generate audio adversarial examples from scratch without utilizing benign audios in ASR . 3 BACKGROUND . Automatic Speech Recognition ( ASR ) . In this paper , we focus on the speech-to-text tasks that are based on neural ASR models . Following previous studies ( Esmaeilpour et al. , 2021 ; Liu et al. , 2020 ; Yakura & Sakuma , 2019 ; Carlini & Wagner , 2018 ) , we stick our attacks on the DeepSpeech ( Amodei et al. , 2016 ) , a state-of-the-art ASR model based on Connectionist Temporal Classification ( CTC ) method ( Graves et al. , 2006 ) . DeepSpeech uses CTC as the input aduios and the corresponding transcriptions are unaligned , viz. , the exact position of each word in the audio sample is unknown . To enable efficient supervised training , a transcription is first enumerated to obtain all alignments . Hence , the CTC loss is minimized to maximize the probabilities over all alignments . CTC Loss . Let X be the audio input domain and Y be the text output domain with dimension |Y| . The ASR model is donoted as F : XN → YN ·|Y| , which takes a N frames x ∈ X as input and outputs a probability distribution F ( x ) over the output domain . Given y as a phrase ( i.e. , a sequence of characters ) , we define a token sequence π being reducible to y if the two operations , namely , removing sequentially duplicated tokens , and deleting all blank tokens on π could produce y . We further denote π as an alignment of y if π can be reduced to y and the length of π equals to the length of prediction F ( x ) . Let Φ ( y ) be the alignment set obtained from targeted transcription y using dynamic programming ( Graves et al. , 2006 ) . Accordingly , the CTC loss can be formulated as , Lctc ( x , y ) = −log [ ∑ π∈Φ ( y ) ∏N i F ( x ) iπi ] , ( 1 ) whereF ( x ) iπi is the probability of the token πi on the ith frame . To get the transcription in inference , a decoder D ( e.g. , greedy decoding or beam search decoding ) is required . Thereby , if an ASR model is well trained , the transcription would satisfy y = D ( F ( x ) ) = f ( x ) , where f ( · ) is a merged denotation of F and D. White-Box Assumption . Following most of previous studies , we use the similar white-box assumption , viz. , the parameter and structure of ASR model is known . Investigating the black-box attack is another direction . Moreover , there are many ways ( Oh et al. , 2019 ; Zanella-Beguelin et al. , 2021 ; Qin et al. , 2019 ) to convert a black-box model to a white-box one . Targeted Attack . Compared with the untargeted attack that only maximizes the word error rate ( WER ) , the targeted attack is a more challenging task since it requires not only the audio perturbation imperceptible , but also the adversarial audio being transcribed to a specified target phrase . Our main focus is on the targeted scenario . Without specification , the adversarial example in following sections refers to the targeted one . Speech Synthesise ( End-To-End Text-To-Speech ) . Text-to-speech ( TTS ) model synthesises waveforms given text phrases as semantic contents . For efficiently synthesising adversarial audios in our SSA , we turn to the end-to-end TTS model that could easily utilize modern parallel processors for faster synthesis speed . In particular , we utilize a conditional variational autoencoder ( Kim et al. , 2021 ) based TTS model to generate natural sounding audios , which is explained in Section 4.2 .

Summary: This paper presents a speech synthesising-based attack (SSA) framework that generates audio examples entirely from scratch to attack automatic speech recognition (ASR) systems. A conditional variational auto-encoder (CVAE) based speech synthesiser is trained based on a combined connectionist temporal classification (CTC) and regularization loss. As compared to existing methods, the proposed approach serves as the first work that generates adversarial samples without audio inputs. The results confirm that the proposed SSA framework can attack the ASR system successfully. A comprehensive analysis of different hyper-parameters to the achievable performance is provided. Moreover, demo audio samples are provided by a website link.