Real-Time Neural Voice Camouflage

1 INTRODUCTION . Automatic speech recognition models are embedded in nearly all smart devices . Although these models have many exciting applications , the concern for the potential of these devices to eavesdrop is significant . It is becoming increasingly important to develop methods that give users the autonomy to safeguard their speech from voice processing software . Fortunately , over the last decade , there has been work demonstrating that neural networks models are easily fooled . For example , they remain vulnerable to small additive perturbations ( Carlini & Wagner , 2018 ) , ambient noise ( Xu et al. , 2020 ) , and unusual examples ( Nguyen et al. , 2015 ) . Predominant methods such as gradient-based methods and their variants have remained the standard approach to generating challenging examples for deep neural networks ( Madry et al. , 2019 ) . However , to achieve this , these methods require the full input upfront , and thus users can not practically use them as they continuously speak . Therefore , the community has increasingly been focusing on researching general , robust methods of breaking neural networks that can be used in real-time . We define robust to mean an obstruction that can not be easily removed , real-time to mean an obstruction that is generated continuously as speech is spoken , and general to mean applicable to the majority of vocabulary in a language . Existing prior work has successfully tackled at least one of these three requirements , but none all three . While some work is real-time ( Chen et al. , 2020 ; Schönherr et al. , 2018 ) , these disruptions can be filtered out as they are constrained to specific frequency ranges . Universal attacks ( Lu et al. , 2021 ) can be similarly subtracted . Gong et al . ( 2019 ) achieved both real-time and robust obstructions , but are limited to a predefined set of ten words . Streaming audio is a particularly demanding domain to disrupt because the calculation needs to be performed in real-time . By the time a sound is computed , time will have passed and the streaming signal will have changed , making standard generative methods obsolete . The sampling rate of audio is at least 16 kHz , meaning the corruption for a given input must be estimated and played over a speaker within milliseconds , which is currently infeasible . Additionally , when attacks are played over-the-air , the attack needs to be loud enough to disrupt any rogue microphone that could be far away . The attack sound needs to carry the same distance as the voice . We introduce predictive attacks , which are able to disrupt any word that automatic speech recognition models are trained to transcribe . Our approach achieves real-time performance by forecasting an attack on the future of the signal , conditioned on two seconds of input speech . Our attack is optimized to have a volume similar to normal background noise , allowing people in a room to converse naturally and without monitoring from an automatic speech recognition system . Forecasting with deep neural networks has already been successfully used in other domains to achieve real-time performance , for instance in packet loss concealment ( Pascual et al. , 2021 ) . In this paper , we demonstrate how and why this approach lends itself particularly well to developing general , robust and real-time attacks for automatic speech recognition models . Our experiments show that predictive attacks are able to largely disrupt the established DeepSpeech ( Amodei et al. , 2016 ) recognition system which was trained on the LibriSpeech dataset ( Panayotov et al. , 2015 ) . On the standard , large-scale dataset LibriSpeech , our approach causes at least a three fold increase in word error rate over baselines , and at least a six fold increase in character error rate . Our method is practical and straightforward to implement on commodity hardware . We additionally demonstrate the method works inside real-world rooms with natural ambient noise and complex scene geometries . We call our method Neural Voice Camouflage . 2 RELATED WORK . Breaking Neural Networks : Szegedy et al . ( 2014 ) first discovered adversarial attacks in computer vision . Since then , a large number of methods to break neural networks have been introduced ( Madry et al. , 2019 ; Kurakin et al. , 2017 ; Carlini & Wagner , 2017 ; Croce & Hein , 2020 ; MoosaviDezfooli et al. , 2016 ; Goodfellow et al. , 2014 ) , where noise optimized by gradient descent fool the state-of-the-art models . Audio adversarial attacks ( Carlini & Wagner , 2018 ; Qin et al. , 2019 ; Yakura & Sakuma , 2019 ; Schönherr et al. , 2018 ) have also been constructed . Gradient based iterative adversarial attacks , while effective , are computationally intensive , and need to see the whole example first before launching the attack . Faster adversarial attacks uses generators to generate attacks ( Xiao et al. , 2019 ) . However , the attacks are still offline . To make the adversarial attack reliable for live speech , the attacker needs to anticipate the future in an online manner . Online Attacks : Real-time attacks are an emerging area of research in machine learning and there have been several initial works . For example , Gong et al . ( 2019 ) develop a reinforcement learning based approach to balance the trade-off between number of samples seen and attack deployment duration . They also optimize a volume trade-off to achieve over-the-air performance . While they learn to disrupt spoken keyword detection ( a predefined set of ten words ) , our approach is able to obfuscate entire sentences . Further , attacks for streaming data with bayesian approaches have been proposed ( Braverman et al. , 2021 ; Seraphim & Poovammal , 2021 ) . However , they are unable to tackle high-dimensional data such as audio . Another direction prior work has taken to create online attacks is to constantly be attacking a certain word ( Li et al. , 2019 ) . Although this works in realtime , it only targets the wake word , and not full sentences . There also have been a few methods that jam microphones by emitting sound outside the range of human hearing . For example , Chen et al . ( 2020 ) developed an approach to emit ultrasonic attacks and Schönherr et al . ( 2018 ) also generate Time ASR ȳt Predictive Attack gθ t − r − δ αt t fψ δ t − r xt−r−δ Speech Attack Figure 2 : We illustrate our problem set-up for predictive attacks . In order to attack the audio starting at time t − r , we need to start computing the attack by time t− r− δ , assuming it takes an upper bound of δ time to record , compute and play the attack . Our approach is able to obtain real-time performance by predicting this attack in the future , given the previous observations of the stream . attacks outside the human hearing range . However , by limiting the attack to specific frequencies , a defender can design a microphone that filters this set of frequencies out . Robustness : Due to the importance of this problem , there has been extensive research into learning robust models ( Madry et al. , 2019 ; Carmon et al. , 2019 ; Wang et al. , 2020 ; Mao et al. , 2019 ; 2020 ; 2021 ) . However , building defenses is challenging , and work has even shown that many basic defenses , such as adding randomness to the input , are not effective ( Athalye et al. , 2018 ) . Among all the defense strategies , adversarial training proposed by Madry et al . ( 2019 ) is the standard defense that has been most widely used . However , adversarial training has the drawback that it improves robustness accuracy at the cost of reducing the original accuracy ( Tsipras et al. , 2019 ) , which is the reason that adversarial training is not used in most real-world applications . In this paper , we show our approach is still effective against these established defenses . Real-time Machine Learning : Interest in real-time artifical intelligence dates back to 1996 , starting with anytime algorithms , which return a solution at any given point in time ( Zilberstein , 1996 ) . More recently , there have been challenges to evaluate vision models in real-time ( Kristan et al. , 2017 ) . Generally , there has been a focus on speeding up forward passes to allow for faster inference , thereby approaching real-time ( Howard et al. , 2017 ) . In addition , there has been recent work in leveraging deep neural network predictions to achieve real-time performance , which has been applied to speech packet loss concealment ( Pascual et al. , 2021 ) . This differs from the previous approaches of improving inference speed . Recently , the community has recently taken an interest in establishing robust metrics and evaluations for real-time inference ( Li et al. , 2020 ) . 3 METHOD . We present our approach for creating real-time obstructions to automatic speech recognition ( ASR ) systems . We first motivate the background for real-time attacks , then introduce our approach that achieves online performance through predictive attack models . 3.1 STREAMING SPEECH RECOGNITION . Let xt be a streaming signal that represents the input speech up until time t. The goal of ASR is to transcribe this signal into the corresponding text yt . The field often estimates this mapping through a neural network ŷt = fψ ( xt ) where the parameters ψ are optimized to minimize the empirical risk minψ E ( x , y ) [ L ( ŷt , yt ) ] . For modeling sequences , the CTC loss function is a common choice for L ( Graves et al. , 2006 ) . In offline setups , we can corrupt the neural network fψ with a standard adversarial attack . These attacks work by finding a minimal additive perturbation vector αt that , when added to the input signal , produces a high loss : arg maxαt L ( fψ ( xt + αt ) , yt ) subject to a bound on the norm of the perturbation ‖αt‖∞ < . Adversarial attacks , such as projected gradient descent ( PGD ) or fast gradient descent , have been widely effective on vision and speech datasets ( Madry et al. , 2019 ; Goodfellow et al. , 2014 ; Carlini & Wagner , 2018 ) . Defending against them both empirically and theoretically remains an active area of research today . Standard adversarial attacks will optimize the perturbation vector αt conditioned on the current position of the stream xt . However , by the time the solution αt is found for a stream , the attack will be obsolete because time will have passed and the condition will have almost certainly changed . Audio is a particularly demanding domain because the high sampling rate ( as high as 48 kHz ) would require attacks to be computed nearly instantaneously ( less than 20 microseconds ) . Furthermore , applying the stale αt to the future xt+δ will not work because the attack vectors are optimized to corrupt the features of their input , which may vary over time . 3.2 PREDICTIVE REAL-TIME ATTACKS . We propose a class of predictive attacks , which enable real-time performance by forecasting the attack vector that will be effective in future time steps . It will invariably take some time for the attack to be computed . For attacks to operate in real-time environments , this means the attack needs to be optimized not for the observed signal , but for the unobserved signal in the future . If our observation of the signal xt is captured at time t and our algorithm takes δ seconds to compute an attack and play it , then we need to attack the signal starting at xt+δ . However , creating these attacks is challenging practically because real-world signals are stochastic and multi-modal . Due to the high uncertainty , generating future speech xt+δ for the purpose of computing attacks is infeasible . Rather than forecasting the signal , we will learn to forecast the attack vector , which encloses all possible variations of the next utterances conditioned on the current input . This attack will learn to “ hedge the bet ” by finding a single , minimal pattern that robustly obstructs all upcoming possibilities . Under the perturbation bound , we model predictive attacks as : αt+δ+r = gθ ( xt ) s.t . ‖gθ ( xt ) ‖∞ < , ( 1 ) where gθ is a predictive model conditioned on the present input speech and paramaterized by θ . To be consistent with our notation , which represents xt as the signal until time t , we include an additional offset r to represent the temporal duration of the attack . To satisfy the constraint on the perturbation bound , we use the tanh activation function to squash the range to the interval [ −1 , 1 ] before multiplying the result by a scalar . This is equal to the product of a predetermined multiplier m and the maximum of the absolute value of the input speech waveform . With predictive attacks , the algorithm for generating obstructions in real-time becomes straightforward . After the microphone observes xt , the speakers need to play αt+δ+r exactly δ seconds later . Since sound is additive modulo reverberation , this will cause a third-party microphone to receive the corrupted signal xt+δ+r + gθ ( xt ) . We found modeling the room acoustics was unnecessary because significant reverberation already breaks state-of-the-art ASR models . We will use neural networks to instantiate the predictive model g. To obtain real-time performance , our feed forward calculation needs to be less than the delay δ into the future . On commodity hardware today , this calculation is on the order of 50 milliseconds .

Summary: This paper proposes a Neural Voice Camouflage (NVC) method that has three important characteristics, which are essential for an NVC method to be used in practical scenarios: general, real-time, and robust. Since the proposed method trains a model to learn predictive attacks without any constraints about input and output, it can be applied to any vocabulary in a real-time scenario, and it is also difficult to defend the attack. On the contrary, the previous gradient-based adversarial attacks take a lot of time to compute the attack, so it is difficult to be used in a real-time scenario. Other than that, other previous methods are trained to attack only a few target words or utilize a pre-defined frequency region that can be easily filtered out.  In experiments, this paper shows that the proposed method is really effective by showing that the WER&CER of an ASR model significantly increases with the method compared to other NVC methods. Furthermore, this paper conducts various analyses on the behavioral characteristics of the method that can give many insights for future work. Moreover, various experiments, which are conducted with considerations about the situation where the method is used in the real world, are also shown in this paper.