HYPOCRITE: Homoglyph Adversarial Examples for Natural Language Web Services in the Physical World

1 INTRODUCTION . Artificial Intelligence ( AI ) has shown the potential of convenience in many domains . With the advance of AI , people are living affluent lives by AI . AI can judge what is difficult for humans to make , classify what humans struggle with , predict what humans can never measure , and even recommend tasks that fall within a pattern ( Naumov et al . ( 2019 ) ) . Due to the development of the AI industry , it is not an exaggeration to say that mankind coexists with AI as many companies in various industries are trying to use AI by grafting it into their domains . As the demand on AI increases , various cloud service providers ( e.g. , Amazon Comprehend ( Amazon ) , Google Cloud Natural Language ( Google ) , Watson Natural Language Understanding ( IBM ) , and Text Analytics ( Microsoft ) are providing easy-to-use Machine-Learning-as-a-Service ( Ribeiro et al . ( 2015 ) ) to people and companies who want to use AI services through their cloud . Among the MLaaS , Natural Language Processing ( NLP ) based on text is one of the important AI services . NLP , which contains various information such as emotional and semantic analysis of text-based data ( Dang et al . ( 2020 ) ; Kamath & Ananthanarayana ( 2016 ) ) , can be used to develop platforms for various recommendation systems . For example , it is possible to provide effective data analysis to corporate management based on quick information delivery by identifying the needs of users and identifying only the core of a system or long article based on the sentiment analysis ? sentimentanalysis ) based on the user ’ s review . Like this various cloud service providers ’ MLaaS superiority is sufficiently proven through many studies . However , although these AI services are very advanced and well-made , security vulnerabilities are still existing . Because these security vulnerabilities can interfere with normal AI services , so cause a fatal problem , the integrity of such services should be protected . This paper shows the security vulnerabilities for natural language web services in the physical world ( Rodriguez et al . ( 2019 ) ) . The key idea of the adversarial examples ( Goodfellow et al . ( 2014 ) ; Creswell et al . ( 2018 ) ) for natural language web services is to replace English characters with other similar international characters ( e.g. , homoglyph ) in order to give the dataset noise ( Boucher et al . ( 2021 ) ) . By using this key idea , parts of text can be appropriately replaced with subtext with malicious meaning through black-box attacks ( Ilyas et al . ( 2018 ) ) for natural language web services in order to cause misclassification . The main contributions of this paper are summarized as follows : • Text adversarial examples for natural language web services in the Physical World : In order to show the feasibility of our attack , we implemented a framework that can generate text adversarial examples for natural language web services in the physical world ( see Section 3 ) . • Untargeted attacks and targeted attacks : For various goals of the adversarial attacks , we carried out the text adversarial attacks for not only non-targeted attacks ( i.e. , misclassification ) but also targeted attacks ( i.e. , targeted misclassification and source/target misclassification ) ( see Section 3 ) . • The performance evaluation of the proposed framework : Through extensive experiments , it is shown that the proposed framework outperforms a baseline framework in terms of both attack success rate and perturbed ratio ( see Section 4 ) . • The impact of human understanding : To evaluate the attack text generated by our proposed adversarial attack model , we used Amazon Mechanical Turk ( Mturk ) how difficult it is to find the attacked word , we conducted a survey using the attack text and obtained and analyzed the success rate of the attack on the survey problem ( see Section 4.2.2 ) . The remainder of this paper is organized as follows . The background and related work of text adversarial examples is given in Section 2 . Section 3 describes the overview of the proposed adversarial attack and explains the process of the text adversarial attack of generating text adversarial examples for natural language web services in the physical world . Section 4 evaluates the performance of the our proposed framework through misclassification attacks for sentiment analysis of natural language web services in the physical world . Section 5 discusses some research challenges for our attack . Finally , Section 6 concludes this paper along with future work . 2 RELATED WORK . Research on Adversarial attacks for NLP models has been presented . In 2016 , ( Papernot et al . ( 2016 ) ) proposes a method to craft a sequential input on Recurrent Neural Network ( RNN ) models to manipulate an output of classifiers . ( Ebrahimi et al . ( 2017 ) ) presents a method to generate adversarial examples for text classification by crafting a few characters of an input string . Unlike ( Ebrahimi et al . ( 2017 ) ) whose attacks were white-box adversarial examples , ( Gao et al . ( 2018 ) ) used black-box adversarial text sequence to make deep learning-based classifiers misclassify . ( Li et al . ( 2018 ) ) shows various methodologies to generate adversarial text for NLP models , and evaluates that popular NLP services for web services are vulnerable to those attacks . In our work , we show a new methodology to generate adversarial text which is not considered in ( Li et al . ( 2018 ) ) , and also show that most of the NLP services are still vulnerable to our attack . ( Wolff & Wolff ( 2020 ) ) proposes homoglyph attacks generating adversarial examples to neural text detectors . Our attacks are targeted to sentimental analysis services in the real world , and we try to perturb every unit of target text ( e.g. , word , sentence , and paragraph ) instead of replacing several letters with homoglyphs in order to show the most effective way to generate adversarial examples . 3 ATTACK MODEL . This section presents the goal , overview , and attack process of HYPOCRITE . The goal of HYPOCRITE is to generate text adversarial examples for a sentiment analysis of natural language web services in the physical world . In other words , the generated adversarial examples can cause misclassification from positive sentiment to negative sentiment or from negative sentiment to positive sentiment . 3.1 HYPOCRITE OVERVIEW . In this subsection , the proposed HYPOCRITE for generating homoglyph adversarial examples is described . The key idea of the HYPOCRITE is to replace English characters with other similar international characters ( i.e. , homoglyph ) in order to give dataset noise . Such dataset noise can cause misclassification different from the original results . This is because the original meaning disappears due to the noise . With this key idea , HYPOCRITE can generate homoglyph adversarial examples by appropriately changing text from a word unit to a paragraph through a black-box attacks for natural language web services . The homoglyph adversarial examples mean text that looks the same to the human , but causes different results through AI services . Figure 1 shows the four homoglyph adversarial examples for sentiment analysis web services ( i.e. , Amazon comprehend , Google cloud natural language AI , IBM Watson natural language understanding , and Microsoft text analytics ) generated by HYPOCRITE . As homoglyph adversarial examples , characters with red color and bold font mean homoglyph that looks the same to our eyes but has different Unicode values . As shown in Figure 1 , although they look like the same text to perception , the result of the sentiment analysis web service is different from positive to negative , respectively . The user study on the perception of human for adversarial examples is explained in detail in Section 4.2.2 , and the score for each MLaaS company ’ s sentiment analysis result is explained in detail in Section 4.1 . 3.2 ADVERSARIAL EXAMPLE GENERATION . Algorithm 1 : Non-Targeted Adversarial Example Generation resulttext ← get sentiment ( target model , text ) ; units← [ word , sentence , paragraph ] ; for unit ∈ units do tokens← tokenize ( resulttext , unit ) ; for token ∈ tokens do resulttoken ← get sentiment ( target model , token ) ; if min sentiment ( resulttoken ) is not original sentiment then Append resulttoken to attackers ; end end attackers← Sort ( attackers ) according to descending of original sentiment ; for resulttoken ∈ attackers do AE ← make adversarial example ( resulttext , resulttoken ) ; resultAE ← get sentiment ( target model , AE ) ; if sentiment of resultAE is changed then return AE ; else if original sentiment score of resultAE is less than original sentiment score of resulttext then resulttext ← resultAE ; end end end end Algorithm 2 : Targeted Adversarial Example Generation resulttext ← get sentiment ( target model , text ) ; unitss← [ word , sentence , paragraph ] ; for unit ∈ units do tokens← tokenize ( resulttext , unit ) ; for token ∈ tokens do resulttoken ← get sentiment ( target model , token ) ; if max sentiment ( resulttoken ) is not target sentiment then Append resulttoken to attackers ; end end attackers← Sort ( attackers ) according to ascending of target sentiment ; for resulttoken ∈ attackers do AE ← make adversarial example ( resulttext , resulttoken ) ; resultAE ← get sentiment ( target model , AE ) ; if sentiment of resultAE is target sentiment then return AE ; else difftarget ← target sentiment score of resultAE − target sentiment score of resulttext ; diffmax ← max sentiment score of resultAE − max sentiment score of resulttext ; if difftarget is more than 0 and difftarget is more than diffmax then resulttext ← resultAE ; end end end end This subsection describes the process of our HYPOCRITE that generates homoglyph adversarial examples for sentiment analysis web services in the physical world . The adversarial example generation is classified into a non-targeted adversarial example generation and a targeted adversarial example generation . The non-targeted adversarial example generation means to generate homoglyph adversarial examples that misclassify a positive sentiment into sentiments other than the positive sentiment , or a negative sentiment into sentiments other than the negative sentiment . The targeted adversarial example generation means to generate homoglyph adversarial examples that misclassify a positive sentiment into target sentiments ( e.g. , negative , neutral , and mixed ) , or a negative sentiment into target sentiments ( e.g. , positive , neutral , and mixed ) . Algorithms 1 and 2 show an adversarial example generation algorithm of the HYPOCRITE . As shown in Algorithms 1 and 2 , for the adversarial example generation , HYPOCRITE consists of five steps : ( i ) Get score of original text ( i ) Tokenize the text , ( iii ) Get score of the tokens and sorting , ( iv ) Make adversarial examples , and ( v ) Verify the effectiveness of the adversarial examples . Through this process , the non-targeted and targeted adversarial examples are generated , respectively . 4 EXPERIMENTS . 4.1 EXPERIMENTAL-SETUP . Dataset We used IMDB Dataset to evaluate our attacks on sentimental analysis ( Maas et al . ( 2011 ) ) . IMDB contains 25,000 reviews for each positive and negative label , respectively . In our experiments , to determine the effect of text length on the success of the adversarial example , the dataset used in the experiment was divided into seven sections by length . The first section is 500 characters or less , and the second section is 500 characters or more and 800 characters or less . The third section is 800 characters or more and 1,100 characters or less , and the fourth section is 1,100 characters or more and 1,400 characters or less . The fifth section is 1,400 characters or more and 1,700 characters or less , and the sixth section is 1,700 characters or more and 2,100 characters or less . The last seventh sections are over 2,100 characters.Then , we randomly sampled 96 review data per divided section . The number of sample reviews ( 96 reviews ) in our evaluation was determined by a statistically recommended sample size when the confidential level is 95 % , the population size is 50,000 , and the margin of error is 10 % . Targeted Models To evaluate our attacks , we performed the attacks on four sentimental analysis services ( Amazon , Google , IBM , and Microsoft ) in real world . Since every service had different sentimental labels and scoring systems , we briefly described labeling and scoring metric for each system in Table 1 . Amazon and Microsoft use four sentimental labels ( Positive , Negative , Neutral , and Mixed ) , and the systems provide each score for all the labels . In addition , the final output as the re- sult is the label which is the biggest score among the sentimental labels . On the other hand , IBM uses three sentimental labels ( i.e. , Positive , Negative , and Neutral ) and Google dose not provide any exact label . Both systems provide overall score of the input texts as final output ranging from -1 to 1 ( The score closer to -1 means negative , and to 1 means positive ) . As there is no sentimental decision provided from the systems , we considered the score less than 0 as negative , and more than 0 as positive . Baseline To compare the performance of our attack , we evaluate our attack with an attack which is the most similar method to ours ( Wolff & Wolff ( 2020 ) ) . In ( Wolff & Wolff ( 2020 ) ) , the attack is targeted to the neural text detectors . Therefore , we performed the baseline attack to our targeted systems and measure the performance of the attack . Evaluation Metrics In order to evaluate the algorithmic performance , we adopt two metrics such as attack success rate and perturbed rate . The average attack success rate is defined as the ratio of generated adversarial examples that cause misclassification to randomly selected samples . In order to show diversity , we evaluated the performance under various misclassification conditions such as non-targeted misclassification and targeted misclassification . The perturbed rate is defined as the ratio of replaced characters to the total characters throughout the sentence .

Summary: The paper proposes an attack method that generates homoglyph adversarial examples for NLP APIs. The method replaces English characters with other international characters that look similar. The paper shows empirically that the method achieves good performance for several real-world APIs.