HYPOCRITE: Homoglyph Adversarial Examples for Natural Language Web Services in the Physical World
Authors: JINYONG KIM
JEONGHYEON KIM
MOSE GU
SANGHAK OHH
GILTEUN CHOI
JAEHOON JEONG
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial Examples
Homograph
Natural Language
Web Services
Physical World

Abstract: Recently, as Artificial Intelligence (AI) develops, many companies in various industries are trying to use AI by grafting it into their domains.
Also, for these companies, various cloud companies (e.g., Amazon, Google, IBM, and Microsoft) are providing AI services as the form of Machine-Learning-as-a-Service (MLaaS).
However, although these AI services are very advanced and well-made, security vulnerabilities such as adversarial examples still exist, which can interfere with normal AI services.
This paper demonstrates a HYPOCRITE for hypocrisy that generates homoglyph adversarial examples for natural language web services in the physical world. This  hypocrisy can disrupt normal AI services provided by the cloud companies.
The key idea of HYPOCRITE is to replace English characters with other international  characters that look similar to them in order to give the dataset noise to the AI engines.
By using this key idea, parts of text can be appropriately replaced with subtext with malicious meaning through black-box attacks for natural language web services in order to cause misclassification.
In order to show attack potential by HYPOCRITE, this paper implemented a framework that makes homoglyph adversarial examples for natural language web services in the physical world and evaluated the performance under various conditions.
Through extensive experiments, it is shown that HYPOCRITE is more effective than other baseline in terms of both attack success rate and perturbed ratio.

Under review as a conference paper at ICLR 2022
HYPOCRITE: HOMOGLYPH ADVERSARIAL EXAM- PLES FOR NATURAL LANGUAGE WEB SERVICES IN THE PHYSICAL WORLD
Anonymous authors Paper under double-bling brindd review
ABSTRACT
Recently, as Artificial Intelligence (AI) develops, many companies in various in- dustries are trying to use AI by grafting it into their domains. Also, for these com- panies, various cloud companies (e.g., Amazon, Google, IBM, and Microsoft) are providing AI services as the form of Machine-Learning- as --svvier (MLaaS). However, although these AI services are very advanced and well-made, security vulnerabiliities such adversarial examples still exist, which interfere with normal AI services. This paper demonstrates a HYPOCRITE for hypocrisy that generates homoglyph adversarial examples for natural language web services in the physical world. This hypocrisy can disrupt normal Al services provided by the cloud companies. The key idea of HYPOCRITE is replace English characters with other international characters that look similar to them in order to give the dataset noise the AI engines. By using this key idea, parts of text can be appro- priately replaced with subtext with malicious meaning through black-box attacks for natural language web services in order to cause misclassification. In order to show attack potential by HYPOCRITE, this paper implemented framework that makes homoglyph adversarial examples for natural language web services in the physical world and evaluated the performance under various conditions. Through extensive experiments, it is shown that HYPOCRITE more effective than other baseline in terms of both attack success rate and perturbed ratio.
INTRODUCTION
Artificial Intelligence (AI) has shown the potential of convenience in many domains. With the ad- vance of AI, people are living affluent lives by AI. Al can judge what is difficult for humans to make, classify what humans struggle with, predict what humans can never measure, and even recommend tasks that fall within a pattern (Naumov al. (2019)). Due to the development of the Al industry, it is not an exaggeration to say that mankind coexists with AI as many companies in various industries trying to use AI by grafting it into their domains. As the demand on AI increases, various cloud service providers (c.g., Amazon Comprehend (Amazon). Google Cloud Natural Language (Googlc) Watson Natural Language Understanding (IBM). and Text Analytics (Microsoft) are providing casy-to-use Machine-Lcarring--s-- -e-iiiiiiee-eee (Ribeiro et al.) (2015)) to people and companies who want to use Al services through their cloud. Among the MLaaS, Natural Language Processing (NLP) based on text is one of the important Al services.
NLP, which contains various information such emotional and semantic analysis of text-based data (Dang et al. 22000) Kamath & Ananthanarayana (2016)), can be used to develop platforms for various recommendation systems. For example, it is possible to provide effective data analysis to corporate management based on quick information delivery by identifying the needs of users and identifying only the core of system or long article based on the sentiment analysis ?sentiment- analysis) based the user's review. Like this various cloud service providers' MLaaS superiority is sufficiently proven through many studies.
However, although these AI services are very advanced and well-made, security vulnerabilities still existing. Because these security vulnerabilities can interfere with normal Al services, so cause fatal problem, the integrity of such services should be protected. This paper shows the security

---

Under review as a conference paper at ICLR 2022
vulnerabilities for natural language web services in the physical world (Rodriguez al. (019 The key idea of the adversarial examples( Goodfellow et al. (2014); Creswell et al. (2018)) for natu- ral language web services is to replace English characters with other similar international characters (e.g., homoglyph) in order to give the dataset noise (Boucher et al. (02T)). ). By using this key idea, parts of text can be appropriately replaced with subtext with malicious meaning through black-box
attacks[lyaa et al. (2018)) for natural language web services in order to cause misclassification.
The main contributions of this paper are summarized as follows:
Text adversarial examples for natural language web services in the Physical World: In order to show the feasibility of our attack, we implemented a framework that can gener- ate text adversarial examples for natural language web services in the physical world (see Section 3
Untargeted attacks and targeted attacks: For various goals of the adversarial attacks, we carried out the text adversarial attacks for not only non-targeted attacks (i.e., misclassifica- tion) but also targeted attacks (i.e., targeted misclassification and source/target misclassifi- cation) (see Section
The performance evaluation of the proposed framework: Through extensive experi- ments, it is shown that the proposed framework outperforms a baseline framework in terms of both attack success rate and perturbed ratio (see Section 4
The impact of human understanding: To evaluate the attack text generated by our pro- posed adversarial attack model, we used Amazon Mechanical Turk (Mturk) how difficult it is to find the attacked word, conducted survey using the attack text and obtained and analyzed the success rate of the attack on the survey problem (see Section 4.2.2).
The remainder of this paper is organized as follows. The background and related work of text adver- sarial examples is given in Section 2 Section describes descrienss the overview of the proposed adversarial attack and explains the process of the text adversarial attack of generating text adversarial exam- ples for natural language web services in the physical world. Section 4 evaluates the performance of the our proposed framework through misclassification attacks for sentiment analysis of natural language web services in the physical world. Section 5 discusses some research challenges for our attack. Finally, Section 6 concludes this paper along with future work.
2 RELATED WORK
Research on Adversarial attacks for NLP models has been presented. In 2016, (Papernot et al. (2066) proposes a method to craft a sequential input on Recurrent Neural Network (RNN) mod- els to manipulate an output of classifiers. (Ebrahimi et al. (2017) presents method to generate adversarial examples for text classification by crafting a few characters of an input string. Unlike (Ebrahimi et al. (2017)) whose attacks were white-box adversarial examples, (Gao et al. (2018) used black-box adversarial text sequence to make deep learning-basee based classifiers misclassily. (Li et al. (2018) shows various methodologies to generate adversarial text for NLP models, and eval- uates that popular NLP services for web services are vulnerable to those attacks. In our work, we show new methodology generate adversarial text which is not considered in (Li et al. (2018). ).
and also show that most of the NLP services are still vulnerable to our attack. (Wolfi & Wolff (2020)
proposes homoglyph attacks generating adversarial examples neural text detectors. Our attacks are targeted to sentimental analysis services in the real world, and we try to perturb every unit of target text (c.g., word, sentence, and paragraph) instead of replacing several letters with homoglyphs in order to show the most effective way to generate adversarial examples.
3 ATTACK MODEL
This section presents the goal, overview, and attack process of HYPOCRITE. The goal of HYP- OCRITE is to generate text adversarial examples for a sentiment analysis of natural language web services in the physical world. In other words, the generated adversarial examples cause mis- classification from positive sentiment to negative sentiment or from negative sentiment to positive sentiment.
2

---

Under review as a conference paper at ICLR 2022
Original Text
This film really bad, bad that even Christopher Lee cannot save it. poor story an even poorer script and just plain bad direction makes this truly outstanding horror film, the outstanding being that is the only horror fimm that can honestly say would never ever watch again. This garbage make Plan nine from outerspace look like oscar material.
Amazon Comprehend (Amazon)
Original Result: Negative (0.9986)
Adversarial Example Result: Positive (0.7041)
This film really bad, so bad that even Christopher Lee cannot save it. A poor story an even poorer script and just plain bad direction makes this truly outstanding horror film, the outstanding part being that is the only horror film that ican honestly say would never ever watch again. This garbage make Plan nine from outerspace look like oscar material.
Google Cloud Natural Language Al (Google)
Original Result: Negative (0066)
Adversarial Example Result: Positive (0.4)
This film really bad, so bad that even Christopher Lee cannot save A poor story an even poorer script and just plain bad direction makes this truly outstanding horror film, the outstanding being that is the only horror film that can honestly say would never ever watch again. This garbage make Plan nine from outerspace look like oscar material.
Watson Natural Language Understanding (IBM)
Original Result: Negative (0.877)
Adversarial Example Result: Posttiee (0.3055)
This film is really bad, so bad that even Christopher Lee cannot save it. A story an even poorer script and just plain bad direction makes this a truly outstanding horror film, outstanding part being that is the only horror film that honestly say would never ever watch again. This garbage make Plan nine from outerspace look like oscar material.
Text Analytics (Microsoft)
Original Result: Negative (1.0)
Adversarial Example Result:
Posilive (0.58)
This film really bad, so bad that even Christopher Lee cannot save A poor story an even poorer script and just plain bad direction makes this truly outstanding horror film, the outstanding being that is the only horror limm that can honestly say would never ever watch again. This garbage make Plan nine from outerspace look like oscar material.
The homoglyph adversarial examples for sentiment analysis web services (i.e., Amazon comprehend, Google cloud natural language AI, IBM Watson natural language understanding, and Microsoft text analytics) generated by HYPOCRITE
Figure 1:
3.1 HYPOCRITE OVERVIEW
In this subsection, the proposed HYPOCRITE for generating homoglyph adversarial examples is deseribed. The key idea of the HYPOCRITE is to replace English characters with other similar international characters (i.e., homoglyph) in order to give dataset noise. Such dataset noise cause misclassification different from the original results. This is because the original meaning disappears due the noise. With this key idea, HYPOCRITE can generate homoglyph adversarial examples by appropriately changing text from a word unit to a paragraph through a black-box attacks for natural language web services. The homoglyph adversarial examples mean text that looks the same to the human, but causes different results through Al services. Figure [ shows the four homoglyph adversarial examples for sentiment analysis web services (i.e., Amazon comprehend, Google cloud natural language AI, IBM Watson natural language understanding, and Microsoft text analytics) generated by HYPOCRITE, As homoglyph adversarial examples, characters with red color and bold font mean homoglyph that looks the same to our eyes but has different Unicode values. As shown in Figure T although they look like the same text to perception, the result of the sentiment analysis web service is different from positive to negative, respectively. The user study on the perception of human for adversarial examples is explained in detail in Section 4.2.2 and the score for each MLaas company's sentiment analysis result is explained in detail in Section 4.
3

---

end Under review as a conference paper at ICLR 2022
3.2 ADVERSARIAL EXAMPLE GENERATION
Algorithm 1: Nar-Ttree Adversarial Ex-
Algorithm 2: Targeted Adversarial Example
ample Generation
Generation
resulltent < get sentiment(target model, tear);;
result get.eetimennttrrgt.mmddd.
text);;
wnits + [word, sentence, sentence.paragrappll
wnitss < wword, sentencee,aragraaae
for unil umits do
for unit units do
tokens + tokenize(resultteas, unit); for token E tokens do
tokens + tokenize(resulleent unit); for token tokems do
resultiaken +
resulttoken +
get sentiment((argel model, token);
get.sentimen((argget.mdd
loken);
if min aeii_sentimeenrresutttaa not original sentiment then
if max sentiment( resullzoken) is not
target sentiment then
Append resultzoken
Append resultzoken to attackers:
attackers;
end end
end end
attackers + Sor((atacckess) according to descending of original sentiment;
attackers + Sort(attackers) according ascending of target sentiment;
for resultioken attackers do
for resullaoken E attackers do AE f- make_ade www.gvvearrileeeappperrul.leeeee resulttoken):
AE
make adwersarial..xamppeeee.ee example(resulteest resulttoken);
resultAE +
result +
get pantiment((rrgel.moll AE); if sentiment of result is changed then
set.settieenttarge.mmmm AE);
sentiment of result AE is target sentiment then return AE;
return AE;
else
else
if original sentiment score of result less than original sentiment score of resultiezt then
dif ftarget +- target sentiment score of result target sentiment score resulttents diffmaz max sentiment score of result max sentiment score of resultrezti
resulltezt +- result end
end
di Starget is more than 0 and di f ftarget more than di f fmnr then resultiezt +- result end end
end
end
end
This subsection describes the process of our HYPOCRITE that generates homoglyph adversarial examples for sentiment analysis web services in the physical world. The adversarial example gener- ation is classified into non-targeted adversarial example generation and targeted adversarial ex- ample generation. The non-targeted adversarial example generation means to generate homoglyph adversarial examples that misclassify a positive sentiment into sentiments other than the positive sentiment, or negative sentiment into sentiments other than the negative sentiment. The targeted adversarial example generation means to generate homoglyph adversarial examples that misclas- negative
sify a positive sentiment into target sentiments (e.g.. negative, neutral, and mixed), sentiment into target sentiments (e.g.. positive, neutral, and mixed). Algorithms [ and show an adversarial example generation algorithm of the HYPOCRITE. As shown in Algorithms and 2 for the adversarial example generation, HYPOCRITE consists of five steps: (i) Get score of original text (i) Tokenize the text, (iii) Get score of the tokens and sorting, (iv) Make adversarial examples,

---

Under review as a conference paper at ICLR 2022
and (v) Verify the effectiveness of the adversarial examples. Through this process, the non-targeted and targeted adversarial examples are generated, respectively.
4 EXPERIMENTS
4.1 EXPERIMENTAL-SETT
Dataset We used IMDB Dataset to evaluate our attacks on sentimental analysis (Maas et al. (201 T) IMDB contains 25,000 reviews for each positive and negative label, respectively. In our experiments, to determine the effect of text length on the success of the adversarial example, the dataset used in the experiment was divided into seven sections by length. The first section is 500 characters less, and the second section is 500 characters or more and 800 characters or less. The third section is 800 characters more and 1,100 characters less, and the fourth section is 1,100 characters more and 1,400 characters or less. The fifth section is 1,400 characters or more and 1,700 characters or less, and the sixth section is 1,700 characters more and 2,100 characters less. The last seventh sections are over 2,100 characters. Then, we randomly sampled 96 review data per divided section. The number of sample reviews (96 reviews) in our evaluation was determined by a statistically recommended sample size when the confidential level is 95%, the population size is 50,000, and the margin of error is 10%.
Targeted Models
To evaluate our attacks, we per- formed the attacks on four senti- mental analysis services (Amazon, Google, IBM, and Microsoft) in real world. Since every service had dif- ferent sentimental labels and scoring systems, we briefly described label- ing and scoring metric for each sys- tem in Table
Label
Score metric
Decision
Amazon
Prsitive, Negutive, Neatral, Mixed
(Tor each label)
Score <1
(The biggest score)
Label
Google
<Score
Score
TToal seorc)
IBM
Posilive, Negutive, Neutral
*1 <Score <
(Total
Score
<Seore <1
Label
Microsoft
Positive, Negative, Neutral, Mixed
[For each label)
(The biggest sourc)
Four targeted models. Label stands for labels pro- vided by each system, Score metric for sentimental score used by the system, and Decision for final output provided by the system.
Table I:
Amazon and Microsoft use four sen- timental labels (Positive, Negative, Neutral, and Mixed), and the systems provide each score for all the labels. In addition, the final output as the re- sult is the label which is the biggest score among the sentimental labels.
On the other hand, IBM uses three sentimental labels (i.e., Positive, Negative, and Neutral) and Google dose not provide any exact label. Both systems provide overall score of the input texts as final output ranging from to 1 (The score closer means negative, and means positive). As there is no sentimental decision provided from the systems, we considered the score less than 0 as negative, and more than as positive.
To compare the performance of our attack, we evaluate our attack with an attack which the most similar method to ours (Wolff & Wolff (2020)). In (Wolff & Wolff (2020), the attack is targeted to the neural text detectors. Therefore, performed the baseline attack to our targeted systems and measure the performance of the attack.
Baseline
Evaluation Metrics In order to evaluate the algorithmic performance, we adopt two metries such as attack success rate and perturbed rate. The average attack success rate is defined as the ratio of generated adversarial examples that cause misclassification to randomly selected samples. In order to show diversity, we evaluated the performance under various misclassification conditions such as non-targeted misclassification and targeted misclassification. The perturbed rate is defined as the ratio of replaced characters to the total characters throughout the sentence.
4.2 RESULTS
This section contains a summary of the performance of a targeted attack and non-targeted attack through the black-box attack against four different platforms such Amazon, Google, IBM, and
5

---

Under review as a conference paper at ICLR 2022
Web Service Platform
Attack Model
Source
Destination
HYPOCRITE
HOMOGLYPH
Suecess Rate (%)
Perturbed Rate (%)
Success Rate (%)
Perturbed Ratel %)
Non-targeted
5.64
68.75
15.52
Negative
86,46
6.61
27.23
15.37
Positive
Neutral
92.26
17.62
37.05
15.31
Amazon
Mixed
39.67
13.41
7.29
15.09
Non-largeted
98.2
13.05
27.08
15.22
Negative
Positive
47.47
11.41
2,98
12.6
Neutral
81.75
17.98
18.15
13.07
Mixed
10.65
2.83
13.14
Non-targeted
99.40
7.24
78.27
6.62
Positive
Negative
14.52
8.41
Google
Neutral
99.40
5.92
77.98
6.63
Nou-targeted
100.00
5.26
55.65
6.75
Negative
Positive
94,49
15.64
6,70
8.26
Neutral
99.85
5.92
55.80
6.68
Non-targeted
99.55
8.81
61.90
15.33
Positive
Negative
98.81
8.21
58.04
15.38
Neutral
22.77
47.62
4.32
13.82
IBM
Non-targeted
98 22
11.78
7.44
13.23
Negative
Positive
12.98
Neutral
22.02
46.24
1,34
10.68
Non-targeted
15.34
Positive
Negative
Neutral
64.22
15.34
18.89
21.49
15.68
15.27
Microsoft
Mixed
80.79
2.39
21.75
14.91
Nou-targeted
99.85
5.36
33.78
15.28
Negative
Posilive
96.43
21.73
3.57
10.82
Neutral
92.28
36.
4.91
12.89
Mixed
88.09
3.87
25.89
14.92
Table Overall performance evaluation of our attack model and baseline attack model
HYROCRITE HOMOGUPPH
HYPOCRITE HOMOCLFPH
5
IMPOCAIT 1OMDDGYRI
IMPOCRITE HDMM0G1991
12
(a) Pos Non
(b) Pos to Neg
(c) Neu
(d) Pos Mix
Text Langth
Text Langth
Tax: Length
Toxt Longo
(e) Neg to Non
(1) Neg to Pos
(g) Neg to Neu
(h) Neg to Mix
Figure 2: The impact of the Amazon web service platform according to non-targeted and targeted
adversarial attacks
Microsoft. Table shows the successful attack rate and perturbed rate each attack. Broadly, our attack is divided into types such targeted attack and non-targeted attack. Non-targeted attack means misclassifying the original sentiment classification into another classification other than the existing one. On the other hand, the targeted attack is perturb a positive or negative classification so that it may be perceived as the specific destination. As shown in Table although the perturbation rates are similar, HYPOCRITE generally performs higher than the baseline (i.e., HOMOGLYPH), We will describe the impact of Web service platform in the next subsection in detail.
4.2.1 THE IMPACT OF WEB SERVICE PLATFORM
Amazon This section demonstrates the performance of adversary success rate for Amazon. As shown in Figure shown, HYPOCRITE generally shows better performance than HOMOGLYPH on target and non-target attacks. To be detailed, positive (i.e., non-targeted attack) attack,

---

Under review as a conference paper at ICLR 2022
Aap. Abaut Sqoccia hale 100
2ap AED Seo0ss Patt Ph
Bay Arr SeOCI5 hate 1%0
X
IMPOCTITE
HICNCGL/HH
-0-
HMPOCIITE HONDAYYHI
APPOOCRITE HOMONGIVH
Text Length
Text Length
Text Length
(a) Pos to Non
(b) Pos to Neg
(c) Pos to Neu
Avg ACTOc SUOCIIS RATO [N)
Ang 2c.00 Succcs PL00
ACOOk Secccs AJ00
HYPOIIITE
HONDCL/TH
IYPOCHITE IICNGGLPHI
IMPOORITE HOMOGLYH
(d) Neg Non
(e) Neg to Pos
(l) Neg to
Figure 3: The impact of the Google web service platform according to non-targeted and targeted
adversarial attacks
20g Aackk Sapccsa Fulle [N)
MCOLI SAUCIS Puloo PH0
4C23 SAUCISS PLtTO
HYPOCRITE HOMODIYPHI
HOMOCEFPH
A/9
Avg
That Length
Teat length
Text Lengeh
(a) Pos to Non
(b) Pos to Neg
(c) Pos to Neu
Aog. 2ctock Succes Ral0 (5)
DeU SEOOR SuccOs Ro [4]
Ael JEDOR Succes Ra10 N4
IMPOCTITE HCMOCLYHI
HYPOCMTE
HYPOCNIT
HONOOLTTM
HOMOGL/PH
Teat Length
Text Length
(d) Neg Non
(e) Neg Pos
(f) Neg to Neu
Figure 4: The impact of the IBM web service platform according to non-targeted and targeted adversarial attacks
when HOMOGLYPH performed up to 68.75% HYPOCRITE performed 100%. Both models' per- formances of negative to non are lower than positive to non. When HOMOGLYPH performed 27.08 percent, HYPOCRITE still performed superior success rate of 94.20 percent. In targeted attack cases, except for negative to positive, positive to mix, and negative to mix, comparing with HO- MOGLYPH showed less than 40 percent, HYPOCRITE performed significantly high performance. Both models are having low performance on negative to positive, positive to mix, negative to mix but HYPOCRITE is showing significantly higher performance than HOMOGLYPH about 20 times difference. Both models' impact changes belonging to length dependency were hardly be found.
Google This section consists of performances attacks against Google. Figure F shows the impact of the text length in Google web service platform. As shown in Fig. 3 since Google API does not provide for targeted attacks with results of mixed, there are only two scenarios for each targeted attack (positive or negative to opposite and neutral). In case of non-targeted attack, HYPOCRITE) is showing utmost performance with approximate 100% performance and in contrast, HOMOGLYPH in Google is showing about performance of 66.96%. In targeted attack case, similar to non-targeted attack case, HYPOCRITE also shows good performance OCRITE appears relatively low performance in positive to negative, it still performs 2 to times higher than HOMOGLYPH. Lastly, within the text length perspective, HOMOGLYPH performed decreased with length, HYPOCRITE shows consistently high performance regardless of length.
target attack scenarios. Although HYP-

---

Under review as a conference paper at ICLR 2022
Honck (50
IMPOOCRITE HIDMOOLOPH
LIYROCRITE JIONOGUPHI
40-
2
IMPOCAITE
IOMOGIYAI
Length
Text Length
Text Length
Text Length
(a) Pos Non
(b) Pos Neg
(c) Pos Neu
(d) Pos Mix
IYPOCIITE HCNDOLIPH
IMPOERITE
IMPOCTITE HCMOCLYM1
IMPOCRITE HDMOSLU911
12
HDNCCL//H1
(e) Neg to Non
(1) Neg to Pos
(g) Neg to Neu
(h) Neg to Mix
Figure 5: The impact of the Microsoft web service platform according non-targeted and targeted
adversarial attacks
IBM This section shows another performance against IBM through figure 4 As shown in Figure 4 there is a significant different between two model in non-targeted attack. While HOMOGLYPH is showing intermediate performance in positive to non, shows fairly low performance. In contrast, HYPOCRITE shows fairly high performance attack case, unusual result can be found. There were no significant differences compared other platforms within the result of positive to negative and negative to positive that HYPOCRITE demon- strated better performance than HOMOGLYPH. Yet, unlike other web service platforms, both HYP- OCRITE and HOMOGLYPH show poor performance in positive to neutral and negative to neutral. Indeed, during our experiments, the original text was not classified as neutral by IBM. Nevertheless, the baseline showed result close to 0%, whereas HYPOCRITE showed an average of 22.3%.
both scenarios of non-targeted attack. In targeted
Microsoft This section demonstrates the performance of adversary attack against Microsoft. As shown in Figure [S] HYPOCRITE shows eminently better performance than HOMOGLYPH on target and non-target attacks. In fact, on positive to non (i.e., non-attack, while HOMOGLYPH performed up to 64.22 percent, HYPOCRITE performed 100 percent. Even though HOMO- GLYPH performed 33.78 percent on negative to non, HYPOCRITE performed 99.85 percent which is near to 100 percent. In targeted attack cases, HYPOCRITE showed generally high success rates that are above 90 percent except positive to mix and negative to mix. Com- pare that, HOMOGLYPH'S highest performance only 25.89 percent. Even HYPOCRITE showed less performance on negative to mix and positive to mix, it still performed above 80 percent when HOMOGLYPH only performed around 20 percent. The reason why the perfor- mance of the attack targeting mixed is lower than other attacks is that the shorter text length provokes the lower performance therefore, the overall success rate of HYPOCRITE decreases.
4.2.2 USER STUDY
We conducted survey of homoglyph adversarial exam- ples using Amazon Mechanical Turk (Mturk) to evaluate the effectiveness of our attack model, and we asked public users to find the words that are changed by homoglyph.
In addition, in order to investigate the effect of text length on the success of the adversarial attack, the IMDB dataset used in the experiment was divided into 7 groups by length. The survey was conducted with random questions selected from each group.
Dess
Toxt Longth
Figure 6: The user study for perception of the homoglyph adversarial examples generated by HYPOCRITE
To calculate the success rate the adversarial example of our model, defined the attack success rate by quan-
8

---

Under review as a conference paper at ICLR 2022
tifying the number of attack words found by the experi- mental participants in the given text. Figure shows the specific average attack success rate of expriment partici- pants per platform and the average attack success rate per text length group. Looking Figure the longer the text is given the participant, the higher the attack success rate. In particular, the first group had the lowest at 93.63%, and the seventh group with the longest text length was the highest at 98.33%. This shows that the shorter the target of an adversarial attack, the easier it is for users to recognize the attack.
Through a user study, it was found that the adversarial attack text made by our proposed model was difficult to distinguish with the human eye, so even though it was actually an adversarial attack, it was very difficult for users to recognize.in the user study we experimented with, after informing the user that the word was wrong, we asked the user to look for the wrong or strange word, but in real life (i.e., the situation where the mistake is not told in advance)even if there is a mistake in the word, it is expected that the success rate of the attack targeting the user will be further increased by the word superiority effect (Baron Thurston 1973)) that unconsciously recognizes the word correctly.
5 RESEARCH CHALLENGES
Effectiveness of perturbed letters in every word In our work, we perturbed the whole words to generate adversarial examples. However, every single letter in the words could be perturbed with letter which looks the same but has different unicode value. From the results in WWolff & Wolff (020) that replacing all the vowels to generate the adversarial examples was the most effective attack, there might be significant pattern in letter perturbation of the words to make the adversarial attack more effective. In other words, when the letter which has significant effect on the success of the attack, the attack will be more effective.
The financial cost to evaluate the results To test the performance our attacks, we used sentiment analysis APIs provided by real world companies. However, every time we checked the results using APIs, the use APIs should be payed. Therefore, we had to sample a part of the whole dataset, and also could not try the further experiments which deal with perturbing letters to find which letters important to the score of sentimental analysis.
CONCLUSION
In this paper, we studied adversarial attacks using homoglyph against natural language web services in the physical world. The experimental results demonstrate that our HYPOCRITE) is more effective than other baseline in terms of both attack success rate and perturbed ratio for four popular web service platforms. Also, through the user study, we showed our homoglyph adversarial examples was difficult for users to recognize as a perturbed text. As future work, we will extend the field of our HYPOCRITE to attack to various web services with natural language in addition to sentiment analysis.
REFERENCES
Amazon. Amazon comprehend. URL htttp:///waa amazon wwt.:coppeeecc//cccc.ccmoo
Jonathan Baron and Ian Thurston. An analysis of the word-superiority effect. Cognitive psychology, 4(2):207-228, 1973.
Nicholas Boucher, Ila Shumailov, Ross Anderson, and Nicolas Papernot. Bad characters: Imper- ceptible nlp attacks. arXiv preprint arXiv:2106.09898 2021.
Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. Generative adversarial networks: An overview. IEEE Signal Processing Magazine, 35 (1):53-65, 2018.
Nhan Cach Dang, María N Moreno-Garcie and Fernando De la Prieta. Sentiment analysis based on deep learning: A comparative study. Electronics, 9(3):483, 2020.

---

Under review as a conference paper at ICLR 2022
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:/712.067551 2017.
Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pp. 50-56. IEEE, 2018.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Google. Google cloud natural language.
URL http:://aLoud.ggoglee..oo
nnturral-annuage
IBM. Watson natural language understanding.
URL
https://www: http://wwwwmmmcmmmmmo/o/o//
watson Lelssonnnatural--- htg.:o/otttuuu--aauugggueeeeeeaa
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference on Muchine Learning, pp. 2137- 2146. PMLR, 2018.
S Sowmya Kamath and VS Ananthanarayana. Semantics-based web service classification using morphological analysis and ensemble learning techniques. International Journal of Data Science and Analytics, 2(1):61-74, 2016.
Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-worke applications. arXiv preprint arXiv: 2018.
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng. and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pp. 142-150, 2011.
Microsoft. Text analytics.
Mturk. Amazon mechanical turk. URL BRRLhttps:// wttp:://wwwwmttukkkkkk com/
Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundara- man, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson Azzolini, et al. Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv: 990000990000001, 2019.
Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. Crafting adversarial input sequences for recurrent neural networks. In MILCOM 2016-2016 IEEE Military Communi- cations Conference, pp. 49-54. IEEE, 2016.
Mauro Ribeiro, Katarina Grolinger, and Miriam AM Capretz. Mlaas: Machine learning as a service. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 896-9022 IEEE, 2015.
Carlos Rodriguez, Shayan Zamanirad, Reza Nouri, Kirtana Darabal, Boualem Benatallah, and Mor- tada Al-Banna. Security vulnerability information service with natural language query support. In International Conference Advanced Information Systems Engineering, pp. 497-512. Springer, 2019.
Max Wolff and Stuart Wolff. Attacking neural text detectors. arXiv preprint urXiv:2002.11760 2020.
10


Reviews:
[{"review_id": "oTlQdAVQZ7w", "reviewer": "Reviewer_JKRL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n1. Interesting problem and impressive results.\n\nCons:\n\n1. Limited evaluation.\n2. Incomplete discussion/comparison with prior art, including https://arxiv.org/pdf/2104.05996.pdf\n\n\n", "summary_of_the_paper": "The authors propose the injection of human imperceptible unicode characters that induce evasion effects in neural based text recognition platforms. ", "summary_of_the_review": "Detailed Comments:\n\n1. The comparison with Boucher et al. is unfair and mostly incomplete. The authors need to better position their work w.r.t prior art to highlight their contributions. \n2. Work by Boucher et al. suggests that the proposed attack can be used for a variety of different tasks apart from sentiment detection. Can the authors try the same?\n3. The authors do not evaluate any defense methodologies against these attacks. Can the authors comment on the impossibility of defenses? For example, what if one were to convert the string to remove all unicode characters (as a sanitization step)?\n4. There’s no evaluation to suggest that the modified strings would fool a human. Was there any MTurk study performed to support these claims?\n5. What are some other use cases of the purported attack? Can the authors comment on this? ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "niMPwkaKd-W", "reviewer": "Reviewer_QX9M", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe paper conducts experiments on real-world industry platforms to showcase the performance. The paper shows good empirical results on attack success rate on the APIs.\n\nWeaknesses:\nThe novelty of the proposed method is not demonstrated very well in the paper. The algorithms described in section 3.2 are basically doing brute force searching with the sorting on the sentiment scores queried from the APIs. It is unclear how much technical contribution this method provides.\nIn the experimental comparison section, the paper only shows the results on the attack success rate and perturbed rate but does not mention the number of queries used by each method. As the authors said in the paper, the queries to the APIs need to be paid and are expensive, the query number is actually a very important metric to show. My intuition is that since the proposed method is a pretty straightforward search, the query number should be high. \n\nMinor:\n“based on the sentiment analysis ?sentiment-analysis) based on the user’s review” on page 1.\n", "summary_of_the_paper": "The paper proposes an attack method that generates homoglyph adversarial examples for NLP APIs. The method replaces English characters with other international characters that look similar. The paper shows empirically that the method achieves good performance for several real-world APIs.\n", "summary_of_the_review": "The technical novelty of the proposed method is limited. The empirical evaluations are not solid enough to demonstrate significant contributions.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "00TySmBeIP", "reviewer": "Reviewer_y6Cv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is very poorly written and hard-to-follow, full of grammar and language errors. Asides from this, I'm having hard time understanding the contribution of this work over Boucher et al.'s \"Bad‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌ Characters: Imperceptible NLP Attacks\" paper, which has been around for more than 4 months now. Boucher et al.'s paper both investigate homoglyph attacks and more (like the invisible characters) with better depth and they also propose some easy countermeasures against these attacks (which would prevent the proposed attacks).", "summary_of_the_paper": "This paper proposes using homoglyphs to attack commercial NLP models for sentiment classification. Homoglyphs look like the characters in English language but they are encoded differently and, therefore, treated differently by the model. They experiment on various MLaaS models and show that their attack can effectively cause both targeted and untargeted misclassifications.", "summary_of_the_review": "- The contributions over \"Bad‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌ Characters: Imperceptible NLP Attacks\" paper is not clear at all, the authors cite this work but have not evaluated as a baseline or discussed the differences of their work.\n\n- Poorly written, makes reviewing painful, I would recommend revising, reorganizing and proofreading. For example, the sentence in the abstract, \"This  hypocrisy can disrupt normal AI services provided by the cloud companies.\", here, I have never seen the word \"hypocrisy\" instead of the word \"attack\", it seems like an effort to paraphrase or use translation software from another language.\n\n- No mention of easy and trivial countermeasures against this attack.\n\n- Evaluating transferability of the attacks between different services would be useful. The authors complained about the API costs, what if the attacker crafts the adversarial example against a local sentiment classification model and submits it to one of the APIs? Currently, it is not clear how much each adversarial example costs and judging by the algorithm, it might be restrictive.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents attacks against sentiment recognition NLP systems using specially crafted homoglyphs. The novelty of the proposed method is, however, marginal; contributions over some of the related work are unclear. Furthermore, the quality of writing is insufficient. The authors provided no response to reviewers' comments.
Decision: Reject