SlowGuess/NAACL_2022_OCR huggingface dataset:
Yong Xie $^{1}$ , Dakuo Wang $^{2,\dagger}$ , Pin-Yu Chen $^{2}$ , Jinjun Xiong $^{3}$ , Sijia Liu $^{2,4}$  and Sanmi Koyejo $^{1}$
<sup>1</sup>University of Illinois Urbana-Champaign, <sup>2</sup>IBM
$^{3}$ State University of New York at Buffalo,  $^{4}$ Michigan State University
More and more investors and machine learning models rely on social media (e.g., Twitter and Reddit) to gather real-time information and sentiment to predict stock price movements. Although text-based models are known to be vulnerable to adversarial attacks, whether stock prediction models have similar vulnerability is underexplored. In this paper, we experiment with a variety of adversarial attack configurations to fool three stock prediction victim models. We address the task of adversarial generation by solving combinatorial optimization problems with semantics and budget constraints. Our results show that the proposed attack method can achieve consistent success rates and cause significant monetary loss in trading simulation by simply concatenating a perturbed but semantically similar tweet.
The advance of deep learning based language models are playing a more and more important role in the financial context, including convolutional neutral network (CNN) (Ding et al., 2015), recurrent neutral network (RNN) (Minh et al., 2018), long short-term memory network (LSTM) (Hiew et al., 2019; Sawahney et al., 2021; Hochreiter and Schmidhuber, 1997), graph neutral network (GNN) (Sawahney et al., 2020a,b), transformer (Yang et al., 2020), autoencoder (Xu and Cohen, 2018), etc. For example, Antweiler and Frank (2004) find that comments on Yahoo Finance can predict stock market volatility after controlling the effect of news. Cookson and Niessner (2020) also show that sentiment disagreement on Stocktwits is highly related to certain market activities. Readers can refer to these survey papers for more details (Dang et al., 2020; Zhang et al., 2018; Xing et al., 2018).
wallstreetbet7821
raw tweet
Raw Tweet
BHP announces the demerger of its non-core assets - details expected to be filled in on Tuesday.
3:00 PM  $\cdot$  Aug 15, 2014  $\cdot$  Twitter for iPhone
wallstreetbet_adver
adver_retweet
Attack Retweet
Prediction: Negative
BHP announces the demerger of its non-core assets - details expected to be
exercised in on Tuesday.
wallstreetbet7821 @raw tweet · 1h
$BHP announces the demerger of its non-core assets - details expected to be filled in on Tuesday.
4:00 PM  $\cdot$  Aug 15, 2014  $\cdot$  Twitter for iPhone
Figure 1: An example of word-replacement adversarial attack. (Top) benign tweet leads Stocknet to predict stock going UP; (Bottom) adding an adversarial retweet leads Stocknet to predict stock going DOWN.
It is now known that text-based deep learning models can be vulnerable to adversarial attacks (Szegedy et al., 2014; Goodfellow et al., 2015). The perturbation can be at the sentence level (e.g., Xu et al., 2021; Iyyer et al., 2018; Ribeiro et al., 2018), the word level (e.g., Zhang et al., 2019; Alzantot et al., 2018; Zang et al., 2020; Jin et al., 2020; Lei et al., 2019; Zhang et al., 2021; Lin et al., 2021), or both (Chen et al., 2021). We are interested in whether such adversarial attack vulnerability also exists in stock prediction models, as these models embrace more and more human-generated media data (e.g., Twitter, Reddit, Stocktwit, Yahoo News (Xu and Cohen, 2018; Sawhney et al., 2021)). The adversarial robustness is a more critical issue in the context of stock prediction as anyone can post perturbed tweets or news to influence forecasting models. For example, a fake news ("Two Explosions in the White House and Barack Obama is Injured") posted by a hacker using the AssociatedPress's Twitter account on 04/23/2013 erased $136
† Corresponding author dakuo@acm.org. Our code is available at https://github.com/yonxie/AdvFinTweet
Human Language Technologies, pages 587 - 599
billion market value in just 60 seconds (Fisher, 2013). Although the event doesn't fall into the category of adversarial attack, it rings the alarm for traders who use (social) media information for their trading decisions.
To our best knowledge, it is the first paper to consider the adversarial attack in the financial NLP literature. Many attacks modify benign text directly (manipulation attack) and use them as model input; However, in our case, adversarial retweets enter the model along with benign tweets (concatenation attack), which is more realistic as malicious Twitter users can not modify others' tweets. In other words, we formulate the task as a text-concatenating attack (Jia and Liang, 2017; Le et al., 2021): we implement the attack by injecting new tweets instead of manipulating existing benign tweets. Our task is inspired and mimics the retweet function on social media, and uses it to feed the adversarial samples into the dataset. Despite various algorithms are proposed to generate manipulation attack, literature of concatenation attack on classification models is rare, with exceptions Le et al. (2021), Song et al. (2021) and Wang et al. (2020). Our paper provides extra evidence of their difference by investigating their performances in the financial domain.
The main challenge is to craft new and effective adversarial tweets. We solve the task by aligning the semantics with benign tweets so that the potential human and machine readers can't detect our adversarial tweets. To achieve that, we consider the generation task as a combinatorial optimization problem (Zang et al., 2020; Guo et al., 2021). Specific tweets are first selected, which are used as the target of perturbation on a limit number of words within the tweets. We then examine our attack method on three financial forecast models with attack success rate,  $F1$  and potential profit and loss as evaluation metrics. Results show that our attack method consistently achieves good success rate on the victim models. More astonishingly, the attack can cause additional loss of  $23\%$  to  $32\%$  if an investor trades on the predictions of the victim models (Fig. 4).
2 Adversarial Attack on Stock Prediction Models with Tweet Data
Attack model: Adversarial tweets. In the case of Twitter, adversaries can post malicious tweets which are crafted to manipulate downstream models that take them as input. We propose to attack
by posting semantically similar adversarial tweets as retweets on Twitter, so that they could be identified as relevant information and collected as model input. For example, as shown in Fig 1, the original authentic tweet by the user wallstreetbet7821 was "$BHP announces the demerger of its non-core assets - details expected to be filled in on Tuesday." An adversarial sentence could be "$BHP announces the demerger of its non-core assets - details expected to be exercised in on Tuesday". The outcome of the victim model switches to negative prediction from positive prediction when the retweet is added to the input.
The proposed attack method takes the practical implementation into its design consideration, thus has many advantages. First, the adversarial tweets are crafted based on carefully-selected relevant tweets, so they are more likely to pass the models' tweet filter and enter the inference data corpus. Secondly, adversarial tweets are optimized to be semantically similar to the original tweets so that they are not counterfactual and very likely to fool human sanity checks as well as the Twitter's content moderation system.
Attack generation: Hierarchical perturbation. The challenge of our attack method centers around how to select the optimal tweets and the token perturbations with the constraints of semantic similarity. In this paper, we formulate the task as a hierarchical perturbation consisting of three steps: tweet selection, word selection and word perturbation. In the first step, a set of optimal tweets is first selected as the target tweets to be perturbed and retweeted. For each selected tweet in the pool, the word selection problem is then solved to find one or more optimal words to apply perturbation. Word and tweet budgets are also introduced to quantify the strength of the perturbation.
We consider the word replacement and deletion for word perturbation (Garg and Ramakrishnan, 2020; Li et al., 2020). In the former case, the final step is to find the optimal candidate as replacement. A synonym as replacement is widely adopted in the word-level attack since it is a natural choice to preserve semantics (Zang et al., 2020; Dong et al., 2021; Zhang et al., 2019; Jin et al., 2020). Therefore, we replace target words with their synonyms chosen from synonym sets which contain the semantically closest words measured by the similarity of the GLOVE embedding (Jin et al., 2020).
Mathematical Formulation. We consider a multimodal stock forecast model  $f(\cdot)$  that takes tweet collections  $\{c_t\}_{t=1}^T$  and numerical factors  $\{p_t\}_{t=1}^T$  as input, where  $t$  indexes the date when the data is collected. Peeking into the tweet collection, it contains  $|c_t|$  tweets for date  $t$ , namely,  $c_t = \{s_t^1, s_t^2, \dots, s_t^{|\mathbf{c}_t|}\}$ . Each tweet  $s_t^i$  is a text-based sentence of length  $|\mathbf{s}_t^i|$ , denoted as  $s_t^i = (w_t^{i,1}, \dots, w_t^{i,j}, \dots, w_t^{i,|\mathbf{s}_t^i|})$ , for  $i = 1, \dots, |\mathbf{c}_t|$ . A directional financial forecast model takes domains of tweets and numerical factors as input, and yields prediction for stocks' directional movement  $y \in \{-1, 1\}$ :
$$
\hat {y} _ {t + 1} = f (\boldsymbol {c} _ {t - h: t}, \boldsymbol {p} _ {t - h: t}), \tag {1}
$$
$h$  is the looking-back window for historical data.
The hierarchical perturbation can be cast as a combinatorial problem for tweet selection  $m$ , word selection  $z$  and replacement selection  $u$ . The boolean vector  $m$  indicates the tweets to be selected. For  $i$ -th tweet, vector  $z_i$  indicates the word to be perturbed. As for the word perturbation task, another boolean vector  $u_{i,j}$  selects the best replacement. It follows that the hierarchical perturbation can be formulated as
$$
\boldsymbol {c} _ {t} ^ {\prime} = (\boldsymbol {1} - \boldsymbol {m} \cdot \boldsymbol {z}) \cdot \boldsymbol {c} _ {t} + \boldsymbol {m} \cdot \boldsymbol {z} \cdot \boldsymbol {u} \cdot S (\boldsymbol {c} _ {t}), \quad (2)
$$
where  $\cdot$  denotes element-column wise product,  $m\cdot z$  indicates the selected words in selected tweets,  $m\cdot z\cdot u$  indicates selected synonyms for each selected word, and  $S(\cdot)$  is a element-wise synonym generating function. Consequently, given attack loss  $\mathcal{L}$ , generation of adversarial retweets can be formulated as the optimization program  $\min_{m,z,u}\mathcal{L}(c_t^\prime \cup c_{t - h:t},c_{t - h:t}|p_{t - h:t},f)$ , subject to the budget constraints:  $a)\mathbf{1}^T m\leq b_s,b)\mathbf{1}^T z_i\leq$ $b_w,\forall i$  and  $c)\mathbf{1}^T\boldsymbol {u}_{i,j} = 1,\forall i,j$  where  $b_{s}$  and  $b_{w}$  denote the tweet and word budgets. It is worth to stress that perturbation is only applied to the date  $(t)$  when the attack is implemented to preserve the temporal order.
To solve the program, we follow the convex relaxation approach developed in (Srikant et al., 2021). Specifically, the boolean variables (for tweet and word selection) are relaxed into the continuous space so that they can be optimized by gradient-based methods over a convex hull. Two main implementations of the optimization-based attack generation method are proposed: joint optimization (JO) solver and alternating greedy optimization (AGO) solver. JO calls projected gradient descent
method to optimize the tweet and word selection variables and word replacement variables simultaneously. AGO uses an alternative optimization procedure to sequentially update the discrete selection variables and the replacement selection variables. More details on the optimization program and the solvers can be found in Appendix A.
Dataset & victim models. We evaluate our adversarial attack on a stock prediction dataset consisting of 10,824 instances including relevant tweets and numerical features of 88 stocks from 2014 to 2016 (Xu and Cohen, 2018). Three models (Stocknet (Xu and Cohen, 2018), FinGRU based on GRU (Cho et al., 2014) and FinLSTM based on LSTM (Hochreiter and Schmidhuber, 1997)) of binary classification are considered as victims in this paper. We apply our attack to instances on which the victim models make correct predictions.
Evaluation metrics. Attack performance is evaluated by two metrics: Attack Success Rate (ASR) and victim model's  $F1$  drop after attack. ASR is defined as the percentage of the attack efforts that changes the model output. The two metrics gauge the efficacy of the attack and its impact on model performance: More efficient attack leads to higher ASR and more decline of F1. Moreover, we simulate a Long-Only Buy-Hold-Sell strategy (Sawhney et al., 2021; Feng et al., 2019) with victim models, and calculate the Profit and Loss (PnL) for each simulation. Assume a portfolio starts with initial net value $10000 (100%), its net value at the end of test period reflects the profitability of the trading strategy and the underlying model. Consequently, the change in PnLs measures the monetary impact of our attack. More details on the dataset, victim models and evaluation metrics are housed in Appendix B.
Attack performance with single perturbation. The experiment results for the concatenation attack with word replacement perturbation is shown in Table 1 (with tweet and word budgets both as 1). For both JO and AGO, ASR increases by roughly  $10\%$  and F1 drops by 0.1 on average in comparison to the random attack. Such performance drop is considered significant in the context of stock prediction given that the state-of-the-art prediction
accuracy of interday return is only about  $60\%$ .
Model	ASR(%)	F1
NA	RA	JO	AGO	NA	RA	JO	AGO
Stocknet	0	4.5	16.8	11.8	1	0.96	0.84	0.88
FinGRU	0	5.1	16.4	14.1	1	0.95	0.85	0.87
FinLSTM	0	11.9	16.5	19.7	1	0.89	0.85	0.78
Table 1: Performance of the various adversarial attacks. NA: no attack; RA: random attack; JO: joint optimization; and AGO: alternating greedy optimization.
Effect of attack budget. We report the effect of different attack budgets on the attack performance in Fig. 2. We observe that the more budgets allowed (perturbing more tweets and words), the better the attack performance, but the increase is not significant. It appears that the attack performance becomes saturated if we keep increasing the attack budgets. In fact, the attack with budget of one tweet and one word is the most cost effective, provided that it introduces minimum perturbation but achieves a relatively similar ASR.
Figure 2: Effect of attack budgets on ASR with Stock-net as victim model and with JO solver. r-perturb: word replacement; d-perturb: word deletion.
Manipulation vs concatenation attack. We focus on concatenation attack in this paper since we believe it is distinct from manipulation attack. We investigate the difference by applying the same method of tweet generation to implement manipulation attack, where the adversarial tweets replace target tweets instead. The experiment runs with one word budget and one twee budget, and the results are reported in Fig. 3.
Figure 3: Comparison between manipulation and concatenation attacks with word-replacement perturbation method. Stocknet is the victim model.
It is clear that manipulation attack remarkably outperforms concatenation attack in terms of ASR
and F1. Even though the success rate of concatenation attack lags behind the state-of-the-art textual attack, the manipulation attack achieves performance of the same ballpark, which demonstrates the efficacy of optimization-based attack and our solvers. More importantly, it implies that the attack is not transferable between the two tasks, documenting more evidence on language attack transferability (Yuan et al., 2021; He et al., 2021). The bottom line is that they are two different tasks under different assumptions. Researchers should take downstream scenarios into account when develop attack models.
Trading simulation. The ultimate measure of a stock prediction model's performance is profitability. Fig. 4 plots the profit and loss of the same trading strategy with Stocknet as the prediction model with or without the attack – JO is used to generate adversarial retweets. For each simulation, the investor has $10K (100%) to invest; the results show that the proposed attack method with a retweet with only a single word replacement can cause the investor an additional $3.2K (75%-43%) loss to their portfolio after about 2 years.
Figure 4: Profit and Loss with Stocknet as the victim model using Long-Only Buy-Hold-Sell strategy for 2 years with $10K initial investment. Green line: trading using Stocknet without attack; Blue line: concatenation attack with deletion perturbation; Red line: concatenation attack with replacement perturbation.
This work demonstrates that our adversarial attack method consistently fools various financial forecast models even with physical constraints that the raw tweet can not be modified. Adding a retweet with only one word replaced, the attack can cause  $32\%$  additional loss to our simulated investment portfolio. Via studying financial model's vulnerability, our goal is to raise financial community's awareness of the AI model's risks, so that in the future we can develop more robust human-in-the-loop AI architecture (Wang et al., 2019) to cope with this and other real-world attacks, including black-box attack, unknown input domains, etc.
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896, Brussels, Belgium. Association for Computational Linguistics.
Werner Antweiler and Murray Z Frank. 2004. Is all that talk just noise? the information content of internet stock message boards. The Journal of finance, 59(3):1259-1294.
Yangyi Chen, Jin Su, and Wei Wei. 2021. Multi-granularity textual adversarial attack with behavior cloning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4511-4526, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724-1734, Doha, Qatar. Association for Computational Linguistics.
J Anthony Cookson and Marina Niessner. 2020. Why don't we agree? evidence from a social network of investors. The Journal of Finance, 75(1):173-228.
Nhan Cach Dang, María N Moreno-García, and Fernando De la Prieta. 2020. Sentiment analysis based on deep learning: A comparative study. *Electronics*, 9(3):483.
Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. 2015. Deep learning for event-driven stock prediction. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages 2327-2333. AAAI Press.
Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. 2021. Towards robustness against natural language word substitutions. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Fuli Feng, Xiangnan He, Xiang Wang, Cheng Luo, Yiqun Liu, and Tat-Seng Chua. 2019. Temporal relational ranking for stock prediction. ACM Trans. Inf. Syst., 37(2).
Max Fisher. 2013. Syrian hackers claim AP hack that tipped stock market by $136 billion. Is it terrorism? Washington Post.
Siddhant Garg and Goutham Ramakrishnan. 2020. BAE: BERT-based adversarial examples for text classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174-6181, Online. Association for Computational Linguistics.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In *3rd International Conference on Learning Representations*, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. 2021. Gradient-based adversarial attacks against text transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5747-5757, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Xuanli He, Lingjuan Lyu, Lichao Sun, and Qiongkai Xu. 2021. Model extraction and adversarial transferability, your BERT is vulnerable! In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2006-2012, Online. Association for Computational Linguistics.
Joshua Zoen Git Hiew, Xin Huang, Hao Mou, Duan Li, Qi Wu, and Yabo Xu. 2019. Bert-based financial sentiment index and LSTM-based stock return predictability. *ArXiv* preprint, abs/1906.09024.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1875-1885, New Orleans, Louisiana. Association for Computational Linguistics.
Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021-2031, Copenhagen, Denmark. Association for Computational Linguistics.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY,