SlowGuess/NAACL_2022_OCR huggingface dataset:
USA, February 7-12, 2020, pages 8018-8025. AAAI Press.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Thai Le, Noseong Park, and Dongwon Lee. 2021. A sweet rabbit hole by DARCY: Using honeypots to detect universal trigger's adversarial attacks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3831-3844, Online. Association for Computational Linguistics.
Qi Lei, Lingfei Wu, Pin-Yu Chen, Alex Dimakis, Inderjit S. Dhillon, and Michael J. Witbrock. 2019. Discrete adversarial attacks and submodular optimization with applications to text classification. In Proceedings of Machine Learning and Systems 2019, MLSys 2019, Stanford, CA, USA, March 31 - April 2, 2019. mlsys.org.
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. BERT-ATTACK: Adversarial attack against BERT using BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193–6202, Online. Association for Computational Linguistics.
Bill Yuchen Lin, Wenyang Gao, Jun Yan, Ryan Moreno, and Xiang Ren. 2021. RockNER: A simple method to create adversarial examples for evaluating the robustness of named entity recognition models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3728-3737, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Pekka Malo, Ankur Sinha, Pekka J. Korhonen, Jyrki Wallenius, and Pyry Takala. 2014. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65.
Dang Lien Minh, Abolghasem Sadeghi-Niaraki, Huynh Duc Huy, Kyungbok Min, and Hyeonjoon Moon. 2018. Deep learning approach for short-term stock trends prediction based on two-stream gated recurrent unit network. IEEE Access, 6:55392-55404.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1085-1097, Florence, Italy. Association for Computational Linguistics.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging NLP models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856-865, Melbourne, Australia. Association for Computational Linguistics.
Ramit Sawhney, Shivam Agarwal, Arnav Wadhwa, and Rajiv Ratn Shah. 2020a. Deep attentive learning for stock movement prediction from social media text and company correlations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8415-8426, Online. Association for Computational Linguistics.
Ramit Sawhney, Piyush Khanna, Arshiya Aggarwal, Taru Jain, Puneet Mathur, and Rajiv Ratn Shah. 2020b. VolTAGE: Volatility forecasting via text audio fusion with graph convolution networks for earnings calls. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8001-8013, Online. Association for Computational Linguistics.
Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, and Rajiv Ratn Shah. 2021. FAST: Financial news and tweet based time aware network for stock trading. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2164-2175, Online. Association for Computational Linguistics.
Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan. 2021. Universal adversarial attacks with natural triggers for text classification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3724-3733, Online. Association for Computational Linguistics.
Shashank Srikant, Sijia Liu, Tamara Mitrovka, Shiyu Chang, Quanfu Fan, Gaoyuan Zhang, and Una-May O'Reilly. 2021. Generating adversarial computer programs using optimized obfuscations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings.
Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24-54.
Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, and Bo Li. 2020. T3: Treeautoencoder regularized adversarial text generation
for targeted attack. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6134-6150.
Dakuo Wang, Justin D Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-ai collaboration in data science: Exploring data scientists' perceptions of automated ai. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1-24.
Frank Z Xing, Erik Cambria, and Roy E Welsch. 2018. Natural language based financial forecasting: a survey. Artificial Intelligence Review, 50(1):49-73.
Ying Xu, Xu Zhong, Antonio Jimeno Yepes, and Jey Han Lau. 2021. Grey-box adversarial attack and defence for sentiment classification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4078-4087, Online. Association for Computational Linguistics.
Yumo Xu and Shay B. Cohen. 2018. Stock movement prediction from tweets and historical prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1970-1979, Melbourne, Australia. Association for Computational Linguistics.
Linyi Yang, Tin Lok James Ng, Barry Smyth, and Ruihai Dong. 2020. HTML: hierarchical transformer-based multi-task learning for volatility prediction. In WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pages 441-451. ACM / IW3C2.
Liping Yuan, Xiaqing Zheng, Yi Zhou, Cho-Jui Hsieh, and Kai-Wei Chang. 2021. On the transferability of adversarial attacks against neural text classifier. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1612–1625, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066-6080, Online. Association for Computational Linguistics.
Huangzhao Zhang, Hao Zhou, Ning Miao, and Lei Li. 2019. Generating fluent adversarial examples for natural languages. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5564-5569, Florence, Italy. Association for Computational Linguistics.
Lei Zhang, Shuai Wang, and Bing Liu. 2018. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4):e1253.
Xinze Zhang, Junzhe Zhang, Zhenhua Chen, and Kun He. 2021. Crafting adversarial examples for neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1967-1977, Online. Association for Computational Linguistics.
A Mathematical Formation
A.1 Financial Forecast Model
Massive amounts of text data are generated by millions of users on Twitter every day. Among a variety of discussion, stock analysis, picking and prediction are consistently one of the trending topics. And investors often use the Twitter cashtag function (a $ symbol followed by a ticker) to organize their particular thoughts around one single stock, e.g., $AAPL, so that users can click and see the ongoing discussions. Textual data on Twitter is collectively generated by all of its users via posting tweets. Financial organizations and institutional investors often ingest the massive text data in real time and incorporate them or their latent representation into their stock prediction models.
We consider the multimodal stock forecast models that take tweet collections  $\{c_t\}_{t=1}^T$  and numerical factors  $\{p_t\}_{t=1}^T$  as input, where  $t$  indexes the date when the data is collected. The numerical factors are usually mined from historical price, fundamentals and other alternative data sources. In this paper, we assume that the domain of numerical factors is unassailable since they are directly derived from public records. Therefore, the objective of adversary is to manipulate model output by injecting perturbation to the textual domain  $\{c_t\}_{t=1}^T$ . Peeking into the tweet collection, it contains  $|c_t|$  tweets for date  $t$ , namely,  $c_t = \{s_t^1, s_t^2, \dots, s_t^{|c_t|}\}$ . Each tweet  $s_t^i$  is a text-based sentence of length  $|s_t^i|$ , denoted as  $s_t^i = (w_t^{i,1}, \dots, w_t^{i,j}, \dots, w_t^{i,|s_t^i|})$ , for  $i = 1, \dots, |c_t|$ . A directional financial forecast model takes domains of tweets and numerical factors as input, and yields prediction for stocks' directional movement  $y \in \{-1, 1\}$ :
$$
\hat {y} _ {t + 1} = f \left(\boldsymbol {c} _ {\boldsymbol {t} - \boldsymbol {h}: \boldsymbol {t}}, \boldsymbol {p} _ {\boldsymbol {t} - \boldsymbol {h}: \boldsymbol {t}}\right), \tag {3}
$$
where  $h$  is the looking-back window for historical data.
A.2 Attack Model
Let  $c_{t}^{\prime}$  be the perturbed tweet collection at time  $t$  created by solving the hierarchical perturbation problem. To formalize the perturbation task, we introduce boolean vector variable  $m \in \{0,1\}^{n_m}$  to indicate the tweets to be selected. If  $m_i = 1$ , then  $i$ -th tweet is the target tweet to be perturbed and retweeted. Besides, for  $i$ -th tweet, vector  $z_{i} \in \{0,1\}^{n_{z}}$  indicates the word to be perturbed. As for the word perturbation task, another boolean
vector  $\pmb{u}_{i,j} \in \{0,1\}^{n_u}$  selects the best replacement.  $n_m$  and  $n_z$  and  $n_u$  denote the maximum amount of tweets, maximum amount of words in each tweet, and the amount of synonyms for each word, respectively. We identify deletion perturbation as a special case of replacement with  $u_{i,j,k} = 1$  only for padding token, so that the task degenerates to tweet selection and word selection. Let vector  $\pmb{z} \in \{0,1\}^{n_m \times n_z}$  denote  $n_m$  different  $\pmb{z}_i$  vector, and  $\pmb{u} \in \{0,1\}^{n_m \times n_z \times n_u}$  denote  $n_m \times n_z$  different  $u_{i,j}$  vectors. It follows that the hierarchical perturbation can be defined as
$$
\boldsymbol {c} _ {t} ^ {\prime} = (\boldsymbol {1} - \boldsymbol {m} \cdot \boldsymbol {z}) \cdot \boldsymbol {c} _ {t} + \boldsymbol {m} \cdot \boldsymbol {z} \cdot \boldsymbol {u} \cdot S (\boldsymbol {c} _ {t})
$$
s.t.  $\mathbf{1}^T\mathbf{m}\leq b_s,$  (4)
$$
\begin{array}{l} \mathbf {1} ^ {T} \boldsymbol {z} _ {i} \leq b _ {w}, \forall i, \\ \mathbf {1} ^ {T} \boldsymbol {u} _ {i, j} = 1, \forall i, j, \\ \end{array}
$$
where  $\cdot$  denotes element-column wise product,  $b_{s}$  denotes tweet budget,  $b_{w}$  denotes word budget and  $S(\cdot)$  is element-wise synonym generating function.
Adversarial retweets are the then passed into downstream financial forecast model  $f(\cdot)$  along with benign tweets. Attack success is achieved if the adversarial tweets manage to fool the downstream model, and change the model output. Financial forecast model usually takes observation of multiple steps as input to appreciate the temporal dependence. However, adversary can only inject adversarial retweets at present time. That is, when run the model on day  $t$  to predict price movement on day  $t + 1$ , retweets only enter tweet collection for day  $t$ ; collections for days prior to  $t$  remain static. Consequently, generation of successful adversarial retweets is formulated as the following optimization program:
$$
\min  _ {m, z, u} \quad \mathcal {L} \left(\boldsymbol {c} _ {t} ^ {\prime} \cup \boldsymbol {c} _ {t - h: t}, \boldsymbol {c} _ {t - h: t} | \boldsymbol {p} _ {t - h: t}, f\right) \tag {5}
$$
s.t. constraint in (4),
where  $\mathcal{L}$  denotes the attack loss. We adopt the cross-entropy loss for our attack since it is untargeted attack (Srikant et al., 2021). Other classification-related loss may be applied according to adversary's objective. Furthermore, we also add entropy-based regularization to encourage sparsity of optimization variables (Dong et al., 2021).
A.3 Methodology
The challenge of solving program (5) lies in the combinatorial and hierarchical nature. We first relax the boolean variables into continuous space so
that they can be solved by gradient-based solvers. A common workaround for combinatorial optimization is to solve an associated continuous optimization over convex hull (Dong et al., 2021; Srikant et al., 2021). An computationally efficient fashion is to optimize over a convex hull constructed with linear combination of candidate set, and the optimal replacement goes with word with highest weight (Dong et al., 2021). However, this approach doesn't fit in the hierarchical tweet and word selection problem. For example, in order to select the optimal target word, one need to sum over the embedding of all words in the tweet, so the tweet collapses into embedding for one hypothetical word. Similarly, different tweets collapse to one hypothetical tweet, or one hypothetical word when one jointly selects tweets and words.
Joint optimization solver (JO). As a remedy, we propose a joint optimization solver that combines projected gradient descent and convex hull to jointly optimize  $m$ ,  $z$  and  $u$ . Replacement selection is optimized over the convex hull:
$$
\pmb {c} _ {t} ^ {\prime} = (1 - \pmb {m} \cdot \pmb {z}) \cdot \pmb {c} _ {t} + \pmb {m} \cdot \pmb {z} \cdot c o n v (\pmb {u}, S (\pmb {c} _ {t})),
$$
$$
\operatorname {c o n v} (\boldsymbol {u}, S (\boldsymbol {c _ {t}})) = \left\{\sum_ {k} \hat {u} _ {i, j, k} S (w _ {i, j, k}), \forall i, j \right\},
$$
$$
\hat {u} _ {i, j, k} = \frac {\exp (u _ {i , j , k})}{\sum_ {k} \exp (u _ {i , j , k})}.
$$
The problem of (5) is then solved by optimizing  $\hat{\pmb{u}}$ . Unlike  $\pmb{u}$ ,  $\pmb{m}$  and  $\pmb{z}$  are optimized directly via projected gradient descent (PGD). Moreover, when  $\pmb{m}$  is one-hot vector, it determines the tweets to be retweeted, and those retweets are then added into tweet collection. However,  $\pmb{m}$  is continuous during optimization, so we retweet all the collected tweets and add them into tweet collection, which helps generate and back-propagate gradients for all the entries of  $\pmb{m}$ . After the optimization is solved, we map the continuous solution into one-hot vector by selecting top  $b_{s}$  highest  $m_{i}$ .
Alternating greedy optimization solver (AGO). Greedy optimization is usually computational ineffective since a vast amount of inquiries is required when we collect large amounts of tweets and have high attack budget. To mitigate the problem, we alternate the optimization over  $m$ ,  $z$  and  $u$ . The
aforementioned convex hull approach is adopted for finding optimal  $\mathbf{u}$ . The difference lies on the path to solve tweet and word selection problems. More specifically, we alternatively search the optimal target tweets and words which achieve the highest increases in prediction loss. For tweet selection, we mimic the physical attack scenario, and new retweets are added into tweet collection during the greedy search. Depending on the adversary's objective, different metrics may be used to measure the importance of each tweet and word. For example, Alzantot et al. (2018) use predicting probability to determine the selection of words; Ren et al. (2019) propose probability weighted word saliency as criterion for word selection; Jin et al. (2020) calculate the prediction change before and after deletion as word importance.
We evaluate our adversarial attack on a stock prediction dataset (Xu and Cohen, 2018). The dataset contains both tweets and historical prices (e.g., open, close, high, etc) for 88 stocks of 9 industries: Basic Materials, Consumer Goods, Healthcare, Services, Utilities, Conglomerates, Financial, Industrial Goods and Technology. Since we consider the task of binary classification, data instances are supposed to labelled positive and negative for upward and downward movement respectively.
Moreover, it is observed that the dataset contains a number of instances with exceptionally minor price movements. In practice, minor movement is hard to be monetized due to the existence of transaction cost. Therefore, an upper threshold of  $0.55\%$  and a lower threshold of  $-0.5\%$  are introduced. Specifically, stocks going up more than  $0.55\%$  in a day are labeled as positive, those going down more than  $-0.5\%$  are labeled as negative, and the minor moves in between are filtered out. As argued in (Xu and Cohen, 2018), the particular thresholds are carefully selected to balance the two classes.
In addition, the sampling period spans from 01/01/2014 to 01/01/2016. We split the dataset into train and test set on a rolling basis. This special program improves the similarity between distributions of train set and test set, which is widely adopted on temporal dataset. It leaves us 9416 train instances and 1408 test instances in 7 nonconsecutive periods. For the text domain, the dataset contains
57533 tweets in total.
B.2 Victim Models
Stocknet. A variational Autoencoder (VAE) that takes both tweets and price as input (Xu and Cohen, 2018). Tweets are encoded in hierarchical manner within days, and then modeled sequentially along with price features. It consists of three main components in bottom-up fashion. Market Information Encoder first encodes tweets and prices to a latent representation of 50 dimensions for each day. Variational Movement Decoder infers latent vectors of 150 dimensions and then decodes stock movements. At last, a module called Attentive Temporal Auxiliary integrates temporal loss through an attention mechanism. We train the model on the dataset from scratch with the same configurations as Xu and Cohen (2018).
FinGRU. A binary classifier that takes numerical features and tweets as input. All features are encoded sequentially by GRU (Cho et al., 2014) to exploit the temporal dependence. The model adopts the same Market Information Encoder as Stock-net. Latent representation of tweets and prices are then fed into a layer of GRU with attention mechanism to integrate temporal information. We train the model with an Adam optimizer (Kingma and Ba, 2015) and learning rate of 0.005. The checkpoint achieves the best performance on test dataset among 100 epochs is adopted as the victim model.
FinLSTM. A binary classifier identical to FinGRU, but utilizes LSTM (Hochreiter and Schmidhuber, 1997) to encode temporal dependence. The model is trained in the same manner as FinGRU.
Following Srikant et al. (2021), we evaluate the attack on those examples in the test set that are correctly classified by the target models. It provides direct evidence of the adversarial effect of the input perturbation and the model robustness. In the specific application of financial forecast, it makes more sense to manipulate correct prediction than incorrect ones. The following two common metrics are adopted to evaluate attack performance.
Attack Success Rate. ASR is defined as the percentage of the attack efforts that make the victim model misclassify the instances that are originally correctly classified. Mathematically,  $ASR =$
$\frac{\sum_{t}\delta(\hat{y}_{t}^{\prime}\neq y_{t})}{\sum_{t}\delta(\hat{y}_{t} = y_{t})}$ , where  $\hat{y}_t$  is the unperturbed model prediction,  $\hat{y}_t^\prime$  the model prediction with perturbation, and  $y_{t}$  the ground-truth label. ASR characterizes the capability of the attack model, and higher the ASR, the better the attack.
F1 Score. F1 gauges the prediction performance of the victim models. Since we only consider the samples that are correctly predicted, the F1 score in the case of no attack is 1. Apparently, the drop of the F1 score of caused by the perturbation demonstrates the performance of the attack method. Unlike ASR, the drops of F1 score gauge the direct impact on the model performance: more successful attack leads to lower post-attack F1 score.
Profit and Loss. This widely-used financial indicator measures the profitability of a trading strategy. Assume that the initial net values are  $10K (100\%)$ , accumulate profit and loss for each trade, we can then calculate the final net value of the portfolio and profit and loss. A binary financial forecast model can be exploited in many ways, and support various trading strategies, which usually lead to different PnLs. In this paper, we use a simple Long-Only Buy-Hold-Sell strategy (Sawhney et al., 2021; Feng et al., 2019). More specifically, we buy stock(s) on Day  $T$  if the model predicts these stocks go up on Day  $T + 1$ , hold for one day, and sell these stocks the next day no matter what prices will be, and repeat it. We do not short a stock even if the model predicts a negative move in the second day.
Besides, when the model makes positive prediction on more than one stocks, the money is evenly invested to the stock pool of positive prediction. For example, suppose that we stand on day 4 with portfolio value \(12\mathrm{K}\). If the model gives positive prediction on 10 of 88 stocks for day 5, we invest \(10\%\) of the total wealth \((\\)1.2\mathrm{K})\) to each stock, and sell them at closing prices of day 5. The process continues until the end of the test periods, and the resulting net value of the portfolio is used to calculate the profit and loss of the underlying model.
The buy-hold-sell strategy monetizes the prediction performance of financial forecast models by betting on the their predictions. The PnL reflects the profitability of the underlying models, even if it is usually influenced by many other confounding factors. Most importantly, the changes of PnLs caused by perturbation on the victim models only gauge the monetary consequence of our attack,
Model	ASR(%)	F1
NA	RA	JO	AGO	NA	RA	JO	AGO
Stocknet	0	3.6	12.1	11.0	1	0.97	0.89	0.89
FinGRU	0	4.0	10.2	10.6	1	0.96	0.85	0.91
FinLSTM	0	11.9	12.1	11.6	1	0.89	0.89	0.89
Table 2: Results for concatenation attack with deletion perturbation and budgets 1. NA and RA stand for no attack and random attack respectively, serving as benchmarks.
Figure 5: Effect on Profit and Loss of various perturbation methods on FinGRU and FinLSTM.
since all else are equal.
C Supplemental Experiment Results
C.1 Replacement vs deletion perturbation.
We report results for concatenation attack with only the replacement perturbation in the main text in Table 1. Here we also report results for the deletion perturbation in Table 2. Attacks conducted via deletion perturbation in general perform worse than the results of replacement perturbation. We observe ASRs via JO and AGO fall by  $5.1\%$  and  $4.1\%$  respectively compared with the replacement perturbation. Accordingly, F1 slightly increases as attack performance worsens. There is no significant difference between the two optimizers (JO and AGO) in the case of deletion perturbation, but JO is preferable in terms of optimization efficiency.