Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper studies the impact of adversarial attacks on a ML based system for forecasting stock prices. The authors leverage Twitter data in order to enhance stock price prediction. Then, by determining the sensitivity of the model when perturbing the inputs. Then, small changes are applied to the inputs and output is observed. The authors experimented with the Tesla stock price.

This is interesting work and the authors do good work at explaining the different parts. Generally it flows well.

A first comment I have is about the focus of the paper as it studies both feasibility of forecasting as well as adversarial attacks. But, it does not go in depth in neither of these topics. For example, the first part for me is very interesting and having more forecasting approaches as well as adding more features with the additions of time-series would be very valuable.

Another comment I have is that there is a naive assumption. The fact that before sentiment classification we do not do any filtering. This does not make sense for an industrial system. Having a filtering step would greatly benefit the pipeline. The authors could have invested some space on such an approach.

As you conclude that an adversarial trader would need to know the details for the forecasting model in order to attack would you be able to suggest an approach that would not require this knowledge?

A major issue of the paper is that the authors just do predictions in a single stock price. I am not sure how much we can trust the conclusions based on just a single stock. The authors need to add more datasets. It is not enough.

What about having other type of textual features apart from the sentiment? Do you think there is more space to explore here?
review_title: This is generally an interesting paper as it studies adversarial attacks for financial systems that rely on ML systems.

final_score: rating: 5: Marginally below acceptance threshold
review_content: In this paper, the authors studied the problem of adversarial ML in stock price forecasting. They first replicated an industry standard pipeline, which performs a sentiment analysis of Twitter data to forecast trends in stock prices. Then, they show that an adversary can exploit the lack of provenance to indirectly use tweets to manipulate the model’s perceived sentiment about a target company and in turn force the model to forecast price erroneously.

Strength:

1. The topic is interesting. Adversarial ML attracts a lot of attention recently, and applying them to stock price forecasting has clear incentive.
2. The authors plan to release the code and data when this is accepted, which is good for other researchers to reproduce this work.

Weakness:

1. The novelty of the methodology seems limited. Both the stock price forecasting model (FinBERT and DeepAR-G) and the gradient attack method (Papernot et al. 2016) are not proposed by the authors. 
2. The writing can be improved. For example, in the experiment section, the authors used a lot of evaluation metrics such as sharpe ratio, greedy, threshold. Though they might be familiar to the audience in finance industry, they are not very clear to me at first. I'd suggest to give a definition and explanation of those metrics in this section.

Overall comments:

I think the research topic of this paper is interesting, but the methodology seems lack of novelty. In addition, the writing can be further improved. 
review_title: Official Blind Reviewer 2

final_score: rating: 7: Good paper, accept
review_content: Pros: 
The paper is clear with a significant contribution. It performed a sentiment analysis task that can predict trends in the stock market and also showed how an adversary may attack the model using tweets thereby leading to false price prediction. The methodology and the probabilistic forecasting used are excellent in my opinion. 

Cons: 
1. The title of the paper does not describe the actual work done. I suggest that the authors may consider giving a new title to reflect the method they applied. 
2. It is unclear why the attack was done on the test stage but was not shown at implementation. That is, how would this work be reproduced or be beneficial to the community? 

The following can be corrected: 
1. The claim on Page 1, Para 2 of the Introduction needs a citation (Thales used his...) 
2. On page 5, para 1, please briefly describe BERT 

review_title: On the Robustness of Sentiment Analysis for Stock Price Forecasting

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors in this paper propose to use the adversarial ML in the stock price forecasting scenario and demonstrate that an adversary can exploit the lack of provenance to indirectly use tweets to manipulate the prediction model’s perceived sentiment about a target company and in turn force the model to forecast price erroneously. However, I have the following comments on the paper: 

- Dataset collection is not convinced. Authors collect only one company (i.e.,  Tesla) data from Twitter and Yahoo. Experiments in the entire paper are only based on the dataset containing just one company. The authors provide two reasons for not considering other companies (such as Microsoft or Amazon as discussed examples in the paper): "homonyms" and "unintended discussion". However, Tesla also has the similar issue. For example, when Elon Must was discussed on Twitter, it may be about Elon Musk’s other companies (SpaceX) or his arguments that are not related to Tesla (similar to the Bill Gates case in the paper). When people talk about Tesla, they may talk about Nikola Tesla. When people discuss Model 3, they may only express their experience of driving Model 3. The dataset with only one company contains bias to the model that may not be generalized to other cases.

- Lack of novelty. All the technical components in this paper are not new. FinBERT, DeepAR, and Gradient Attack are all from existing works. 
review_title: An interesting topic but several issues in experiments

Meta-review: One reviewer is positive, but that review is not of high quality. The other reviewers agree that this paper is interesting, but has too many limitations to be accepted by a highly competitive venue such as ICLR.
Decision: Reject