Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper proposes a reinforcement-learning-based model for aspect-based sentiment analysis. The RL model trains an agent that tries to walk through the most effective path from the aspect target to determine the final sentiment towards this target. 

In general, the RL model based on the dependency structure could be deemed as a pioneer implementation on aspect-based sentiment analysis. And this approach is able to remove the negative effect brought by irrelevant context. However, there are a few limitations that need to be addressed:
1. The writing needs to be improved. There are many grammatical mistakes that affect the readability, e.g., "To effectively contain the impact from task-irrelevant information..." from the last paragraph in page 1: the misusing of word "contain". "The goal of aspect-based sentiment classification is to predict sentiment polarities (i.e., “positive”, “neutral”, and “negative”) for each given aspect.": "sentiment polarities"->"sentiment polarity".
2. The proposed model is missing many details. For example, when will the policy terminate? It seems from (3) and (4) there could be infinite loop of candidate actions. And what is exactly the policy network in (3)?
3. The RL model in this case is very limited in terms of exploration. From the construction, the candidate actions only contain connected nodes at each timestamp. This may miss important information, e.g., the second aspect in figure 1 could not traverse to "do not". How do you solve this issue? 
4. I am not convinced by the statement that the proposed model is more generalizable compared to baseline models. Where does the generalization ability come from? And I conjecture RL needs even more training data to perform well.
5. As mentioned in the main text, the reward function (6) could be any form, but no experiments are conducted on other forms. The experimental result in Table 2 misses other recent baselines, e.g., "Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification" that actually outperforms this method.
review_title: Interesting RL implementation but with limited contribution.

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposes an approach to aspect-based sentiment classification, which is the task of identifying the sentiment of a specific phrase or entity in a sentence. The paper proposes to do this by first generating a dependency parse of the entire sentence and then using an RL agent to walk the dependency tree starting from the word or phrase to be classified. The state for the RL agent is the LSTM representation of its history. The final state is then used to classify the sentiment of the aspect.
The approach is evaluated on multiple benchmark datasets for the task and compared with many previously proposed approaches. Most of the compared methods don’t make use of the dependency graphs but the authors also compare with one recent graph convolution-based approach that also makes use of the dependency graph. They show improvements over these methods on most of the benchmarks.

Strong Points
- The approach makes use of dependency graph information in an interesting way to learn to find paths that would lead to better aspect-based sentiment classification.
- The proposed method is well explained and easy to follow.
- The results show improvements on many datasets.

Weak Points
- While the paper compares with many baselines, it doesn’t compare with some highly related work [1,2] which uses Transformer text encoders. Ideally, the paper should discuss these and potentially directly compare with these.
- Due to the use of RL, the proposed approach will be more computationally expensive. It will be good to also discuss this and contrast with other methods.

Questions for authors:
- How do you obtain the dependency tree for the sentences? Have you analyzed how errors in dependency parsing affect performance?
- Can you comment on the computation time for the proposed and competing approaches?
- Can you compare and contrast your approach to [1] and [2]? It would be useful to see if using BERT token embeddings as input embeddings for the proposed method still provides competitive performance as these approaches[1,2] report better performance than the proposed approach.
- Most of the datasets used in the experiments are very small. Given the high sample complexity requirement of RL methods, it is a bit surprising that the RL-based approach works at all for this problem. Can you explain if you found this to be an issue in your experiments?

Suggestions:
- It will be good to also mention the word representations and the text encoder (LSTM or something else) that is used in the compared baselines.
- For path length comparisons in Appendix, please add a pointer to it in the experiments section. It will also be good to see the performance at path length 1 (which wouldn’t require any agent walk).
- Many citations are not properly formatted. Instead of “By leveraging textual context and word-level attention learned from deep models Vo & Zhang (2015); Dong et al. (2014)”, it should be cited as “By leveraging textual context and word-level attention learned from deep models (Vo & Zhang, 2015; Dong et al., 2014)”

[1] Target-Dependent Sentiment Classification With BERT https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8864964

[2] Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification https://www.aclweb.org/anthology/2020.acl-main.588.pdf

review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
The paper addresses aspect-based sentiment analysis by running reinforcement learning on the dependency parse of input sentences. The agent learns a policy network to select the most effective walk along edges in the dependency graphs, starting from the target aspect in the input sentences. The state representation is learned with an LSTM. At the end of the path, a sentiment classifier predicts the distribution of the polarity. The reward is the mean squared error between the class label (i.e. sentiment polarity) and the probability predicted by the model. The paper claims that by limiting the agent's budget, the approach forces the agent to discard irrelevant information and focus on the effective paths, enabling the approach to perform well with a small number of training examples.

Strengths: 
- The intuition behind the approach of an agent learning dependency paths that can be used for sentiment analysis is appealing. The paper evaluates the performance on 5 data sets, and compares against baselines such as MemNet, TNet-Lf, and ASGCN. Among the baselines, ASGCN also leverage on dependency parses of the input sentences.

Weaknesses: 
- The paper failed to compare against a number of publications that also uses dependency parses as input to improve aspect based sentiment analysis, e.g., [1], [2], [3], [4] and [5]. [1] and [2] were referenced in the paper, but their results were not included in the comparison. The results in these 5 papers seem considerably better than in the current submission: e.g., for laptops, a number of these papers obtain over 74% F1 compared to the 71.48% in this submission. 

- It seems that there is a big advantage in using dependency parses for sentiment analysis (e.g., [1,2,3,4,5]), so it is unclear how much of the good performance in the current submission is actually from reinforcement learning. While the intuition behind finding dependency paths is appealing,  the ablation that tries to show this is limited to a few examples in the paper. It would be interesting to have a baseline that uses the dependency parse but not reinforcement learning. They could also try and compile some statistics of the relations on the most selected dependency paths, or evaluate a small sample of selected paths with human evaluators.

[1] Syntax-Aware Aspect Level Sentiment Classiﬁcation with Graph Attention Networks
Binxuan Huang and Kathleen M. Carley. EMNLP 2019

[2] Exploiting Typed Syntactic Dependencies for Targeted Sentiment Classiﬁcation Using Graph Attention Neural Network. Xuefeng Bai, Pengbo Liu, and Yue Zhang. ArXiv, submitted on 22nd Feb 2020

[3] Relational graph attention network for aspect-based sentiment analysis. Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. ACL 2020.

[4] Dependency graph enhanced dual transformer structure for aspect-based sentiment classification. Hao Tang, Donghong Ji, Chenliang Li, and Qiji Zhou. ACL 2020.

[5] Modelling context and syntactical features for aspect based sentiment analysis. Minh Hieu Phan and Philip O. Ogunbona. ACL 2020.



review_title: Interesting work, but needs more ablation and comparison with recent work

Meta-review: In this paper, the authors proposed a reinforcement-learning-based model for aspect-based sentiment analysis. As raised by the reviewers, 1) the writing needs to be improved: e.g., presenting the details of the proposed method clearly, citing the references properly, etc. 2) related methods need to be implemented for comparison, 3) the reported results are not SOTA compared with existing methods. Moreover, some technical claims are not convincing, which need to be stated more carefully.

In summary, based on its current shape, this paper is not ready to be published in ICLR.
Decision: Reject