A Multi-Target Aspect-Based Opinion Mining Approach with Attention Mechanism and Transformers
Authors: Mohammed Ayub, Sadam Al-Azani, Mohammed Abdur Rahman, Murtadha Abdulsalam, Hamzah Luqman


Reviewer paper summaries: In this paper, the authors investigate a variety of deep learning models for ABSA for restaurant reviews and sentence-level sentiment analysis for tweets. The primary contribution of this paper is redefining ABSA as a multi-target regression task where token indices are predicted for opinion and aspect. In their experiments, they investigate Bi-GRU, Multi-head Self-attention, LSTMs and the default transformer architecture.
Summary of strengths: 1. The authors investigate a wide variety of model architectures.

2. The applied methodologies seem to be well implemented and are documented in detail.

3. The introduction and methodology sections cover sufficient related work.
Summary of weaknesses: 1. Insufficient argumentation for the preprocessing steps. Some of the decisions made can have a significant impact, such as removing punctuation and stopwords. These choices are not as straightforward as presented in the paper and should be discussed and argued for.

2. In the introduction, the authors mention using SOTA deep learning approaches like transformers in their experiments. In practice, most SOTA approaches make use of embeddings created by transformer models as input for different (more sophisticated) architectures. However, in this paper, the authors make use of GloVe embeddings without discussing this choice.

3.The primary methodological choice is not clearly explained or argued for. For the reader, it is not clear why the authors would define this as a regression task where token indices are predicted. This decision should either be argued for in detail or refer to related work where this approach was found to provide promising results. Intuitively, it does not make sense to predict token indices as a regression task, as these indices refer to discrete tokens. Furthermore, although the authors evaluate their system for sentiment analysis in tweets on existing datasets, the comparison to the baseline scores is highly unusual. The baselines the authors compare to are a feature-based SVM, Naive Bayes model and all-positive/all-negative predicting baselines. These are very weak baselines and the best-performing mentioned by the authors here is their 2021 fine-tuned AraBERT model, which indicates the models for this paper are only a small contribution. Perhaps this paper would have been better as a short paper, where only the aspect-based SA models are investigated and compared.

4. In this paper on Aspect-Based opinion mining, the authors describe experiments on both aspect-based sentiment analysis (not exactly opinion mining) and on sentence-level sentiment analysis on twitter datasets. The title is a bit misleading and the evaluation on these two different tasks does not seem to fit together in a single paper.

5. The paper is not very neat, there are a lot of typos, unclear sentences, language mistakes, the figures are not too clear.
Comments, suggestions and typos: 040-041: This sentence contains too many repetitions of "and", please rephrase this sentence.
047-054: " While high-level sentiment analysis achieved satisfactory results for high-level or cross-granularity levels at the document and sentence levels for some languages such as the Western languages, the performance of current techniques is still so far from satisfactory for fine granularity level analysis such as sentiment analysis at aspect level." --> I believe this statement requires support from references to related work.
071: " hybrid-based models"--> "hybrid models"
084: "The" as an article should be removed at the start of this sentence.
091: With the development of generative LLMs, whether or not these approaches are still cutting edge may be questioned. It is likely better to consider this as the most popular current approach, as this methodology is widely applied.
111: "Performing" should not be capitalized.
118: "Before conclusion, we discus the limitation" --> To conclude, we discuss the limitations...
122: There should be a space before the number of the figure.
128-129: It is extremely unusual to say that anything is mandatory considering methodologies for NLP. It is very common to use pre-processing to remove characters or anonimize the text, but it is definitely not mandatory. In many cases, punctuation and these special characters can contribute to the meaning of a tweet, so there is also work that does not remove these characters.
143: What does it mean to say that stop-words are not considered in this study? Are they actually removed from the text? It is highly unusual to remove function words from the input text before using it as input for transformer models. This choice should be argued for in the paper.
155 requires a citation for GloVe embeddings
162-170: It is not entirely clear why the authors consider ABSA as a regression task where the indices are predicted for the opinion and aspect. This is 
177: Direct Multi-Tsarget 
179-180: "considers all the targets in the inferencing as well" --> this is not a proper sentence
>> please make sure to perform a thorough linguistic review of the paper to improve th writing.
349-350: it is not clear what the authors intend to convey with this sentence.

in Table 1, the authors mention that the English and Arabic tweets are only annotated for sentiment polarity and not for aspects. This is rather problematic and misleading in the scope of this paper, considering that the title mentions ABSA and not general sentiment analysis. Sentence-level or Document sentiment analysis are different tasks compared to ABSA.

Throughout the paper, it should be specified more clearly what type of embeddings are used for each model.

It is a bit strange to see table captions on top of the tables and figure captions at the bottom.
Overall assessment scores: 2