AoM: Detecting Aspect-oriented Information for Multimodal Aspect-Based Sentiment Analysis
Year: 2023

Abstract:
Multimodal aspect-based sentiment analysis (MABSA) aims to extract aspects from textimage pairs and recognize their sentiments. Existing methods make great efforts to align the whole image to corresponding aspects. However, different regions of the image may relate to different aspects in the same sentence, and coarsely establishing image-aspect alignment will introduce noise to aspect-based sentiment analysis (i.e., visual noise). Besides, the sentiment of a specific aspect can also be interfered by descriptions of other aspects (i.e., textual noise). Considering the aforementioned noises, this paper proposes an Aspect-oriented Method (AoM) to detect aspect-relevant semantic and sentiment information. Specifically, an aspect-aware attention module is designed to simultaneously select textual tokens and image blocks that are semantically related to the aspects. To accurately aggregate sentiment information, we explicitly introduce sentiment embedding into AoM, and use a graph convolutional network to model the vision-text and text-text interaction. Extensive experiments demonstrate the superiority of AoM to existing methods. The source code is publicly released at https://github.com/SilyRab/AoM.

heading: 1 Introduction
text: As an important and promising task in the field of sentiment analysis, Multimodal Aspect-Based Sentiment Analysis (MABSA) has attracted increasing attention (Lv et al., 2021; Ju et al., 2021). Given an image and corresponding text, MABSA is defined as jointly extracting all aspect terms from imagetext pairs and predicting their sentiment polarities (Ju et al., 2021).
In this scenario of fine-grained sentiment recognition for multimodal information, the input imagetext pairs are always complex. (1) The semantics of sentence is complex which adds sentiment confusion among different aspects. Take Figure 1 as an
∗Corresponding author.
example, there are 3 aspects in the sentence with 3 different sentiments, The sentiment of “mayor" can be easily confused by the keyword, “Interesting”, which is of positive sentiment. (2) The images contain a large amount of detailed information, and the visual contents are usually related to only one or several of the aspects. For example, as shown in Figure 1, the objects in red boxes are more helpful in analyzing the sentiment of “Mayor Kadokawa” than the other aspects. The complex input greatly challenges the recognition of aspect-based sentiment.
Considering the multimodal input, existing methods are typically dedicated to associated visual and textual contents (Ju et al., 2021; Ling et al., 2022; Yang et al., 2022). Ju et al. (2021) uses imagetext relation to evaluate the contribution of visual contents to aspect sentiment, based on which to determine whether the image is involved in sentiment analysis. Ling et al. (2022) and Yang et al. (2022) align visual representations of objects and their attributes with corresponding textual contents. To summarize, the whole image is directly associated with textual content in these methods. Intuitively, without aligning image blocks to corresponding aspects, the coarse whole-image-text association can introduce aspect-irrelevant visual noise, which further hinders aspect sentiment analysis. In addition, the performance can be further impacted by the
8184
textual noise from the confusion among different aspects.
In this paper, we propose an Aspect-oriented Method (AoM) to mitigate aforementioned noises from both image and text. AoM can detect aspectrelevant information from perspectives of both semantics and sentiment. There are two key modules in AoM: Aspect-Aware Attention Module (A3M) for semantically fine-grained image-text alignment and Aspect-Guided Graph Convolutional Network (AG-GCN) for sentiment information aggregation. In A3M, we first extract aspect features associated with each visual and textual token. And then aspectrelevant token representations are computed based on their relevance to the corresponding aspect features. In AG-GCN, we first explicitly add sentiment embeddings to the obtained representations of visual and textual tokens. A multimodal weightedassociation matrix is constructed containing aspectto-image-block similarity and word-to-word dependency. Then we use a graph convolutional network to aggregate sentiment information according to the constructed multimodal matrix.
The contributions can be summarized as follows: (1) We propose an aspect-oriented network to mitigate the visual and textual noises from the complex image-text interactions.
(2) We design an aspect-aware attention module and an aspect-guided graph convolutional network to effectively detect aspect-relevant multimodal contents from the perspectives of semantic and sentiment, respectively.
(3) Experiments on two benchmark datasets, including Twitter2015 and Twitter2017, show that our approach generally outperforms the state-ofthe-art methods.

heading: 2 Related Work
text: In this section, we review the existing methods for both ABSA and MABSA.

heading: 2.1 Aspect-based Sentiment Analysis
text: In the past few years, Aspect-Based Sentiment Analysis (ABSA) in the textual fields has attracted much attention and gained mature research (Chen and Qian, 2020; Oh et al., 2021; Xu et al., 2020). On the one hand, most recent works are based on the pre-trained language model BERT because of its remarkable performance in many NLP tasks (Liang et al., 2022a). On the other hand, some recent efforts focus on modeling the dependency
relationship between aspects and their corresponding descriptions, in which graph convolutional networks (GCNs) (Chen et al., 2022; Liang et al., 2022b, 2020; Li et al., 2021a; Pang et al., 2021) or graph attention networks (GATs) (Yuan et al., 2020) over dependency with the syntactic structure of a sentence are fully exploited.

heading: 2.2 Multimodal Aspect-based Sentiment Analysis
text: With the enrichment of multimodal users’ posts in social media, researchers find that images offer great supplementary information in aspect term extraction (Wu et al., 2020a; Zhang et al., 2018; Asgari-Chenaghlu et al., 2021) and sentiment analysis (Wu et al., 2022; Li et al., 2021b; Hazarika et al., 2020; Cai et al., 2019). Thus, Multimodal Aspectbased Sentiment Analysis (MABSA) begins to be widely studied. MABSA task can be divided into two independent sub-tasks, i.e., Multimodal Aspect Term Extraction (MATE) and Multimodal Aspectoriented Sentiment Classification (MASC). The former extracts all aspect terms in the sentence at the prompt of the image, and the latter predicts the sentiment polarities for the aspects.
Ju et al. (2021) first realizes MABSA in a unified framework and designs an auxiliary cross-modal relation detection to control whether the visual information will be used in prediction. For capturing cross-modal alignment, Ling et al. (2022) constructs a generative multimodal architecture based on BART for both vision-language pre-training and the downstream MABSA tasks. Yang et al. (2022) dynamically controls the contributions of the visual information to different aspects via the trick that the lower confidence of the results predicted by purely textual is, the more contributions from images will be considered.
On the one hand, the above methods ignore the alignment of fine-grained visual blocks and the corresponding aspects, which introduce irrelevant visual noise. On the other hand, modeling of syntax dependency and sentiment information for aspect descriptions is absent in these methods, which is proved important in sentiment analysis (Liang et al., 2022a; Kalaivani et al., 2022; Xu et al., 2022).
To tackle the aforementioned issues, we propose an aspect-oriented model consisting of AspectAware Attention Module and Aspect-Guided Graph Convolutional Network which respectively work to capture semantic information by fine-grained
image-text alignment and effectively aggregate aspect-relevant sentiment information.

heading: 3 Methodology

heading: 3.1 Overview
text: Task Definition. Formally, given a tweet that contains an image V and a sentence with n words S = (w1, w2, ..., wn), our goal is to acquire the sequence Y representing all aspects and their associated sentiment polarities. We formulate the output of MABSA as Y = [as1, a e 1, s1, ..., a s i , a e i , si, ...a s k, a e k, sk], where a s i , a e i and si depict the start index, end index of the i-th aspect and its sentiment polarity in the tweet, and k is the number of aspects. Model preview. Figure 2 shows the overview of our model architecture, which builds on an encoder-decoder architecture based on BART (Lewis et al., 2019). Between the encoder and the decoder of BART, we creatively implement the Aspect-Aware Attention Module (A3M) and Aspect-Guided Graph Convolutional Network (AGGCN) to align the textual aspect to its associated visual blocks and textual description, simultaneously mitigate interference both from semantics and sentiment among different aspects. In the following subsections, we will illustrate the details of
the proposed model. Feature Extractor. The initial word embeddings are obtained from the pre-trained BART due to its excellent ability of textual representation. The embeddings of visual blocks are obtained by preprocessing via ResNet (Chen et al., 2014) following (Yu et al., 2019). We consider every feature of a visual block or word token as an atomic feature. We add <img> and </img> before and after the visual features, <bos> and <eos> for the textual features. Then, we concatenate the multimodal features as X which is the input of BART encoder.
We can get the multimodal hidden state H = {hV0 , ...hVi , ...hVm, hT0 , ..., hTj , ...hTn} with m visual blocks and n words, where hVi and h T j refer to features of the i-th visual block and the j-th word in the sentence.
3.2 Aspect-Aware Attention Module (A3M)
Since aspects are not specially modeled by BART encoder, we creatively design the Aspect-Aware Attention Module (A3M) aiming to capture aspectrelevant semantic information. For this purpose, we align the multimodal information of target objects and filter out the semantic noise from images.
First, as aspects are usually noun phrases from the sentences, we extract those phrases as the
candidate aspects (CA) with the NLP tool Spacy1. And from the hidden state H of the BART encoder, we obtain the features of all candidate aspects denoted as HCA = {hCA1 , ..., hCAi , ..., hCAl }, where l is the number of noun phrases in the sentence. To get the relationship between candidate aspects and atomic features, we implement an attention-based mechanism guided by the candidate aspects. Given the t-th hidden feature ht, its attention distribution αt over k candidate aspects is obtained by:
Zt = tanh((WCAH CA + bCA)⊕ (WHht + bH)), (1)
αt = softmax(WαZt + bα), (2)
where Zt ∈ R2d×k is the comprehensive feature extracted from both the candidate aspects and the hidden states. HCA ∈ Rd×k denotes the features of candidate aspects. WCA ∈ Rd×d, WH ∈ Rd×d, Wα ∈ R1×2d, bCA, bH and bα are the learned parameters.⊕ is an operator between a matrix and a vector, where the vector is repeated into the appropriate size to concatenate with the matrix. We then get the aspect-related hidden feature hAt by calculating the weighted sum of all candidate aspects following the below equation:
hAt = k∑
i
αt,ih CA i . (3)
For example, if a visual block is strongly associated with the j-th aspect, the corresponding αt,j is approximately 1. hAt would be equal to the aspect semantically. And if the visual block is not related to any specific candidate aspects, both αt and hAt would be zero-like vectors of no information.
Considering that not every visual block can be used for prediction, βt is learned to add up the atomic feature ht and its aspect-related hidden feature hAt . Details are as follows:
βt = sigmoid(Wβ[W1ht;W2h A t ] + bβ), (4)
ĥt = βtht + (1− βt)hAt , (5) where Wβ , W1, W2, bβ are parameters, and [; ] denotes the concatenation operator for vectors. ĥt ∈ Ĥ is the final output of A3M after the semantic alignment and the noise reduction procedure. Thus we get the noiseless and aligned information for every atomic feature.
Pre-training To align the two modalities and reduce noise, we conduct a pre-training task in A3M.
1Spacy: https://spacy.io/
Specifically, we detect the image-text relationship on the datasets TRC (Vempala and Preoţiuc-Pietro, 2019) as illustrated by Figure 3. We first obtain the average feature of image blocks from the output of A3M and then pass it to a fully connected softmax layer, which outputs a probability distribution over whether the image is related to the text. Finally, we use cross entropy loss to train our model.

heading: 3.3 Aspect-Guided Graph Convolutional Network (AG-GCN)
text: The aspect-focused interaction between visual modality and textual modality in A3M concentrates on the context semantics, and that is not adequate for MABSA. Sentiment interference among different aspects still exists and influences sentiment prediction. Thus, we design the Aspect-Guided Graph Convolutional Network (AG-GCN) module to introduce external sentiment information and mitigate emotional confusion among different aspects to a certain extent.
Specifically, for word wi in the sentence, we gain its affective score wSi from SenticNet (Ma et al., 2018) and project it to the space with the same dimension as hAt , with si obtained. Then we add the sentiment feature si to the output of A3M:
wSi = SenticNet(wi), (6)
si = WSw S i + bS , (7)
hSi = ĥi + si, (8)
where WS , bS are the learned parameters. hSi is the feature with affective knowledge.
Next, we build a boolean dependency matrix D among visual blocks and words. First, for the word-to-word part, submatrix DTT representing
dependency tree
the dependency tree2 of the input sentence like Figure 4. If two words can be associated within two generations, the element of DTT would be set to 1, otherwise 0 instead. For example, “Kyoto” is associated with “bit” (child),“a” (grandchild),“about” (father) and “Complain” (grandfather). Second, the visual dependency submatrix DV V is initialized as a diagonal matrix. And as for the word-imageblock dependency, denoted as DTV and equaled to DTV T , we set all the elements in the i-th line of DTV to 1 if the i-th word is an aspect, otherwise 0. And the matrix D is defined as:
D = [ DV V DV T DTV DTT ] , (9)
Considering the different importance of different dependencies, we attach weights onto D with cosine similarity among ĥi as follows:
Aij = DijFcosine_similarity(ĥi, ĥj), (10)
where both D,A ∈ R(m+n)×(m+n), and A is the weighted association matrix.
AG-GCN takes HS from Eq.8 as initial node representations in the graph. For the i-th node at the l-th layer, the hidden state hSi,l is updated by the following equation:
hSi,l = ReLU( n∑
j=1
AijWlh S i,l−1 + bl), (11)
where Wl,bl are learned parameters and we use ReLU as activation function. Significantly, hSi,0 is equal to hSi . Accordingly, we get the final output ĤS from the last GCN layer which is rich in sentiment information. Every underlying aspect aggregates its relevant information from both the image-text pair. Moreover, sentiment confusion of different aspects is weakened because the association matrix makes little interference among different aspects.
2We use spaCy toolkit to construct the dependency tree referring from https://spacy.io

heading: 3.4 Prediction and Loss Function
text: The BART decoder takes the combination of Ĥ , ĤS , and the previous decoder output Y<t as inputs, and predicts the token probability distribution as follows:
H̃ = λ1Ĥ + λ2Ĥ S , (12)
hdt = Decoder(H̃;Y<t) (13)
HT = (W + H̃T )/2 (14)
P (yt) = softmax([HT ;C d]hdt ) (15)
where λ1, λ2 are the hyper-parameters to control the contribution from the two modules. H̃T is the textual part of H̃ . W denotes the embeddings of input tokens. Cd means the embeddings of the [ positive, neutral, negative, <eos>]. The loss function is as follows:
L = −EX∼D O∑
t=1
logP (yt|Y<t, X), (16)
where O = 2M + 2N + 2 is the length of Y, and X denotes the multimodal input.

heading: 4 Experiment

heading: 4.1 Experimental settings
text: Datasets. Our two benchmark datasets are Twitter2015 and Twitter2017 (Yu and Jiang (2019)). As shown in the statistics of Table 1, sentences with multiple aspects take up a considerable part of the two datasets. Implementation Details. Our model is based on BART (Lewis et al., 2019), and the pre-training task is trained for 40 epochs with batch size 64, and for 35 epochs with batch size 16 on MABSA. The learning rates are both 7e-5 and hidden sizes are 768. Hyper-parameters λ1 and λ2 are 1 and 0.5 respectively. Besides, we pre-train A3M on TRC dataset (Vempala and Preoţiuc-Pietro, 2019), which is divided into two groups according to whether the text is represented.
Evaluation Metrics. We evaluate the performance of our model on MABSA task and MATE task by Micro-F1 score (F1), Precision (P) and Recall (R), while on MASC task we use Accuracy (Acc) and F1 following previous studies.

heading: 4.2 Baselines
text: We compare our proposed model with four types of methods listed below.
Approaches for textual ABSA. 1) SPAN (Hu et al., 2019) detects opinion targets with their sentiments. 2) D-GCN (Chen et al., 2020) models dependency relations among words via dependency tree. 3) BART (Yan et al., 2021) solves seven ABSA subtasks in an end-to-end framework.
Approaches for MATE. 1) RAN (Wu et al., 2020b) focus on alignment of text and object regions. 2) UMT (Yu et al., 2020) takes text-based entity span detection as an auxiliary task. 3) OSCGA (Wu et al., 2020c) foucus on alignments of visual objects and entities.
Approaches for MASC. 1) ESAFN (Yu et al., 2019) is an entity-level sentiment analysis method based on LSTM. 2) TomBERT (Yu and Jiang, 2019) applies BERT to obtain aspect-sensitive textual representations. 3) CapTrBERT (Khan and Fu, 2021) translates images into text and construct an auxiliary sentence for fusion.
Approaches for MABSA. 1) UMT-collapse (Yu et al., 2020), OSCGA-collapse (Wu et al., 2020c) and RpBERT-collapse (Sun et al., 2021) are adapted from models for MATE by using collapsed labels to represent aspect and sentiment pairs. 2) UMT+TomBERT, OSCGA+TomBERT are two pipeline approaches by combining UMT (Yu et al., 2020) or OSCGA (Wu et al., 2020c) with
TomBERT (Yu and Jiang, 2019). 3) JML (Ju et al., 2021) is the first joint model for MABSA with auxiliary cross-modal relation detection module. 4) CMMT (Yang et al., 2022) implements a gate to control the multimodal information contributions during inter-modal interactions. 5) VLP-MABSA (Ling et al., 2022) performs five task-specific pretraining tasks to model aspects, opinions and alignments.

heading: 4.3 Main Results
text: In this section, we show the excellent performance of AoM on the two datasets for the three tasks compared with SOTAs.
Performance on MABSA: The results for MABSA are shown in Table 2. First, our AoM far exceeds all text-based models, which means detection of richer visual information and textual information in our model is helpful. Second, multimodal pipeline methods and adaptive methods are generally unsatisfactory, because they ignore the interaction between the semantic information and sentiment for the two sub-tasks. Last, AoM outperforms all multimodal methods in every metric. Especially, AoM achieves the improvement of 2% and 1.2% with respect to F1 in contrast with the second best models on two datasets (VLP-MABSA for Twitter2015 and CMMT for Twitter2017), which demonstrates the effectiveness of learning aspectrelevant visual blocks and textual words compared to focusing on all visual and textual inputs.
Performance on MATE: As shown in Table 3, AoM is ahead of most of the current models and performs the best in Twitter 2015 by 0.3% higher than the second best CMMT on F1. The performance of CMMT in Twitter2017 is 0.8% higher
than ours, probably due to our model wrongly predicting some noun phrases as aspects. But considering the improvement in MASC and MABSA, it is still worthy treating all noun phrases in the sentence as candidate aspects when acquiring aspectrelevant visual information.
Performance on MASC: Table 4 shows the performance of MASC. It is exciting that our model outperforms the second-best results by 1.5% and 2.6% in accuracy, 2.1% and 3.2% points in F1 score on Twitter2015 and Twitter2017. It demonstrates that AoM has the ability to detect aspect-related sentiment information from both images and text, even disturbed by other noisy aspects.

heading: 4.4 Ablation Study
text: In this section, we research the effectiveness of each component in AoM, the results are shown in Table 5.
W/o A3M&AG-GCN shows that after removing the two specially designed modules, the per-
formance declines by 2.1% on Twitter2015 and 1.9% on Twitter2017. It fully demonstrates their contributions to learning effective information.
W/o A3M&TRC performs worse after removing A3M including the pre-training on TRC. It proves the necessity of modeling semantic alignment between visual blocks and aspects in A3M. With the alignment, AG-GCN can obtain appropriate aspect-image-block and text-text association.
W/o TRC pre-training shows a slight drop after we remove the TRC pre-training on A3M, which implies relevant pre-training task is useful for the model to learn better parameters.
W/o AG-GCN displays the performance without AG-GCN, declining by 0.42% on Twitter2015 and 0.9% on Twitter2017. It means that AG-GCN does make the prediction focus on specific aspects related to blocks and words with syntax dependencies. In other words, the multimodal interference from other aspects can be mitigated.
W/o SenticNet is the model without sentiment information in AG-GCN. Its performance shows adding external affective knowledge can enhance the sentiment comprehension of the model.
W/o TRC&AG-GCN is the BART model only with our A3M module. We can see from Table 5 that w/o TRC&AG-GCN improves w/o A3M&AGGCN by 1.5% and 0.8%. So it is effective to align the fine-grained visual block to related aspect and reduce irrelevant information.

heading: 4.5 Case Study
text: To better analyze how the Aspect-Aware Attention Module and Aspect-Guided Graph Convolutional Network work, we present the case study as follows.
Figure 5 displays two examples with predictions from VLP-MABSA (Ling et al., 2022),
A3M
Inputs
AG-GCN
BART+A3M and our AoM. In example (a), VLPMABSA misses the aspect “Golden State Warriors”, gets an incomplete aspect “Oklahoma City Thunder” and wrongly predicts the sentiment. It may be caused by the interference from the visual region which represents pride expression of a person. However, BART+A3M gets all right predictions due to the ability of aspect-oriented attention. In example (b), compared with our whole model, BART+A3M wrongly predicts the sentiment of “Daniel Radcliffe” which should be negative. We attribute the wrong prediction to lacking syntax association which benefits sentiment transmission. In other words, AG-GCN contributes to the correctness.

heading: 4.6 Attention Visualization
text: To investigate the effectiveness of detecting aspectrelevant information, we visualize the attention process as shown in Figure 6.
For A3M: (i) Figure 6-(I.a) shows the attention weights of candidate aspects computed according to the images. We can see that “Mayor Kadokawa” is the most relevant aspect. (ii) Figure 6-(I.b) shows the proportions of the visual information reserved at the last step in A3M, where we weighted add up the representations of visual blocks and the corresponding aspects. The heat map shows that the visual information associated with “Mayor Kadokawa” is reserved to a great extent, while the helpless information from other blocks is disregarded as noise. It demonstrates that attention in A3M is able to detect aspect-relevant information.
For AG-GCN: (i) Figure 6-(II.a) shows the word-to-word part of the weighted association matrix. The matrix effectively excludes sentiment interference from other aspects by adding syntax dependency information. For example, the sentiment of “mayor” cannot be influenced by irrelevant keywords, such as “Complain” and “thanks”. (ii) Figure 6-(II.b) shows the dependencies between visual blocks and words. (iii) Specifically, we visualize the visual attention of aspects “Kyoto” (see Figure 6-(II.c) left) and “Mayor Kadokawa” (see Figure 6-(II.c) right). We can see that “Kyoto” pays more attention to the pictures hanging on the wall which are full of Japanese elements related to the place, while “Mayor Kadokawa” focus more on the joyful expressions of the two people. (iv) Figure 6-(II.d) shows the words and image blocks “Mayor Kadokawa” focused on in sentiment transmission. It’s obvious that these attentions are helpful for the prediction.

heading: 5 Conclusion
text: In this paper, we proposed an aspect-oriented model (AoM) for the task of multimodal aspectbased sentiment analysis. We use two specially designed modules to detect aspect-relevant information from the semantic and sentiment perspectives. On the one hand, to learn aspect-relevant semantic information especially from the image, we construct the Aspect-Aware Attention Module to align the visual information and descriptions to the corresponding aspect. On the other hand, to detect the aspect-relevant sentiment information,
we explicitly add sentiment embedding into AoM. Then, a graph convolutional network is used to aggregate the semantic and sentiment embedding under the guidance of both image-text similarity and syntax dependency in sentences. The experimental results on two widely used datasets demonstrate the effectiveness of our method.
Limitations
Though our proposed method outperforms current state-of-the-art methods, there are still many challenges we should overcome in future research. First, for colloquial expression which confuses current dependency tree parser, we should come up with new solutions. Second, emotional prediction of tweet posts describing current issues needs external knowledge, which is absent in existing research.

heading: Acknowledgments
text: We thank anonymous reviewers for their valuable comments. This work was supported by the Natural Science Foundation of Tianjin, China (No.22JCJQJC00150, 22JCQNJC01580), the National Natural Science Foundation of China (No.62272250), Tianjin Research Innovation Project for Postgraduate Students (No.2022SKYZ232), and the Fundamental Research Funds for the Central Universities (No. 63231149).

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: We discuss it in section Limitations.
7 A2. Did you discuss any potential risks of your work? Our research is foundational research and not tied to particular applications.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: In the abstract and section 1 Introduction.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: In section 3 Methodology and 4 Experiment.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: In section 3.1 Overview and 4 Experiment.
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The datasets used in section 4 and pre-trained models in section 3 are in public domain and licensed for research purposes.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? In section 4 Experiment.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. The data is in public domain and licensed for research purposes.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? In section 4 Experiment.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. In section 4 Experiment.
C 3 Did you run computational experiments?

heading: In section 4 Experiment.
text: 7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? We only use the most commonly used pre-trained models and the parameters or GPU hours are not focus of our research.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? In section 4 Experiment.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? In section 4 Experiment.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? In section 4 Experiment.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Limitations
Though our proposed method outperforms current state-of-the-art methods, there are still many challenges we should overcome in future research. First, for colloquial expression which confuses current dependency tree parser, we should come up with new solutions. Second, emotional prediction of tweet posts describing current issues needs external knowledge, which is absent in existing research