A Multi-task Framework with Enhanced Hierarchical Attention for Sentiment Analysis on Classical C hinese Poetry: Utilizing Information from Short Lines (2024)

Abstract: Classical Chinese poetry has a long history, dating back to the 11th century BC. By investigating the sentiment expressed in the poetry, we can gain more insights in the emotional life and history development in ancient Chinese culture. To help improve the sentiment analysis performance in the field of classical Chinese poetry, we propose to utilize the unique information from the individual short lines that compose the poem, and introduce a multi-task framework with hierarchical attention enhanced with short line sentiment labels. Specifically, the multi-task framework comprises sentiment analysis for both the overall poem and the short lines, while the hierarchical attention consists of word- and sentence-level attention, with the latter enhanced with additional information from short line sentiments. Our experimental results showcase that our approach leveraging more fine-grained information from short lines outperforms the state-of-the-art, achieving an accuracy score of 72.88% and an F1-macro score of 71.05%.

## introduction classical chinese poetry generally refers to chinese poetry composed in certain traditional forms and written in classical chinese, a special terse, rhythmic and musical version of chinese that is different from the modern chinese today and mainly used in classical literature. as one of the earliest forms and most important genres of traditional literature, classical chinese poetry is a crucial carrier of the thoughts and emotions of ancient chinese literati. 1 in fact, it is widely believed that "emotion" is the essence of classical chinese literary culture @xcite . by investigating the emotion expression in classical chinese poetry, we are able to have a glimpse of the personal and social issues involved in the poetry, through which we can 1 https://www.zgbk.com/ecph/words?@xmath0. however, compared with the study of emotion in other text genres we encounter in modern life, such as news articles @xcite and reviews @xcite , there is less research on poetry @xcite , not to mention on classical chinese poetry @xcite . actually, it is more challenging to process ancient chinese texts than modern chinese text. on the one hand, the classical chinese has some old vocabulary and phrases that have been replaced or fallen out of usage in modern chinese. for instance, the pronouns in ancient chinese are quite different from those in modern chinese, as shown in figure 1. on the other hand, the grammar of ancient chinese is more concise and flexible compared to modern chinese. as for classical chinese poetry, it is considered as a more complicated synthesis of imagery and symbolism, allusions, quotations and derivation, and antithesis @xcite . all these factors make it a difficult task to conduct sentiment analysis in classical chinese poetry. to analyse sentiment in classical chinese poetry, previous studies have explored different methods, for example, constructing sentiment lexicons @xcite , transferring knowledge from modern chinese @xcite , or extracting imagery words @xcite . although these studies improved the general performance for the task of sentiment analysis in classical chinese poetry, by utilizing special words in the poems or drawing upon knowledge beyond the poems, they did not consider the compositional structure of the poems. usually, a classical chinese poem comprises several short lines, which may show different emotions, and in return, contribute to the overall emotion expression of the poem. thus in this paper, for the task of sentiment analysis of classical chinese poetry, we propose to take the sentiment of short lines into consideration by using a multi-task framework with a hierarchical attention network, which includes the sentiment analysis task of both the overall poem and the short lines of which the poem is comprised. we will show that, by leveraging the sentiment information from the short lines, we can outperform the current state-of-the-art in sentiment analysis of ancient chinese poetry. ## dataset building a classical chinese poetry corpus for sentiment analysis is not trivial, since full comprehension of the poems is one of the most important preconditions for sentiment labelling, which sets high expectations for annotators. thanks to the work of @xcite , we now have a finegrained sentimental poetry corpus (fspc), which we are going to utilize in our experiments. fspc is composed of 5,000 classical chinese poems. each poem and each line in the poem are manually annotated by experts in chinese literature with five classes, ranging from negative and implicit negative, over neutral, and to implicit positive and positive, as shown in figure 2 . compared with negative and positive, implicit sentiments refer to the emotions that are suggested or hinted at but not directly stated. due to the unbalanced label distribution, the implicit sentiments are merged into negative or positive, respectively, in later experiments. it should also be noted that the sentiment of each line can be different from each other, as shown in figure 3 , with the sentiment of the first line being neutral or implicit while the last line is often aligned with the overall sentiment of the poem @xcite . this suggests each line of the poem may have a varied contribution to the holistic sentiment of the whole poem, and it would be beneficial if we take into consideration the unique sentiment label of each line in the sentiment analysis of the whole poem. ## method to investigate whether the task of sentiment analysis for the whole poem can benefit from the introduction of short line information, we use a framework that fine-tunes a pre-trained model with labels of both the lines and the overall poem, as shown in figure 4 . data pre-processing each poem consists of 4 lines, and along with the poem, sentiment labels of both the lines and the overall poem are used for the pre-trained model fine-tuning. the use of "|" as separators between short lines in fspc makes it easy to identify the line border. feature extraction and fine-tuning the pretrained model is used both in the feature extraction and fine-tuning stage. in the former case, the architecture of the pre-trained model is preserved, taking in the texts and outputting the feature vectors for the next stage. in the fine-tuning stage, two linear transformation layers are added at the end of the pre-trained model as classifiers for sentiment in the short lines and the overall poem, respectively. both classifier 1 and classifier 2 use word-level attention, and sentence-level attention is also applied in classifier 2. for both the lines and the overall poem, the cross-entropy loss between the predicted logits and the true labels is calculated. the two loss values are then added to get a combined loss, based on which backpropagation is performed to compute gradients for model parameter updates. it should be noted that for the short lines, each of them is independently encoded, and the encoding of the whole poem and the short lines are conducted separately. enhanced sentence-level attention to exploit the short line label information in the sentiment prediction task of the overall poem, additional information derived from the line labels is integrated into the sentence-level attention scores. output evaluation for the evaluation of the model predictions, both accuracy and f1-macro score are applied. accuracy is one of the most intuitive performance evaluation metrics, but it may be misleading due to its sensitivity to class imbalance, which is why we also report the f1-macro score. ## prediction analysis with the ten-fold cross-validation in experiment 2, we obtained predictions for all 5,000 poems using the sikuroberta model in the multi-task framework with line-label-enhanced hierarchical attention, as shown in figure 5 . first of all, it is clear that the model performs better with poems with clear sentiment polarities, i.e., negative and positive, reaching an accuracy of around 80%, while the model has only fair predictions with neutral poems, with an accuracy of about 52%. for the negative and positive poems, more wrong predictions are distributed on their nearest sentiment neighbour, i.e., neutral, rather than the far/opposite sentiment neighbours, i.e., positive and negative respectively, suggesting the model's ability to differentiate between positive and negative poems. as for the neutral poems, the model achieves an accuracy of 52.03%, which is far less than the performance with negative and positive poems, indicating that differentiating between neutral and implicit sentiment is still challenging. although only about half are correctly predicted, around one-third of the wrong predictions are predicted as negative and the rest as positive, with the latter outnumbering and even doubling the former, indicating that the model tends to have more non-negative predictions on neutral poems. to gain more insights into this tendency, we further investigated the short line labels of 194 false negative poems and 443 false positive poems which are labelled as neutral by the annotators as shown in figure 5 . for both the false negative and false positive groups of poems, we plotted the sentiment label distribution of each line group (i.e. all first lines are joined into one group, the same holds for the second lines, etc.), and the results are shown in figure 6 and 7 . as shown in figure 6 , for the 194 neutral poems that are predicted as negative by the model, line 1 and line 2 seem to hold sentiment labels that contain almost equal percentages of negative and positive labels. in line 3, however, there is more negative sentiment labelled than positive and this tendency continues in line 4, where the negative sentiment reaches 41.75%, which is more than four times of the number of positive annotations. on the other hand, as shown in figure 7 , when we consider the sentiment annotations at the line level of the 443 neutral poems that were predicted as positive by the model, we see an opposite tendency with always more positive sentiment labels than negative ones. based on the sentiment distribution across short lines, we hypothesize that although the model fails to make correct predictions for about half of the neutral poems, it still senses the more implicit sentiment polarities in the poems, leading to negative predictions for the more negative poems and positive predictions for the more positive poems. moreover, we also investigate specific poems where there is a significant difference between the true labels and the predictions. by significant difference, we refer to the cases where the poem is classified with the complete opposite polarity of its labeled polarity, e.g. the overall sentiment of the poem is predicted as positive while the true label is negative, or vice versa. of the 5,000 poems, there were in total 144 poems classified with this opposite polarity, and we found that this opposite polarity labelling happens both at the line level and the poem level. figure 8 shows an example poem with its manual annotations and model predictions for the short lines and the overall poem. for the short lines, the model predicted positive instead of neutral for the first and the third lines, and predicted neutral instead of negative for the last line, generally upgrading the short line sentiment toward the polarity of positive, which might lead to the positive overall prediction. however, if contextual or topic information of the poem, which is "farewell", would be provided, and if the model learns from other poems that "farewell" is usually related to negative emotions, the possibility of the model predicting this poem as overall negative would increase. ## conclusion in this paper, we introduced a multi-task framework with enhanced hierarchical attention for sentiment analysis on classical chinese poetry. this multi-task framework consists of two sub-tasks, the sub-task of sentiment analysis on the short lines in the poem, and the sub-task of sentiment analysis on the overall poem. for the latter task, a hierarchical attention network composed of word-and sentence-level attention was applied. furthermore, to further utilize the information from short lines, additional information from short line sentiment was introduced to the sentence-level attention. experiments on the fspc dataset show that our framework, compared with the single-task setup and based on the pre-trained model sikubert, yields a increase of macro f1 from 66.27% to 68.51%. we also experimented with other pretrained models, and the best performance is demonstrated by sikuroberta, with an accuracy of 72.88% and an f1-macro of 71.05%, thus largely outperforming the state-of-the-art with an increase 5.78% in accuracy and 6.25% in f1-macro score @xcite . we also investigated the sikuroberta-based model predictions in more detail. we found that the model achieves an accuracy of about 80% with negative and positive poems, but only an accuracy of around 50% with neutral poems, which might be related to the "implicit emotion" writing style favoured in classical chinese poems. moreover, a further look at the predictions on the neutral poems suggested that the incorrect predictions are subtly aligned with the more prominent positive or negative sentiment polarities in the short lines, which indicates that regardless of the incorrect predictions on the neutral poems, the model still senses whether the poems are more positive or negative.