Semantic-Based Opinion Summarization (2021)

Abstract: The amount of information available online can be overwhelming for users to digest, specially when dealing with other users’ comments when making a decision about buying a product or service. In this context, opinion summarization systems are of great value, extracting important information from the texts and presenting them to the user in a more understandable manner. It is also known that the usage of semantic representations can benefit the quality of the generated summaries. This paper aims at developing opinion summarization methods based on Abstract Meaning Representation of texts in the Brazilian Portuguese language. Four different methods have been investigated, alongside some literature approaches. The results show that a Machine Learning-based method produced summaries of higher quality, outperforming other literature techniques on manually constructed semantic graphs. We also show that using parsed graphs over manually annotated ones harmed the output. Finally, an analysis of how important different types of information are for the summarization process suggests that using Sentiment Analysis features did not improve summary quality.

## introduction with the advance of web technologies during the last decades, a numerous amount of textual data is produced constantly, especially user generated content within social media and e-commerce domains, which include opinions towards many entities, such as products, organizations, and others. such quantity of information is virtually impossible for other users to assimilate by their own, which creates a need for automated content selection and summarization methods, originating the opinion summarization research area, which integrates both the text summarization and sentiment analysis areas. @xcite . there are two main types of summaries: extracts, which are composed by copying excerpts from the original texts literally, and abstracts, consisting of new text constructions derived out of the information obtained from the input data. some authors in the literature call attention to the need of incorporating semantic knowledge into the summarization process @xcite @xcite , particularly for producing abstracts, since humans usually create summaries by rewriting, paraphrasing and, mainly, interpreting texts @xcite . this kind of information can be explicitly formalized as semantic representations, such as predicate argument structure (pas) @xcite , abstract meaning representation (amr) @xcite @xcite and others. there are also arguments promoting the usage of explicit semantic representations to deal with texts from the opinions genre -in which the sentiment expressed by the author is an important information -as this type of linguistic knowledge is able to associate correlated information scattered throughout the texts and also to deal with many different semantic phenomena (such as negations and multiword expressions) that can alter the interpretation that would be obtained in a word-by-word basis @xcite . however, there are, to the best of our knowledge, no works that focus on integrating these representations for opinion summarization. in this context, this paper focuses on exploring techniques of text summarization based on explicit semantic representations upon opinions. as a base to instantiate the semantic knowledge, we used the abstract meaning representation (amr) @xcite , which has already been used in previous research in text summarization @xcite @xcite and has also been adapted for the portuguese language in other former works @xcite . as a result, we present one of the first applications of methods based on explicit semantic representations for opinion summarization, showing that the random forest machine learning algorithm produced the best summary graphs, according to the metrics applied, outperforming other amrbased approaches from the literature for manually annotated graphs. the results also indicate, in the same manner as previous works @xcite , that the quality of the amr graphs used have an impact on the output. the experiments also suggest that using sentiment analysis features do not improve summary quality. in short, the main contributions of this paper are the following: the remaining of the text is organized as follows: in section 2, we present a general notion of the amr. later, in section 3, previous works in text summarization are outlined with main focus on the ones based on the chosen semantic representation (amr). our methods are shown in section 4 followed in section 5 by some important notes about the experiments executed, whose results are then introduced in section 6. finally, some conclusions and further discussing are made in section 7. ## abstract meaning representation sentences in amr are represented as rooted directed graphs in which nodes correspond to concepts and edges to relations between them. an example can be seen in figure 1 . concepts can be of different kinds, e.g framesets (possible-01, recommend-01, ler-01) originated from separate repositories containing predicates and their corresponding arguments. for the english language, this repository is called propbank foot_1 @xcite , whilst for brazilian portuguese, there is verbobrasil foot_2 @xcite . the concepts can also be words from the base language, extracted directly, as they are, from the sentence (e, eu, coisa) or derived by some annotation procedure provided in the guidelines, such as the transformation of the comparative "melhor" (better) into a degree relation between "mais" (more) and "bom" (good) (bom, mais). there are also concepts that are specific of the representation formalism (e.g., have-degree-91). all edges are of two types: argument relations defined for each predicate (arg0, arg1, arg2, arg3) and the ones determined by the amr project (op1, op2). as the original representation was developed specifically for the english language, there is need for adaptations of the guidelines foot_3 towards other languages. for brazilian portuguese, this initiative was made by anchiêta and pardo (2018b), followed by @xcite , dealing with some phenomena that are not so common in english, such as indeterminate and hidden subjects. ## related work there are two main areas related to our work: summarization based on semantic representations and opinion summarization, each with a long history in the literature. thus, we present the works most related to ours, those using the abstract meaning representation and other opinion summarization methods. ## methods as discussed before, the usage of explicit semantic information represents an interesting path for both text summarization and sentiment analysis, consequently for opinion summarization. thus, we chose the abstract meaning representation as an instance of semantic knowledge. the starting point of our research is, therefore, all the previous works which exploited this representation in the process of summarizing texts @xcite @xcite , even if for other genres rather than opinions. first, some preprocessing is necessary. as amr is a sentence-level representation, we need to combine multiple sentences into a multi-document graph. this has been done, in this work, similarly to @xcite by merging every node that contains the same concept. some semantic unitssuch as named entities or dates -are represented in multiple nodes, which are collapsed before the merging, only allowing the combination of entities with the same overall information (same name or date). this was applied upon all comments about each product, resulting in a multi-document semantic representation of all opinions about some given book or electronic device. ## experiments setup as a source of opinions, we used the opisums-pt corpus 6 (condori et al., 2015), comprising the opinions from reli @xcite (concerning 13 books) and also comments about 4 electronic products, obtained from the buscapé e-commerce website foot_6 . this corpus also comprehends 10 summaries for each product, 5 extractive and 5 abstractive, created by 14 human specialists. thus, the corpus contains 171 opinionative documents with a total of 1,502 sentences. following the amr guidelines for the portuguese language, 404 sentences from the corpus were manually annotated foot_7 . two products of the opisums-pt corpus have been fully annotated into amr to this date. thus, in this work, they are used as a gold standard test set. in order to compare how the quality of the graphs used with each method interfere in the results, we used an automatic amr parser (anchiêta and pardo, 2018a) to annotate two similar products foot_8 , resulting in a total of 4 products in the test set. we also made use of the parser in order to create an artificial training corpus for the proposed supervised approaches (via score optimization or machine learning) from the 13 remaining products of the opisums-pt, resulting in 65 examples (pairs of opinions graphs and their corresponding summary graph), with a total of 3,290 amr sentences, from which 388 are considered relevant to compose the summaries. another important resource used is the b2w-reviews01 corpus 10 (real et al., 2019), which was used, in conjunction with the reli @xcite corpus (excluding the two products used for the testing) to calculate the document frequency term in the tf-idf counts. for evaluation, we adopted two metrics from the amr research community: smatch @xcite and sema @xcite , which are used to compare two amr graphs (the automatic summary graph and its corresponding gold standard manually annotated graph). as the opisums-pt corpus provides 5 summaries for each product, all of them have been used to evaluate the summarization output, however, due to lack of space, we report only the average of these values. we also would like to call attention to the fact that the simulated annealing optimization is sensitive to its random initialization, so it was evaluated by running the same experiment 10 times with different starting weights. the values reported in this paper are the average upon all experiments. ## results the results for every experiment executed are shown in as can be seen, the random forest algorithm using the machine learning method without levi graphs, proposed in this work, had the best results among the manual annotated comments, resulting in a smatch improvement of 15% from the best literature method @xcite . meanwhile, the integer linear programming based ones @xcite were the best for the parsed sentences, indicating that their method is more robust with regards to the quality of the input graphs. the quality of the graphs is, however, still an important factor to have satisfactory summaries, since the usage of parsed amr representations caused a drop on the metrics for each approach, even among the supervised ones, which were trained over an artificial parsed corpus. it is also important to notice that the usage of levi graphs to select important relations did not actually improve the results, suggesting that the set of rules for edge selection and path expansion, originally developed by @xcite , are also suitable for a multi-document purpose. comparing our rule-based method with the original technique of @xcite , the adoption of the tf counts over tf-idf decreased the quality of summaries, refuting our initial hypothesis that the idf term would harm the content selection. regarding the score optimization method, it did not vary much from its original counterpart -the one from @xcite -indicating that the scoring did not improve from one to the other. 11 using ramp loss. for training: η = 1.0, = 1.0, as used by the original authors. we do, however, highlight that this does mean that the attributes set is not fit to the problem, since it worked well with the random forest algorithm. thus, the research of other types of weight optimization techniques, such as genetic algorithms as used in the work by @xcite , may lead to better results. it is also expected that the inclusion of other linguistic features may improve the outcome @xcite . from the optimized weights, however, we can see how important each of the developed features are. this is accomplished by calculating the correlation of the absolute value for the weights with the evaluation metrics used. this result is shown in figure 2 , in which the more intense the green color is, the more important its corresponding feature is. as can be seen in the heatmap, the most important features to be noticed are the average neighbor degree, the eigenvector centrality and the depth, with some attention to the clustering coefficient feature. on the other hand, the most detrimental features are those which are assigned to deeper shades of pink. there are not much features which can be considered effectively harmful to the quality of the summaries -i.e. there are no features which have such a high negative correlation to the evaluation metrics -but it can be observed that both the tf-idf and the hubs features do have a stronger negative correlation. figure 2 : correlation heatmap of the optimized weights including the evaluation metrics may not have had an impact over the output quality. the sentiment analysis features (sentiment and aspect, commonly used in opinion summarization approaches) do not appear to have a role of importance during the summarization process, even though we are dealing with opinions. the feature analysis also highlights our previous observation that the feature set comprises other types of important information. although there was not much difference (using smatch or sema) between the score optimization method and the original one from @xcite , the results were obtained taking into account different features (e.g., eigenvector centrality vs. tf-idf). this indicates that there is indeed room for improvement by combining these forms of knowledge. ## conclusion the incorporation of semantic knowledge via explicit representations into the summarization process is a fruitful field of research. in this context, this paper focused on the development of methods for opinion summarization that took advantage of the semantics encoded within amr graphs. the experiments indicate that our machine learning approach, using the random forest algorithm, produced better summary graphs compared to other amr-based summarization techniques. we also confirm the findings of @xcite , stating that the quality of the input graphs used is important to ensure the creation of good summaries. an analysis of the feature set also sug-gests that using sentiment and aspect information does not actually improve the results. as future work, we recommend the further development of the feature set, since it has been shown to be effective for a machine learning approach, including mainly new linguistic features, as originally proposed by @xcite . we also advocate that exploring other optimization techniques can be an important path of investigation, such as the genetic algorithm approach used by @xcite . there is also a significant improvement to be done during the merging of sentence graphs into a single document representation. currently, only the concept labels are taken into account, however the usage of coreference resolution can be a valuable enhancement towards the creation of a consistent multi-sentence representation @xcite @xcite .