# A Weighted PageRank-Based Bug Report Summarization Method Using Bug Report Relationships
## 7 Limitations and Improvements  
### Limitations  
The first limitation of this paper is the incompleteness of sentence separation in the text preprocessing phase. Bug reports, unlike general documents, do not consist of clearly distinguishable sentences. In some cases, bug reports simply list keywords or have a mixture of non-natural code and information. As mentioned in Section 3.3, we used NLTK, a Python-based natural language processing library. Additionally, we separated the sentences by defining special rules for bug reports. However, we observed that the separation was not complete. To solve this problem, we searched manually for sentences that were separated and then manually corrected the obvious statement separation errors. To completely remove such sentence separation errors, a sentence separation method for bug reports was required. If sentences are clearly separated, the accuracy of selecting right sentences for summarization would be improved.  
The second limitation is that we created our own experimental data and the golden summaries were constructed by the first author of this paper alone. To mitigate the risk arising when we create our own experimental data, we checked the 36 bug reports in the previous data, found 7 bug reports that have dependency relationships, and ran the same experiment as Experiment 1-1 on the 7 bug reports, as described in Section 6.1. The experiment showed that, while PRST yielded 50.4% precision value, RBRS yielded 55.9%. The 5.5% improvement of precision is similar to the 6.41% improvement of precision in Experiment 1-1 (cf. Section 5.1.1). Also, to mitigate the risk arising when only one participant creates the experimental data, we conducted another experiment, where we recruited three participants to perform annotation tasks and create a golden summary, as described in Section 6.2. This additional experiment showed 7.33% improvement of precision, which is also an improvement rate similar to the precision improvement of 6.41% obtained in Experiment 1-1.  
The third limitation is the improvement rate of our proposed method RBRS in comparison with the previous method PRST. The previous method showed 9.47% improvement in precision, 7.87% improvement in recall, and 8.92% improvement in F-score. On the other hand, when duplicates relationships in a baseline method are included, our method shows 1.98% improvement in precision, 2.05% improvement in recall, and 2.02% improvement in F-score. When duplicates relationships in a  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Technique** & **Precision** & **Recall** & **F1-Score** \\ \hline PRST & 42.40\% & 40.09\% & 41.21\% \\ RBRS & 49.73\% & 47.50\% & 48.60\% \\ \hline \hline \end{tabular}
\end{table}
Table 8: Additional results using the golden summary created by three participants.  
baseline method are excluded, our method shows 6.41% improvement in precision, 6.14% improvement in recall, and 6.28% improvement in F-score. While our experiment shows that our proposed method can significantly improve the accuracy of summarizing bug reports in the absence of the duplicates relationships, we also should seek a method to improve the summarization accuracy significantly in the presence of the duplicates relationships. One way to improve it would be the adoption of a deep learning technique for textual summarization.