Question: How is the filtering of irrelevant text arranged in Lv.2 preprocessing？
Evidence: The SemEval-2010 benchmark dataset BIBREF0 is composed of 244 scientific articles collected from the ACM Digital Library (conference and workshop papers). The input papers ranged from 6 to 8 pages and were converted from PDF format to plain text using an off-the-shelf tool. The only preprocessing applied is a systematic dehyphenation at line breaks and removal of author-assigned keyphrases. Scientific articles were selected from four different research areas as defined in the ACM classification, and were equally distributed into training (144 articles) and test (100 articles) sets. Gold standard keyphrases are composed of both author-assigned keyphrases collected from the original PDF files and reader-assigned keyphrases provided by student annotators.
Long documents such as those in the SemEval-2010 benchmark dataset are notoriously difficult to handle due to the large number of keyphrase candidates (i.e. phrases that are eligible to be keyphrases) that the systems have to cope with BIBREF6 . Furthermore, noisy textual content, whether due to format conversion errors or to unusable elements (e.g. equations), yield many spurious keyphrase candidates that negatively affect keyphrase extraction performance. This is particularly true for systems that make use of core NLP tools to select candidates, that in turn exhibit poor performance on degraded text. Filtering out irrelevant text is therefore needed for addressing these issues.
In this study, we concentrate our effort on re-assessing keyphrase extraction performance on three increasingly sophisticated levels of document preprocessing described below.
Table shows the average number of sentences and words along with the maximum possible recall for each level of preprocessing. The maximum recall is obtained by computing the fraction of the reference keyphrases that occur in the documents. We observe that the level 2 preprocessing succeeds in eliminating irrelevant text by significantly reducing the number of words (-19%) while maintaining a high maximum recall (-2%). Level 3 preprocessing drastically reduce the number of words to less than a quarter of the original amount while interestingly still preserving high recall.
Level 2: Similarly to (Nguyen and Luong, 2010; Lopez and Romary, 2010), we retrieve6 the original PDF files from the ACM Digital Library. We then extract the enriched7 textual content of the PDF files using an Optical Character Recognition (OCR) system8, and perform document logical structure detection using ParsCit (Kan et al., 2010)9. We use the detected logical structure to remove author- assigned keyphrases and select only relevant elements: title, headers, abstract, introduction, related work, body text10 and conclusion. We finally apply a systematic dehyphenation at line breaks and run the Stanford CoreNLP suite.
Answer: They perform document logical structure detection using ParsCit (Kan et al., 2010). They use the detected logical structure to remove author- assigned keyphrases and select only relevant elements: title, headers, abstract, introduction, related work, body text and conclusion. They finally apply a systematic dehyphenation at line breaks and run the Stanford CoreNLP suite.