## 4 Results and Discussion  
As mentioned earlier, this study has three distinct goals: to find a better Python package for extracting text from PDF files, to find a better NLP framework for sentence boundary detection, and to evaluate the performance of PDF extraction packages in terms of memory footprint and runtime. For this experiment, a dataset consisting of 10 scientific articles on the topic of Electric Double Layer Capacitors from different journal publishers is used. All the articles are downloaded in PDF format and converted into text using various popular Python PDF packages, namely Pypdf2, Pymupdf, Pdfminer.six, Pdfttotext, Tika, and Grobid. Then, sentence boundary identification is applied to the extracted PDF documents using three different NLP packages, namely NLTK, Spacy, and Gensim. Then, the correctly identified sentences are matched with the ground truth file using the difflib package. For the sentence boundary identification experiment, precision, recall and f-1 values are used as performance measures. The precision, recall and f-1 values are calculated using Equations (2)-(4), respectively. \[Precision=\frac{Sentence_{correct}}{Sentence_{extract}} \tag{2}\]  
\[Recall=\frac{Sentence_{correct}}{Sentence_{standard}} \tag{3}\]  
\[F1=2*\frac{Precision*Recall}{Precision+Recall} \tag{4}\]  
Here, \(Sentence_{correct}\) stands for the correctly identified sentences from the extracted sentences extracted from the documents. On the other hand, \(Sentence_{extract}\) stands for the total extracted sentences from the documents. \(Sentence_{standard}\) stands for the total number of sentences in the ground truth file. Table 3 represents the comparison results of precision, recall, and f-1 values obtained in the experiment. Since three NLP frameworks are used to identify the sentence boundary, the experiment shows that the Spacy NLP framework always has higher precision and a higher f-1 score for each PDF-to-text extraction package than the other two frameworks, namely NLTK and Gensim. The PDF-to-text extraction package that achieved the highest precision and f-1 score is Grobid, which achieves 91% and 93% with the Spacy framework, respectively, while the NLTK and Gensim frameworks achieve 88% and 86% precision scores and 90% and 89% f-1 scores, respectively. This experimental result shows that the Grobid package performs better with Spacy in identifying complete sentences from the extracted plain text. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Package**} & \multicolumn{2}{c}{**Precision**} & \multicolumn{3}{c}{**Recall**} & \multicolumn{3}{c}{**F-1**} \\ \cline{2-10}  & **NLTK** & **Spacy** & **Gensim** & **NLTK** & **Spacy** & **Gensim** & **NLTK** & **Spacy** & **Gensim** \\ \hline Pypdf2 & 0.61 & 0.86 & 0.23 & 0.61 & 0.86 & 1.00 & 0.61 & 0.86 & 0.37 \\ Pymupdf & 0.63 & 0.70 & 0.18 & 1.00 & 1.00 & 1.00 & 0.75 & 0.82 & 0.30 \\ Pdfminer.six & 0.63 & 0.70 & 0.19 & 1.00 & 1.00 & 1.00 & 0.75 & 0.82 & 0.31 \\ Pdftotext & 0.62 & 0.69 & 0.27 & 1.00 & 1.00 & 1.00 & 0.74 & 0.81 & 0.42 \\ Tika & 0.63 & 0.70 & 0.20 & 1.00 & 1.00 & 1.00 & 0.75 & 0.82 & 0.75 \\ Grobid & 0.88 & 0.91 & 0.86 & 0.93 & 0.96 & 0.94 & 0.90 & 0.93 & 0.89 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison score of precision, recall, and f-1 scores of different PDF-to-text extraction Python packages for different NLP frameworks. Figure 2: Precision, recall, and F-1 scores of different PDF-to-text extraction Python packages for different NLP frameworks. The precision and f-1 values of Grobid are also higher than the other five packages in all NLP frameworks. Table 4 shows the match ratio results for all documents for all PDF-to-text conversion tools. The match ratio value is calculated using the Equation (1). The match ratio is calculated by comparing the documents converted by all the tools with the base documents, one by one. From the results, it can be seen that the Grobid tool has achieved the best average match ratio, 81%, when extracting text from PDF documents.