## 4 Implementation and Experiments
### Results and Evaluation
#### 4.2.1 Qualitative Results
The _domain names crawled_ column defines the number of different domain names crawled, e.g., the diabetes topic contains the lowest number of seed URLs crawled, but it is the second topic with the highest number of domain names crawled, which means that the seed URLs in the diabetes topic are connected to many other domains. The _Web pages analyzed_ column defines the total number of web pages analyzed by the proposed SFWC. The columns _Accepted_, _Rejected_, and _Error_ distribute the number of the web pages crawled into those whose content is related with the corresponding topic, not related with the corresponding topic and the web pages that could not be processed due to an error (e.g., HTTP 400 error) because the URL was an image (e.g., [[http://example.com/image.jpg](http://example.com/image.jpg)]([http://example.com/image.jpg](http://example.com/image.jpg))) or a PDF file. In accordance with the results, the crawled Wikipedia's seed URLs obtained the highest number of web pages accepted (the web page content is related with the corresponding topic), but it also contains the highest number of errors, produced because the URLs contain an image (jpg files) instead of text. In accordance with the results from Tables 6 and 7, seed URLs from Wikipedia obtained the best results in comparison with those obtained with the seed URLs from Google. Seed URLs from Google contain information from several domain names. The seed URLs from Wikipedia contain almost only URLs to other Wikipedia pages. Additionally, the seed URLs from Wikipedia have the same web page structure and format. The seed URLs from Google do not share the same structure, and the content could be in different formats. In this sense, URLs crawled from Google were less accepted because the \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Topic**} & **Seed URLs** & **Seed URLs** & **Seed URLs** & **Seed URLs** \\  & **(Google)** & **Crawled** & **Not Crawled** & **Not Processed** \\ \hline \multirow{2}{*}{Computer science} & \multirow{2}{*}{50} & 11 & 32 & 7 \\  & & (22\%) & (64\%) & (14\%) \\ \hline \multirow{2}{*}{Diabetes} & \multirow{2}{*}{50} & 9 & 30 & 11 \\  & & (18\%) & (60\%) & (22\%) \\ \hline \multirow{2}{*}{Politics} & \multirow{2}{*}{50} & 22 & 18 & 10 \\  & & (44\%) & (36\%) & (20\%) \\ \hline \multirow{2}{*}{**Total**} & \multirow{2}{*}{**150**} & **42** & **80** & **28** \\  & & **(28\%)** & **(53.33\%)** & **(18.67\%)** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Google page results statistics. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Topic**} & **Seed URLs** & **Seed URLs** & **Seed URLs** & **Seed URLs** \\  & **(Wikipedia)** & **Crawled** & **Not Crawled** & **Not Processed** \\ \hline \multirow{2}{*}{Computer science} & \multirow{2}{*}{50} & 26 & 24 & 0 \\  & & (52\%) & (48\%) & (0\%) \\ \hline \multirow{2}{*}{Diabetes} & \multirow{2}{*}{50} & 31 & 19 & 0 \\  & & (62\%) & (38\%) & (0\%) \\ \hline \multirow{2}{*}{Politics} & \multirow{2}{*}{50} & 39 & 10 & 1 \\  & & (78\%) & (22\%) & (2\%) \\ \hline \multirow{2}{*}{**Total**} & \multirow{2}{*}{**150**} & **96** & **53** & **1** \\  & & **(64\%)** & **(36\%)** & **(0.6\%)** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Corpus Wikipedia pages statistics. domain names are heterogeneous, and the content could drastically change in format and structure from one URL to another. Evaluation  
The evaluation is based on the Harvest Ratio [31; 32; 41] (HR) measure shown in Equation (5). According to [PERSON] and [PERSON] [42], the HR is the primary metric to evaluate a crawler performance. The HR measure the rate at which relevant web pages are acquired and irrelevant web pages are filtered off from the crawling process:  
\[\textit{Harvest Ratio}=\frac{||R_{p}||}{||T_{p}||} \tag{5}\]  
where \(R_{p}\) corresponds to those web pages accepted by the system and evaluated as correct and \(T_{p}\) corresponds to the total accepted web pages downloaded by the SFWC, evaluated as correct or incorrect.