## 4 Implementation and Experiments
### Results and Evaluation
#### 4.2.1 Qualitative Results
```  
Listing 1: SPARQL query to retrieve the number of documents containing the word \"algol\" from the KRS. The evaluation was conducted by four human raters and performed over a stratified random sample of the crawled web pages. This kind of sample was selected to maintain consistency in the results since a human rater evaluates the results from each topic. The first step of the stratified random sample consists of calculating the sample size from the whole data (see Equation (7)):  
\[n=\frac{N\sigma^{2}Z^{2}}{(N-1)\epsilon^{2}+\sigma^{2}Z^{2}} \tag{7}\] \begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Topic**} & **Arithmetic** & **Standard** & **Threshold** \\  & **Mean (\(\mu\))** & **Deviation (\(\sigma\))** & **Range (\(\mu\pm\sigma\))** \\ \hline Computer science & 5.01 & 1.56 & [3.45,6.57] \\ Diabetes & 3.19 & 1.54 & [1.65,4.73] \\ Politics & 4.84 & 1.54 & [3.30,6.38] \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Threshold range by topic. where \(n\) is the sample size, \(N\) is the size of the corpus, \(\sigma\) is the standard deviation, \(Z\) is the confidence value, and \(\epsilon\) is the sample error rate. The second step consists of calculating the sample size for the accepted and rejected web pages (see Equation (8)):  
\[n_{i}=n*\frac{N_{i}}{N} \tag{8}\]  
where \(n_{i}\) corresponds to the sample size of accepted or rejected web pages, \(N_{i}\) is the total web pages for accepted or rejected, and \(N\) is the size of the corpus. Table 9 shows the sample values for accepted and rejected web pages for Google (G) and Wikipedia (W). The sample, for Google and Wikipedia, was randomly selected. Tables 10 and 11 show the HR results for each rater and the summary per topic for the seed URLs from Google and Wikipedia, respectively. According to the results from the Tables 10 and 11, the proposed SFWC was consistent with the results for the seed URLs from Google and Wikipedia. These results demonstrate that the KRS and the similarity measure selects the most relevant concepts for each topic. The KRS describes the nouns from the input corpus and the arithmetic mean and the standard deviation establish a threshold to determine which nouns are the most representative for the topic. The similarity measure defines if web page content is related to the given topic or not if the result is between the predefined threshold. The combination of KRS and the similarity measure help to select the most related web pages. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **R1** & **R2** & **R3** & **R4** & **Average** \\ \hline \multirow{2}{*}{Computer science} & 70/93 & 74/93 & 65/93 & 65/93 & 68.5/93 \\  & (75.27\%) & (79.57\%) & (69.89\%) & (69.89\%) & (73.66\%) \\ \hline \multirow{2}{*}{Diabetes} & 26/29 & 23/29 & 26/29 & 28/29 & 25.75/29 \\  & (89.66\%) & (79.31\%) & (89.66\%) & (96.55\%) & (88.79\%) \\ \hline \multirow{2}{*}{Politics} & 110/151 & 106/151 & 93/151 & 114/151 & 105.75/151 \\  & (72.85\%) & (70.20\%) & (61.59\%) & (75.50\%) & (70.03\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Harvest rate values for Google seed URLs per rater. R1 is for rater 1, R2 is for rater 2, etc. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Topic**} & \multicolumn{2}{c}{**Total**} & \multicolumn{2}{c}{**Sample Size**} & \multicolumn{2}{c}{**Sample Size**} & \multicolumn{2}{c}{**Sample Size**} \\  & \multicolumn{2}{c}{**Examples**} & \multicolumn{2}{c}{**(N)**} & \multicolumn{2}{c}{**for Accepted**} & \multicolumn{2}{c}{**for Rejected**} \\  & \multicolumn{2}{c}{**G**} & \multicolumn{2}{c}{**W**} & \multicolumn{2}{c}{**G**} & \multicolumn{2}{c}{**W**} & \multicolumn{2}{c}{**G**} & \multicolumn{2}{c}{**W**} \\ Computer science & 879 & 2589 & 303 & 335 & 93 & 142 & 210 & 192 \\ Diabetes & 957 & 3083 & 284 & 342 & 29 & 157 & 255 & 185 \\ Politics & 1893 & 3878 & 374 & 350 & 151 & 251 & 223 & 99 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Sample size calculation for accepted and rejected web pages.