# A Focused Event Crawler with Temporal Intent
## 5 Experiments and Discussion
### Experiment 2: Effectiveness of Our Focused Crawler  
This experiment uses the mainstream best-first crawling strategy-based focused crawler as a benchmark for comparison. The best-first crawling strategy mainly uses the cosine similarity between topic keywords and web content as the similarity calculation and assigns URL priority using web content similarity and anchor text similarity. Our temporal intent focused crawler is mainly based on the formula in Section 4. In the experiments, the two methods use the same initial URL, topical keywords, web page request method, and web page parsing method, while the differences are in the topic representation method, topic relevance calculation strategy, and URL priority assignment method. Finally, 2500 web pages were crawled by each of the two methods to evaluate the crawling performance. The experimental results for the two crawling methods are shown in Figure 4. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Event Name**} & **Time Interval When** & **Time When the Initial** & & \\  & **the Initial Search** & **Search Volume Index** & **Detected Start Time** & **Actual Start Time** \\  & **Volume Index Is Zero** & **Is Greater than Zero** & & \\ \hline WenChuan earthquake & [2004-01, 2008-04] & 2008-05 & 2008-05 & 2008-05-21 \\ YuShu earthquake & [2004-01, 2010-03] & 2010-04 & 2010-04 & 2010-04-14 \\ Typhoon HaiYan & [2004-01, 2013-10] & 2013-11 & 2013-11 & 2013-11-03 \\ Haiiti earthquake & [2004-01, 2009-12] & 2010-01 & 2010-01 & 2010-01-12 \\ Indian Ocean Tsunami & [2004-01, 2004-11] & 2004-12 & 2004-12 & 2004-12-26 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparisons between the detected and actual start time. Figure 3: Examples of Google Trends. Figure 4a depicts the cumulative number of relevant web pages. In other words, for any number \(n\) of crawled web pages, the total number of the crawl up to that point is reported. From Figure 4a, it can be seen that in the initial stage of crawling (the first 600 web pages), the number of relevant web pages crawled by the two methods is almost the same. This is mainly because they use the same initial URL, and the web pages obtained in the initial stage are chained from the initial URL. The number of relevant pages crawled by our method was significantly higher than that crawled by the traditional best-first method after crawling 600 pages. For example, our focused crawler obtains 1138 relevant web pages while the traditional one obtains 930 relevant web pages when the two crawlers both collected 1400 pages. Therefore, it is intuitive that the crawling effect of the focused crawler in this paper is superior to that of the traditional method. In order to perform a quantitative comparative analysis, the precision metric is computed according to Equation (17). Figure 4b depicts the cumulative precision of the two crawlers. As can be seen in Figure 4b, the difference in precision between the two methods is small for the first 600 crawled web pages, with a difference of 0.1% in their cumulative average precision. After that, the precisions of our method are consistently higher than those of the traditional best-first method, with an average improvement of 10.28%. The greatest improvement in precision is achieved when a total of 974 web pages are crawled, with an increase of 25.21%. The results indicate that the ability of our method to find pages related to a topic event is better than the traditional method. There are two main reasons why our method is able to achieve such excellent results. First, our method can identify the temporal intent of the event (i.e., the start time and the quantified temporal distribution) and can filter out pages published before the event using a relevance calculation strategy based on the start time. More importantly, a new URL priority assignment method is designed based on the quantified temporal distribution, which can prioritize the crawling of pages published at the time of the event.