# A Focused Event Crawler with Temporal Intent
## 1 Introduction  
With the availability of ever-increasing World Wide Web resources, there is a strong need to improve the collection of web data on events to support analysis of similar nature or man-made disasters [1; 2], emergency risk assessment [3], terrorist attacks comparisons [4], and other event responses in the future. For example, it is possible to compare events or analyse trends using the information on similar events that have occurred in previous years. In general, events have an explicit or implicit temporal intent that includes when the event starts, when the event ends, and the temporal distribution of the event [5]. A simple example of this is the event \"Typhoon Haiyan\" with a temporal intent between the 3 and the 12 November 2013. This means that more web data about this event was published during this period. Therefore, temporal intent should be taken into account in the collection and archiving of web data on key events. Currently, there are two ways to automatically collect and archive web data on key events: general-purpose crawler-based methods and focused crawler-based methods [6; 7]. The general-purpose crawler-based methods utilize general-purpose search engines (e.g., Google and Baidu), their application programming interfaces (APIs), or universal crawlers with pre-defined queries to crawl the Internet for web data collection and archiving [8]. However, the results of these methods usually contain enormous duplicates or unrelated information [9; 10], which can make the collection and archiving process tedious and time-consuming. For example, a search on Google with the pre-defined query \"Typhoon Haiyan\" on 3 April 2017 yields 433,000 results, which is reduced to 45,500 results when restricted tothe time period between 3 and 12 November 2013. Although the start and end times have already reduced the search results, there is still a huge amount of duplicate and irrelevant information left. This is because there is a lot of web data on other topics containing one or more keywords in the query. Therefore, a focused crawler, which intends to automatically collect as many web pages as possible that are relevant to the given topic and to keep the amount of irrelevant web pages collected to a minimum [11; 12], has already been adapted to collect web data about key events [13]. These traditional researches focus on how to use web page content or hyperlink relationships to collect relevant web pages, or in what order to collect them. However, little attention is paid to the temporal intent of the event, which also plays an important role in the crawling process. For example, the start time can be used to filter out irrelevant web pages, and the temporal distribution of an event can have an impact on the priority of crawling web pages (a mathematical proof is described in Section 3.1). Thus, how to use temporal intent in a focused crawler to collect event information more efficiently remains an open problem. In order to solve the above mentioned problem, this paper tries to understand the temporal intent of events and proposes a novel focused event crawler with temporal intent for automatically collecting web data about events. In contrast to previous approaches that only take into account factors such as event keywords, web page content, and hyperlink information, our approach additionally takes into account the temporal distribution of events while maintaining the above factors. This can improve the precision of the crawler by reducing the influence of irrelevant information. Our main contributions can be summed up as follows. * We propose a new automated method for detecting temporal intent on events. This method uses Google Trends data to automatically and quantitatively estimate the start time and the temporal distribution of events, in contrast to previous manual judgements based on expert experience. * We propose a new focused crawling framework that incorporates the temporal intent of events.