Abstract:
Information is generated indicative of frequency of search terms presented to at least one online search service. As event indications, indicative of user interaction generally with front end servers, are being provided for persistent storage, ones of the event indications that are indicative of search events are detected. The detected ones of the search event indications are processed and it is determined, based at least in part thereon, by location, frequency data indicative of a frequency of each of a plurality of search terms presented to the at least one online search service. An indication of at least some of the frequency data is caused to be associated with indications of locations to which the frequency data corresponds. For example, the frequency data may be displayed superimposed on a map.

Description:
BACKGROUND 
   It is known to process a search index, of an online search service, to determine a level of popularity of query terms made in a given time period, such as during a given day. Conventionally, such popularity determinations are made based on transaction data in a data warehouse, and it can take up to two to three days for the search query data in the data warehouse to be processed and the search index (with information about the popularity level of query terms) to be built and cleansed. Furthermore, the resolution of the popularity determinations (for example, temporally and geographically) may be limited. 
   SUMMARY 
   In accordance with an aspect of the invention, information is generated indicative of frequency of search terms presented to at least one online search service. As event indications, indicative of user interaction generally with front end servers, are being provided for persistent storage, event indications that are indicative of search events are detected. The detected search event indications are processed and it is determined, based at least in part thereon, by event origin location, frequency data indicative of a frequency of each of a plurality of search terms presented to at least one online search service. An indication of at least some of the frequency data is caused to be associated with indications of event origin locations to which the frequency data corresponds. For example, the frequency data may be displayed superimposed on a map including event origin locations. 
   Processing the detected ones of the search event indications and determining based at least in part thereon, by event origin location, frequency data indicative of a frequency of each of a plurality of search terms presented to the at least one online search service may include updating the determination of frequency data based on newly processed detected search event indications. Updating the determinations of frequency data based on newly processed detected search event indications may, for example, include applying an algorithm that automatically decays frequency values over time. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1 , which does not illustrate the present invention, illustrates an architecture of a system in which front end web servers, including front end web servers handling search events, are producing event data based on incoming user requests, including search queries each comprising at least one search term. 
       FIG. 2  schematically illustrates event data provided to the data collectors are further provided to a data warehouse via a “data highway,” and a filter selects search event data. 
       FIG. 3  illustrates an example architecture of the statistics generation block of  FIG. 2 . 
       FIG. 4  illustrates schematically how various arrays are “dumped” to a data structure accessible for display. 
       FIG. 5  illustrates an example of a display that may be generated based on generated statistics. 
       FIG. 6  is a flowchart of a method in accordance with an embodiment of the invention, to cause determination and display of the level of popularity of query terms provided to one or more search services. 
   

   DETAILED DESCRIPTION 
   The inventors have realized that it can be useful for the level of popularity of query terms to be determined relatively quickly. For example, the determination may be useful in the process of marketing the keywords for advertisement targeting and/or other monetization. For example, it may be possible to target a “fresh” keyword or keywords before the market for that keyword is saturated. The inventors have additionally realized that, if the popularity determination can be resolved geographically and/or temporally, this may also provide advantages in marketing a keyword or keywords. 
   In accordance with an aspect, a method/system is described to determine high-frequency keywords in a search-stream, with a low latency. Additionally, the stream may be partitioned geographically (e.g., by city or other event origin location) and/or temporally (e.g., by hour or by time of day such as morning, afternoon, evening, etc.), allowing a user of the determinations to discern geographic and temporal differences in monetizing specific keywords. Search-events from front-end web-servers handling the search queries are processed in real-time. The latency of the frequency determinations may be less than one hour, or even under five minutes. Indications of the high-frequency keywords may be displayed, for example, relative to a map to allow for easy navigation. 
     FIG. 1 , which does not illustrate the present invention, illustrates an architecture of a system in which front end web servers FEa  102   a , FEb  102   b , FEc  102   c , . . . , FEx  102   x , including front end web servers handling search events, are producing event data  105  based on incoming user requests  103 , including search queries each comprising at least one search term. The event data  105  is provided to data collectors DC 1   108 ( 1 ) and DC 2   108 ( 2 ) via paths Pa  106   a , Pb  106   b , Pc  106   c  and Pd  106   d . In general, there may be numerous front end web servers, data collectors and paths; a small number are shown in  FIG. 1  and throughout this patent description for simplicity of illustration. The particular paths may be determined according to a path configuration  104 , for example, as described in U.S. patent application Ser. No. 11/734,067, filed on Apr. 11, 2007. U.S. patent application Ser. No. 11/734,067 is incorporated by reference at least for its disclosure of methods to determine path configurations. 
   The data collectors may be, for example, computers or computer systems in one or more data centers. A data center is a collection of machines (data collector machines) that are co-located (i.e., physically proximally-located). The data centers may be geographically dispersed to, for example, minimize latency of data communication between front end web servers and the data collectors. Within a data center, the network connection between machines is typically fast and reliable, as these connections are maintained within the facility itself. Communication between front end web servers and data centers, and among data centers, is typically over public or quasi-public networks (i.e., the internet). 
   Referring now to  FIG. 2 , the event data provided to the data collectors DC 1   108 ( 1 ) and DC 2   108 ( 2 ) via paths Pa  106   a , Pb  106   b , Pc  106   c  and Pd  106   d  are further provided to a data warehouse  202  via what may be thought of as a “data highway”  204 . A filter  206  selects events that are from front end web servers that are search web servers. 
   For example, every event may be indicated by an event record that includes fields whose contents characterize the event. For example, an event record may include a field whose contents identify a “host name” or “space id” corresponding to a front end server that generated the event. A “space id” is a unique key that identifies the page contents and category. The contents of these event record fields may thus be highly indicative of whether the event is a search event. Other fields in an event record may include a time stamp, the URL accessed by a user, and an IP address via which a computing device accesses a search service (e.g., a computing device that executes a browser to access the search service). Other fields may be dependent on the type of event such as, for example, a field for a query term if the event is a search event. 
   We now discuss the data highway  204  in greater detail. The data highway  204  is a data pipeline via which a stream of events generated by data producers are provided to data consumers. The pipeline has been denoted as a “highway,” because there are multiple off ramps but, in one example, the pipeline is unidirectional. The highway may comprise multiple stages. Each stage forwards the events toward the data consumers, which in turn can be another stage, so the stages may be chained together. Additionally, each stage may include at least one filter to filter an event based on characteristics of the event. For instance, an event filter may be configured to “only give events that have been generated by data producer Y” or “only give events that occurred between 12:30 and 1:00 PM.” Event filters may be combined together using Boolean logic (e.g., give events that come from data producer Y that occurred between 12:30 and 1:00 PM). In general, events that are filtered may also continue along their original path to the data warehouse  202 . 
   For events determined by the filter  206  (e.g., from the host name or via some other means) to be a search event or highly likely to be a search event, statistics are generated ( 208 ) in real time on the basis of some characteristic evidenced by the events. More particularly, statistics are generated to correlate events having particular characteristics with the search query terms presented to the search service. An indication of the generated statistics is caused to be displayed  210 . 
   In the  FIG. 2  example, the statistics are generated based on geographic locations associated with the origins of the events. That is, statistics are generated to correlate events of originating from particular geographic locations or regions with the search query terms of those events. The geographic locations may be determined, for example, by correlating the IP address via which a computing device accesses the search service, as indicated in an event record, to a known or discerned location associated with that IP address. As another example, a geographic location associated with an event origination may be determined based on one or more cookies in the event record for that event. For example, for an event corresponding to a logged-in Yahoo! user, the geographic location for that event may be determined based on known demographic information maintained by Yahoo! and corresponding to that logged-in Yahoo! user. 
     FIG. 3  illustrates an example architecture of the statistics generation block  208  of  FIG. 2 . In block  302 , the search events are partitioned by geographic location or other characteristic indicated in the search event and which is to be correlated to the search query terms to generate the statistics. Thus, for example, if the statistics are to be generated to correlate particular geographic locations to the search query terms presented to the search service from those geographic locations, then the events are partitioned by geographic location. (For example, a geolocation targeting call, to a specialized geolocation program, may be employed to determine the latitude &amp; longitude associated with the search event based on the IP address associated with the search event.) Thus, for example, the processing of events may be partitioned into distinct disjoint sets to distribute the statistics generation processing. Partitioning based on geography can be useful given that, in general, events from multiple geographic locations are not processed to determine the most popular search terms in any one particular geographic location. The partitioning facilitates scaling of the statistic generation processing for greater numbers of events and/or for additional geographic locations or other event characteristics. 
   Partitions  304   a ,  304   b ,  304   c , . . . ,  304   x  generate statistics regarding frequency of search terms correlated to search event characteristics such as geographic location. It is noted that while, in some examples, each partition may handle statistics generation for one or more search event characteristic values (and/or aggregation of search event characteristic values), in general, statistics generation for a particular one or more search event characteristics values (or aggregation of search event characteristic values) is not distributed across multiple partitions. Thus, for example, partition  304   a  may generate statistics  306   a   1 ,  306   a   2  and  306   a   3  for three respective search event characteristic values (and/or aggregation of search event characteristic values). Similarly, partition  304   b  may generate statistics  306   b   1  and  306   b   2  for two respective search event characteristic values (and/or aggregation of search event characteristic values); partition  304   c  may generate statistics  306   c   1  and  306   c   2  for two respective search event characteristic values (and/or aggregation of search event characteristic values); and partition  304   x  may generate statistics  306   x  for one search event characteristic value (and/or aggregation of search event characteristic values). 
   The statistics  306   a   1 ,  306   a   2  and  306   a   3  (generated by partition  304   a );  306   b   1  and  306   b   2  (generated by partition  304   b );  306   c   1  and  306   c   2  (generated by partition  304   c ) and  306   x   1  (generated by partition  304   x ) are generated and maintained locally by each respective partition  304   a ,  304   b ,  304   c , . . . ,  304   x . The generated statistics are occasionally (e.g., periodically) “dumped” to a location  308  accessible for display  210 . 
   In accordance with one example, the statistics generation employs a traffic analysis algorithm that generates, on a rolling basis, an indication of the most popular search query terms based on the events presented to it. The traffic analysis algorithm may be configured to account not only for frequency, but also for temporality, in the statistics generation. 
   As a specific example, the algorithm may be such that frequency values for search query terms automatically decay over time. A simplistic example of one particular implementation is now described. In the example, an array is used by each partition  304  to accumulate statistics regarding frequency of search terms represented by events presented to that partition. In particular, each item of the array indicates one search query term and a frequency associated with that search term. In the example, it is assumed that the array is of size three and, initially, the items of the array are empty. If the following search term data is sent into the partition: 
   {“a”, “b”, “c”, “a”, “a”, “d”, “d”, “b”} 
   the first three letters will be added (step 1) since, as each of the first three letters is processed, the array is not full yet. The column “current state of array” includes each keyword and the current calculated frequency for that keyword. The star is the moving pointer to a ‘current’ index in the array). After step 1, the “current” index is to the first item in the array, containing “a.” 
                                       Step   Data already processed   current state of array   data to be processed                   1   {‘a’, ‘b’, ‘c’}   {*‘a’-&gt;1, ‘b’-&gt;1, ‘c’-&gt;1}   {‘a’, ‘a’, ‘d’, ‘d’, ‘b’}       2   {‘a’, ‘b’, ‘c’, ‘a’}   {*‘a’-&gt;2, ‘b’-&gt;1, ‘c’-&gt;1}   {‘a’, ‘d’, ‘d’, ‘b’}       3   {‘a’, ‘b’, ‘c’, ‘a’, ‘a’}   {*‘a’-&gt;3, ‘b’-&gt;1, ‘c’-&gt;1}   {‘d’, ‘d’, ‘b’}       4   {‘a’, ‘b’, ‘c’, ‘a’, ‘a’, ‘d’}   {‘a’-&gt;2, *‘b’-&gt;1, ‘c’-&gt;1}   {‘d’, ‘b’}       5   {‘a’, ‘b’, ‘c’, ‘a’, ‘a’, ‘d’, ‘d’}   {‘a’-&gt;2, ‘d’-&gt;1, *‘c’-&gt;1}   {‘b’}       6   {‘a’, ‘b’, ‘c’, ‘a’, ‘a’, ‘d’, ‘d’, ‘b’}   {*‘a’-&gt;2, ‘d’-&gt;1, ‘b’-&gt;1}   { }                    
At step 2, the second “a” has been processed. Since the “current” index is already pointing to an item in the array that contains “a,” the frequency for “a” is incremented to two at step 2. At step 3, the third “a” has been processed. Like at step 2, since the “current” index is already pointing to an item in the array that contains “a,” the frequency for “a” is incremented to three at step 3.
 
   At step 4, a “d” has been processed. Since the “current” index points to an item in the array that contains “a” and does not contain “d,” the frequency for “a” is decremented and the “current” index is moved to point to the next item in the array. At step 5, another “d” has been processed. Since decrementing the frequency for “b” (the item to which the “current” index points) results in a frequency of zero for “b,” the “b” is replaced with the “d,” and the “d” is indicated by a frequency of one.’ Furthermore, the “current” index is moved to point to the next item of the array, containing “c.” At step 6, an additional “b” has been processed. Since the “current” index pointed to “c” with a frequency of one, the “c” is replaced with the “b” and the “b” is indicated by a frequency of one. Furthermore, the “current” index is moved to point to the “a.” 
   Thus, at any time, the array includes indications of search term frequencies (perhaps weighted in view of the operation of the algorithm that maintains the indications) for a particular geographic location (or, more generally, for a particular characteristic value on which the events were categorized).  FIG. 4  illustrates schematically how the various arrays (denoted by reference numerals  406   a   1 ,  406   a   2 , and  406   a   3 ;  406   b   1  and  406   b   2 ;  406   c   1  and  406   c   2 ; and  406   x   1 ) are “dumped” to a data structure  408  (e.g., a shared memory hash table) accessible for display. In the  FIG. 4  example, the data records in the structure  408  each include the query indications and frequency indications (e.g., “a1” may indicate a particular query term and its associated frequency), as well as including an indication of the geographic location the query indications are for and a time corresponding to the record (e.g., the last time a query indication was updated, or the time the data of the record was provided from a partition). 
     FIG. 5  illustrates an example of a display  502  that may be generated based on the generated statistics. The portion  504  of the display  502  graphically indicates (e.g., in a map format) geographic locations for indications of query terms and associated frequencies have been determined. In some examples, depending on a level of zoom for which the portion  504  is configured, the locations indicated in the portion  504  may be a function of the frequency of query terms for the locations. In the  FIG. 5  example, a user may cause a cursor to hover over the graphic corresponding to a particular location (in the  FIG. 5  example, location (a)), which then causes a “bubble”  508  to be displayed providing information about the query terms for that particular location. The information may be, for example, information regarding query terms and corresponding frequency for the particular location. Furthermore, an additional portion  506  may display the query terms for that particular location (e.g., the query terms may be displayed in a scrolling manner in the portion  506 ). 
     FIG. 6  is a flowchart of a method in accordance with an embodiment of the invention, to cause determination and display of the level of popularity of query terms provided to one or more search services. At step  602 , event records are received at data collectors from front end servers. At step  604 , the event records are provided from the data collectors to a data warehouse. This includes filtering the event records for the search events. At step  606 , the filtered search events are partitioned based on geographic location indicated in the search records. At step  608 , each partition generates statistics regarding frequency of the search terms for geographic locations of that partition. At step  610 , display of generated statistics regarding frequency of the search terms is caused for particular ones of the geographic locations. It is noted that the sequencing shown in  FIG. 6  is not meant to imply that such a sequencing is required, as it should be apparent that some of the processing may occur in parallel and/or asynchronously. For example, the steps may occur in “real time” (e.g., in a pipelined fashion) with little or no batching. In fact, in some examples, there may be more buffering than batching. 
   We have thus described a method/system to determine frequency of keywords (search query terms) in a search-stream, with a low latency. Additionally, the stream may be partitioned geographically (e.g., by city) and/or temporally (e.g., by hour or by time of day such as morning, afternoon, evening, etc.), allowing a user of the determinations to discern geographic and temporal differences, e.g., in monetizing specific keywords. Search-events from front-end web-servers handling the search queries are processed in real-time. The latency of the frequency determinations may be less than one hour, or even under five minutes. Indications of keyword frequency, may be displayed, for example, relative to a map to allow for easy navigation.