Abstract:
Targeting parameters are generated for a media buy plan for advertisements to be displayed in conjunction with presenting web pages, based on a history of search events. Key phrases are received relative to a subject of the advertisements to be displayed. The received key phrases are provided as proposed key phrases to determine, from search events indicative of historical data of uses of a search service, a first subportion of search events for queries of the search service with the proposed key phrases and a second subportion of search events for queries of the search service not with the proposed key phrases. Classification processing is applied to determine potential targeting parameters associated with the first subportion and with the second subportion to identify potential targeting parameters that, statistically, contribute to membership in the first sub-population and in the second sub-population, respectively. Statistics are associated with the potential targeting parameters, based on the historical data, indicative of factors usable to determine whether to use the potential targeting parameters as actual targeting parameters of the media buy plan.

Description:
BACKGROUND 
     Online marketing, through targeted advertising, has become a popular method for advertisers to reach customers who may be particularly interested in the products/services offered by those advertisers. Generally, a media buy plan of a graphical(display) advertising “campaign” includes a designation of parameters to trigger display of the advertising of the campaign including, for example, characteristics of a user to whom the advertising is to be displayed and/or identification of web page domains on which the advertising is to be displayed. 
     An advertiser may work with a “sales planner” to determine what are thought to be appropriate targeting parameters such as user characteristics and web page domains to target for display advertisements to reach a desirable target audience. Such determination is typically made heuristically. However, this heuristic determination can be difficult, particularly in a “cold start” situation, in which the advertiser and any advisor (such as an advisor affiliated with the online service providing the marketing) may not have enough relevant experience to determine the targeting parameters heuristically. 
     For example,  FIG. 1  is a flowchart that broadly illustrates a campaign flow. At  102 , start-of-campaign recommendations are provided. This would typically be provided by a sales planner. At  104 , the advertisement campaign is run. At  106 , in-campaign recommendations may be provided. 
     SUMMARY 
     In accordance with an aspect, targeting parameters are generated for a media buy plan for advertisements to be displayed in conjunction with presenting web pages, based on a history of search events. Key phrases are received relative to a subject of the advertisements to be displayed. The received key phrases are provided as proposed key phrases to determine, from search events indicative of historical data of uses of a search service, a first subportion of search events for queries of the search service with the proposed key phrases and a second subportion of search events for queries of the search service not with the proposed key phrases. Classification processing is applied to determine potential targeting parameters associated with the first subportion and with the second subportion to identify potential targeting parameters that, statistically, contribute to membership in the first sub-population and in the second sub-population, respectively. Statistics are associated with the potential targeting parameters, based on the historical data, indicative of factors usable to determine whether to use the potential targeting parameters as actual targeting parameters of the media buy plan. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a flowchart that broadly illustrates a campaign flow. 
         FIG. 2  schematically illustrates an architecture of a system in which parameters associated with previous searches for keywords corresponding to the product of the campaign may be processed and analyzed to infer recommendations regarding targeting parameters that may be appropriate to reach a desirable target audience. 
         FIG. 3  illustrates an example of a portion of a result table that provides information on searching carried out with respect to various keywords relative to the product of the campaign. 
         FIG. 4  illustrates an example table of network usage and user profiles for users, relative to various keywords. 
         FIG. 5  illustrates an example decision tree resulting from classification processing of network usage and user profiles. 
         FIG. 6  is a flowchart illustrating an example of processing to develop start-of-campaign recommendations. 
         FIG. 7  is a simplified diagram of a network environment in which specific embodiments of the present invention may be implemented. 
     
    
    
     DETAILED DESCRIPTION 
     The inventor has realized that it can be useful, relative to developing a media buy plan for a display advertising campaign, to process and analyze characteristics associated with a history of events, some of which are search events using particular keywords deemed to be relevant to a product of the campaign, to infer recommendations of targeting parameters (which, generally, are not keyword-based or, at least, comprise parameters other than keyword-based parameters) regarding the campaign. In general, these inferred recommendations are directed to start-of-campaign recommendations, such as in  102  of  FIG. 1  discussed in the Background. 
       FIG. 2  schematically illustrates an architecture of a system in which the characteristics associated with search events may be processed and analyzed to infer recommended targeting parameters regarding a campaign. Referring to  FIG. 2 , a sales planner  202  provides initial key phrases  204  have been determined (e.g., by the sales planner  202 ) to be good candidates for keywords that relate to the subject of the proposed advertising campaign. For example, if the subject of the proposed advertisement campaign is “Rogaine men&#39;s hair loss treatment,” possible initial keywords may include “Rogaine,” “men” and “hair loss.” 
     The initial key phrases  204  are run through a search word expansion processing  206  to generate “N” search keywords  208  (in this context, “keywords” are not limited to a single word in the conventional sense of what is a “word”) to evaluate. The search word expansion processing returns “similar” terms (e.g., processing to return words with the same root, synonyms, perhaps even words having the same root as the synonyms, and other processing). For example, see the discussion at http://searchmarketing.yahoo.com/dever/docs/V2/reference/data/RelatedKeywordRequestTyp.php which describes parameters to a RelatedKeywordRequestType object that is available from Yahoo via a web service API. The result is a table that includes the “N” search keywords.  FIG. 3  illustrates an example of a portion of such a table. In the  FIG. 3  example, a first column  302  includes the “N” search keywords. A second column  304  provides an estimate of the number of monthly searches for each of the “N” search keywords. A third column provides a user interface via which a sales planner  202  or other user may cause one of the “N” keywords to be added to a list of “M” keywords, which are presumably the search keywords the sales planner  202  deems to be most relevant to the product/service of the advertisement campaign. 
     Returning to  FIG. 2 , the system includes a data warehouse  212  that includes transaction records. As referred to by Yahoo!, and as used herein, the transaction records are known as “bcookies.” An extraction processing  214  may be performed to retrieve a list of all bcookies that have transaction information relating to a search using any of the “M” search keywords  210  (denoted as being in the class of “Search=Y). If the list of bcookies is large, sampling methodologies may be employed to reduce computation time to later process this reduced set of bcookies. The extraction processing  214  may also include processing to retrieve bcookies that have transaction information for searches not using any of the “M” search keywords  210  (denoted as being in the class of “Search=N”). 
     Potential targeting parameters, which in this specific example includes network usage and user profiles  216 , are retrieved from the data warehouse  212  for the retrieved bcookies so, in one example, a record is generated for each bcookie, including available network usage and user profile information. For example,  FIG. 4  illustrates an example of a resultant table  400  of network usage and user profiles. The example table  400  includes columns corresponding to network usage data  402  (in the example, indicating number of page views for various Yahoo! properties); corresponding to demographic data  404  (in the example, corresponding to gender, age, ZIP code, etc.); and corresponding to behavioral data  406  (in the example, corresponding to whether the user is likely to be interested in buying cars denoted by “BT/Auto”). A column  408  indicates whether the user has searched on any of the “M” search keywords. 
     Returning again to  FIG. 2 , classification processing  218  may be performed on the information of the retrieved bcookies, to generate a decision tree  220 . More particularly, the decision tree  220  is a decision tree representing factors that contribute to membership in the class of Search=Y relative to membership in the class of Search=N. Other classification representations may be utilized as well. A decision tree can be flattened out to yield targeting rules. 
     In addition, statistics may be associated with the potential targeting parameters, based on the historical data, indicative of factors usable to determine whether to actually target an advertising campaign to those potential targeting parameters. For example, recall and precision metrics  222  may be estimated for each leaf node, based on data extracted using extraction processing  224  according to a list  226  of properties, positions and targeting options corresponding to the leaf nodes of the decision tree  220 . Recall is an indication of the bcookies represented by a particular leaf node as a percentage of the total number of bcookies represented by the decision tree. Precision is an indication of the conciseness of the candidate set. That is, precision is an indication of how many bcookies represented by a particular leaf node are in the Search=Y class. Furthermore, the statistics may be indicative of expected cost of targeting with respect to the characteristics represented by a particular leaf node, based on factors such as an expected number of impressions, cost per impression, and available inventory. 
       FIG. 5  illustrates an example decision tree. Each leaf node (i.e., a leaf node is a node of the decision tree that has no children nodes) has associated with it a precision and recall measurement or estimate, such as is mentioned above. Referring to the  FIG. 5  example decision tree (portions of the decision tree  500  are not shown in  FIG. 5 ), the decision represented by the root node  502  relates to age. In particular, the decision is for age less than or equal to 53.4 (for the left branch  504 ) and for age greater than 53.4 (for the right branch  506 ).  FIG. 5  illustrates, for each leaf node of the decision tree  500 , the number of users with “Search=Y” and the number of users with “Search=N.” For example, the leaf node  508  indicates 175 users with “Search=Y” and 52 users with “Search=N.” 
     Another leaf node of the decision tree  500  is leaf node  510 . As shown in  FIG. 5 , the leaf node  510  is reached in the decision tree  500  by determining that a particular user is of age less than or equal to 53.4 (branch  504 ), is not a popmail user (branch  512 ) and has mail page views greater than 47 (branch  514 ). Referring specifically to leaf node  510 , there are 9 users with “Search=Y” and 2 users with “Search=N.” Furthermore, relative to the total size of the dataset (e.g., the number of records in the  FIG. 4  table  400 ) for which the  FIG. 5  decision tree has been generated, the associated precision and recall measurement or estimate may be viewed. With regard to the example leaf node  510 , assuming a total size of the dataset on which classification is based being 664, the recall is determined to be 11/664 and the precision is determined to be 9/11. For example, for the various example leaf nodes, the statistics in the following Table 1 may be generated. (The first row corresponds to leaf node  510  in  FIG. 5 .) 
     
       
         
               
               
               
               
               
               
               
               
             
               
               
               
               
               
               
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                   
                   
                   
                   
                 Estimated 
                 Estimated 
                   
                   
               
               
                   
                   
                   
                   
                 Daily 
                 relevant 
                 Daily 
                 Relevant 
               
               
                 Rule 
                 CPM 
                 Recall 
                 Precision 
                 Inventory 
                 reach 
                 Spend 
                 CPM 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 [age &lt;= 53.4], 
                 $1.00 
                 1.60% 
                 81% 
                 16000 
                 12960 
                  $16.00 
                  $1.23 
               
               
                 [popmail = 0], 
                   
                   
                   
                   
                   
                   
                   
               
               
                 [MailPVS = Y] 
                   
                   
                   
                   
                   
                   
                   
               
               
                 [age &gt;= 53.4], 
                 $1.20 
                 10.60% 
                 21% 
                 106000 
                 22260 
                 $127.20 
                  $5.71 
               
               
                 [popmail = 1], 
                   
                   
                   
                   
                   
                   
                   
               
               
                 [MailPVS = Y] 
                   
                   
                   
                   
                   
                   
                   
               
               
                 [age &lt;= 53.4], 
                 $0.45 
                 12.60% 
                 1% 
                 126000 
                 1260 
                  $56.70 
                 $45.00 
               
               
                 [popmail = 0], 
                   
                   
                   
                   
                   
                   
                   
               
               
                 [MailPVS = N] 
               
               
                   
               
               
                 *Relevant CPM = the cost of showing 1000 impressions to actually targeted audience (this will increase relative to CPM as precision reduces) 
               
             
          
         
       
     
     Referring back to  FIG. 2 , the decision tree, recall metrics as well as campaign information  230  such as budget, reach, campaign date, expected click through rates, etc. are utilized in processing  228  to determine whether to recommend particular potential targeting parameters as actual targeting parameters. Results  232  are displayed as targeting rules for use in the start-of-campaign recommendations (such as start-of-campaign recommendations  102  in  FIG. 1 ). 
     For example, referring to Table 1, for leaf node  510 , the recall is 1.6% (which is 11/664) and the precision is 81% (which is 9/11). For the example, it is assumed that the cost of buying inventory of the type represented by node  510  is $1 per 1000 impressions (i.e., $0.001). Continuing with the example, it may be assumed that the total number of impressions expected in any one day for the properties represented by the decision tree is one million. This would result in an estimated number of impressions satisfying the conditions for leaf node  510  being equal to 1 million*1.6%, or about 16,000 (i.e., total impressions*recall). The total number of impressions in category “search=Y” would be expected to be 81% of 16,000, or about 12,960. So if the advertiser considered buying all of the inventory matching the targeting parameters corresponding to leaf node  510 , it would cost about 16,000*$0.001, or $16 per day. Similar statistics are available for targeting parameters corresponding to each leaf node of the decision tree as shown, for example, in the second and third rows of Table 1. 
     Thus, the results  232  may include listings of various potential targeting parameters that, in view of the statistics associated with the targeting rules, correspond to the campaign information  230 . The results  232  may further include information as to cost and expected impressions, for example, for the various targeting rules and/or combinations of targeting rules. Using the results  232 , start-of-campaign recommendations may be provided. 
     We now discuss, relative to the flowchart in  FIG. 6 , an example of processing to develop start-of-campaign recommendations in accordance with a broad aspect. At  602 , initial key phrases are provided. At  604 , which is an optional step, the initial key phrases are expanded. At  606 , it is determined what sub-populations of users (e.g., by processing transactions records from a data warehouse) have searched on the key phrases and have not searched on the key phrases. At  608 , classification processing is applied to identify potential targeting parameters relative to the subpopulations who have searched on the key phrases and who have not searched on the key phrases. At  610 , potential targeting parameters identified in the classification processing are examined to generate recommendations of actual targeting parameters regarding a media buy plan. 
     We have described a system/method to process and analyze characteristics associated with previous searches to infer recommendations regarding a media buy plan. Embodiments of the present invention may be employed to infer recommendations regarding a media buy plan in any of a wide variety of computing contexts. For example, as illustrated in  FIG. 7 , implementations are contemplated in which a diverse network environment may be employed, using any type of computer (e.g., desktop, laptop, tablet, etc.)  702 , media computing platforms  703  (e.g., cable and satellite set top boxes and digital video recorders), handheld computing devices (e.g., PDAs)  704 , cell phones  706 , or any other type of computing or communication platform. 
     According to various embodiments, a method of determining the similarity class such as described herein may be implemented as a computer program product having a computer program embodied therein, suitable for execution locally, remotely or a combination of both. The remote aspect is illustrated in  FIG. 7  by server  708  and data store  710  which, as will be understood, may correspond to multiple distributed devices and data stores. 
     The various aspects of the invention may also be practiced in a wide variety of network environments (represented by network  712 ) including, for example, TCP/IP-based networks, telecommunications networks, wireless networks, etc. In addition, the computer program instructions with which embodiments of the invention are implemented may be stored in any type of computer-readable media, and may be executed according to a variety of computing models including, for example, on a stand-alone computing device, or according to a distributed computing model in which various of the functionalities described herein may be effected or employed at different locations.