Patent Publication Number: US-2015088665-A1

Title: Computerized systems and methods related to controlled content optimization

Description:
CROSS-REFERENCE TO RELATED APPLICATION(S) 
     This application claims the benefit of U.S. Provisional Patent Application No. 61/882,845, filed on Sep. 26, 2013, the entire disclosure of which is expressly incorporated herein by reference to its entirety. 
    
    
     BACKGROUND 
     1. Technical Field 
     The present disclosure relates generally to the field of data processing and online advertising and digital content systems. More particularly, and without limitation, the present disclosure relates to computer-implemented systems and methods for controlling the display of digital content and advertisements to users over an electronic network, such as the Internet. The present disclosure also relates to systems and methods for optimizing promotion of such content and advertisements. 
     2. Background Information 
     Publishers of content on the Internet tend to be dependent upon advertising for revenue. For example, various types of web sites, blogs, social networks, and web-based service sites use advertising as a significant source of income to offset the costs associated with offering content and/or services to their users. When a user requests a specific web page, a corresponding web server (either alone or in conjunction with other servers), may identify advertisements or other content to be displayed as part of the web page. For example, a server may attach instructions for a client computer to request an appropriate advertisement from an ad server. Additionally, or alternatively, instructions may be provided to insert an image or other content associated with the ad into the web page. 
     News web sites are one type of web site that rely on advertisements for generating revenue. Such sites may provide various pieces of electronic content for users, including articles, editorials, and/or videos, for example. The content may be independently generated by staff writers and/or compiled from other sources. On news web sites and other content-rich sites, care is often taken to deliver relevant advertisements to users who are reading or viewing the content on the web site. Such web sites may receive money from advertisers, such as manufacturers or resellers, in order to display such content. In some situations, the content may be associated with a “budget” or “goal,” such as a desired maximum number of clicks on, engagement with, or impressions of, that content or related content. For example, if a manufacturer wishes to reach 10,000 potential customers regarding a new product offering, the manufacturer may pay the web site owner for 10,000 impressions of an advertisement for that product. 
     In order to adaptively select content promotions that will yield the desired number of impressions, clicks, or engagements, in the desired amount of time, a bidding system may be used where each content promotion bids on particular promotion slots in an “auction” process. Each content promotion&#39;s bid in each action may be determined based on past performance (for example, click through rate (“CTR”) or other interaction metrics) of the content promotion, the particular promotion slot, or other attributes. 
     However, a poorly-chosen system for generating bids for this auction process can lead to a small number of promotions winning a disproportionately high amount of bids. This can cause problems because advertising budgets for those promotions can be used up too quickly. Other content promotions that do not win many auctions thus may not have enough engagement data with which to generate economically sound bids likely to win later auctions. This may create a self-perpetuating cycle by which content promotions that are not highly successful to begin with receive relatively few impressions, and may bid too much for each received impression. 
     In view of the foregoing, there is a need for improved systems and methods for generating control signals used for optimizing the display of content promotions for web sites. Moreover, there is a need for improved systems and methods for controlling the rate at which content promotions are displayed, by changing the variables used by each promotion to generate bids for the ability to appear in a particular promotion slot. One of ordinary skill will understand from this disclosure that other uses for these embodiments are possible as well. 
     SUMMARY 
     In accordance with embodiments of the present disclosure, computer-implemented systems, methods, and computer-readable media are provided. Exemplary embodiments of the present disclosure include systems and methods for generating control signals for affecting bid data (e.g., price, allocation, and/or uncertainty) for content promotions. The control signals may be based on reference data, such as a desired level of engagement during a time period, and engagement measurements for that content promotion during a previous time period. These control signals may be utilized to affect bid data for use in an auction process. The auction process may be implemented as a market clearing process that selects the highest-bidding content promotion for display in an open promotion slot. 
     By way of example, a computerized method is provided, the method being implemented with at least one processor. The method comprises receiving a candidate promotion and a partition and receiving control data and previous engagement data associated with the partition. The method further comprises, for each partition, generating an adjusted model of the control data over a first time period, generating a factor based on the adjusted model and the previous engagement data, and generating bid price control and bid uncertainty control adjustments. The method further comprises generating a bid price and a bid uncertainty for each candidate promotion. The method then comprises a step of, for each open promotion slot, performing a market clearing process using eligible promotions, using bid prices and bid uncertainties associated with each eligible promotion. 
     Computer-readable media and systems implementing the above method are also provided for. Additional embodiments and related features of the present disclosure are described herein. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate several embodiments and aspects of the present disclosure, and together with the description, serve to explain certain principles of the presently disclosed embodiments. 
         FIG. 1  illustrates an exemplary embodiment of a system for implementing embodiments of the present disclosure. 
         FIG. 2  illustrates an exemplary embodiment of a control system for use with embodiments of the present disclosure. 
         FIG. 3  illustrates an exemplary process of exploration, exploitation, and control for a set of candidate promotions, consistent with embodiments of the present disclosure. 
         FIG. 4  illustrates graphs depicting a sample relationship between bid price, bid uncertainty, and impression volume resulting from each combination of bid price and bid uncertainty. 
         FIG. 5  illustrates an exemplary computer system for implementing embodiments of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
     Embodiments of the present disclosure will now be described. However, it is understood that further embodiments are disclosed in  FIGS. 1-5 , taken together or alone. 
     Embodiments of the present disclosure are generally directed to systems, methods, and computer-readable media for serving content promotions (e.g., Internet advertisements, images, videos, embedded applications) to users. Promotions are selected using a market clearing process that selects a content promotion for each available promotion slot. These promotion slots may be related to a particular position on a static webpage, a dynamic (e.g., periodically updating) webpage, a web site, or the like. A promotion slot, moreover, may refer to a particular location (e.g. below a text area on a webpage) or a general location (e.g., along the left side of each webpage associated with a web site). 
     Embodiments of the disclosed systems, methods, and media involve receiving feedback data on engagements (e.g., clicks or other interactions) related to the exploitation of those content promotions, such as when the content promotions are promoted to users in available promotion slots. Embodiments also involve receiving goals, or “reference data,” such as a desired number of engagements for said content promotions, and adjusting factors related to the content promotions in order to meet such goals. 
     In some embodiments, content promotions may be assigned to particular groups or “partitions” of content promotions. Content promotions can be grouped into a content partition based on advertising campaigns associated with the content promotion. For example, if a manufacturer wants to issue a 40% discount on a particular product, the manufacturer may create a set of content promotions advertising that discount. Each of these related content promotions can be grouped into a single partition so as to avoid over-presentation of that discount to users. Partitions may also be defined based on genre. For example, content promotions that relate to stories about sporting events may be grouped into a “sports” partition while content promotions that relate to stories about international news may be grouped into an “international” partition. In some embodiments, a single content promotion can be a part of one or more partitions. So, continuing this example, a content promotion for a story about the Olympics could be a part of both the “sports” and the “international” partitions. 
       FIG. 1  illustrates an exemplary embodiment of a system  100  for implementing embodiments of the present disclosure. As shown in  FIG. 1 , system  100  includes estimation system  101 , bidding system  102 , and network  103 . Network  103  may be implemented as any known or unknown data network, such as the Internet, an Intranet, a cellular, wireless, wired, or other network. Network  103  may also comprise devices such as web servers that contain web pages. Each web page may have one or more promotion slots for exploiting content promotions. 
     The modules represented in  FIG. 1 , in some embodiments, may be implemented based on the disclosures in U.S. Patent Application Publication No. 2010/0262497 (filed Apr. 10, 2009; titled “Systems and Methods for Controlling Bidding for Online Advertising Campaigns”) or U.S. Patent Application Publication No. 2013/0197994 (filed Nov. 5, 2012; titled “Systems and Methods for Displaying Digital Content and Advertisements over Electronic Networks”), the disclosures of which are hereby incorporated by reference herein. 
     Estimation system  101  and bidding system  102  may be implemented as hardware, software, firmware, or a combination thereof. Estimation system  101  can be implemented to operate in a discrete or real-time fashion. Estimation system  101  comprises two modules, exploration/exploitation module  104  and control module  105 . 
     Exploration/exploitation module  104  receives feedback data from network  103 . Feedback data, in some embodiments, comprises data associated with content promotion exploitation, such as click-through rate, engagement rate, impression rate, or the like. The data may be associated with particular groups of users (e.g., “males” or “female technology enthusiasts between the ages of 25 and 40”) and their particular interactions with particular content promotions and/or partitions of such content promotions. Exploration/exploitation module  104  is configured to generate nominal price and nominal uncertainty bids for each content promotion, based on feedback data associated with that content promotion. In some embodiments, exploration/exploitation module  104  generates prices and uncertainty bids on only one content promotion at a time, using only data associated with that particular content promotion. 
     Control module  105  receives feedback data from network  103  and reference data, and generates bid price control signal adjustments and bid uncertainty control signal adjustments. The reference data received by control module  105  can be received from a user or a controlling system. The reference data comprises, for example, a desired pacing (e.g., a speed) for the exploitation of the content promotions, an allocation signal (e.g., affecting how often a promotion should win an auction) associated with the content promotions for bidding in a market clearing process, or other reference data representing desired outcomes. The reference data may be determined in advance (e.g., based on an advertising contract associated with the content promotion) or may be based on a decision to promote more/fewer of one type of content promotion. In some embodiments, control module  105  generates price and uncertainty bid control signal adjustments on only one content promotion at a time, using only data associated with that content promotion, and only generates such adjustments for content promotions associated with partitions. 
     Bidding system  102  can be implemented to operate in a real-time fashion. Bidding system  102  comprises two modules, Heisenberg perturbation module  106  and market clearing module  107 . 
     Heisenberg perturbation module  106  receives bid data for each content promotion from exploration/exploitation module  104  and control module  105 . Bid data includes, for example, a nominal bid price, a bid allocation, or a bid uncertainty. Heisenberg perturbation module  106  calculates final bid prices based in part on the received bid data. In some embodiments, if a content promotion is associated with a partition, a final bid price for that content promotion may be calculated based on a bid price and a calculated bid control signal adjustment. 
     Market clearing module  107  receives a perturbed bid for each content promotion from Heisenberg perturbation module  106 . Market clearing module then performs a market clearing process for each available promotion slot to determine a content promotion for promoting in that slot, and presents the content promotion to a user in that promotion slot. In some embodiments, presenting the content promotion comprises receiving an impression request from network  103 , and sending it to a server on network  103  for inserting into a web page. One of skill will understand that other arrangements are possible as well. 
       FIG. 2  illustrates an exemplary embodiment of control module  105  for use with embodiments of the present disclosure. Control module  105 , in some embodiments, generates control signal adjustments only for promotions that are associated with partitions. As shown in  FIG. 2 , control module  105  includes a response controller  201 , an adjustment controller  202 , a gain controller  203 , a seasonality controller  204 , a price controller  205 , and an uncertainty controller  206 . Each of response controller  201 , adjustment controller  202 , gain controller  203 , seasonality controller  204 , price controller  205 , and uncertainty controller  206  may be implemented in hardware, software, firmware, or a combination thereof. 
     The controllers in control module  105 , in some embodiments, are designed to ensure that a plant gain factor (K p )and a controller gain factor (K c (k)) remain related to one another. These factors are used to calculate dynamics for control module  105  to generate the adjustments necessary to obtain the desired outcome (e.g., a particular pacing or desired number of engagements). For example, control module  105  may establish an inverse relationship between K c (k) and K p  (e.g., 
     
       
         
           
             
               
                 
                   
                     
                       K 
                       c 
                     
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                   ≈ 
                   
                     
                       K 
                       m 
                     
                     
                       K 
                       p 
                     
                   
                 
                 , 
                 
                   
                     K 
                     m 
                   
                   ≠ 
                   0 
                 
               
               ) 
             
             . 
           
         
       
     
     This inverse relationship may change in real-time and thus a change in one value (e.g., K p ) requires a change in the other value (e.g., K c (k)). The overall input-output relationship (y(k)) of control module  105  may be represented as: 
         y ( k )=(1 +g ( k )) f ( u   p ( k ),  u   u ( k )); where         g(k) is a T-periodic function (where g(t)&gt;−1 and g(t+T)=g(t) for all t);   u p (k) is a bid price adjustment; and   u u (k) is a bid uncertainty adjustment.       
     Response controller  201  is configured to generate a desired feedback response (y m (k)) that will yield the desired rate of feedback for a particular promotion in a particular partition. In some embodiments, this rate may be based on the reference data (ū c (k)) and may be determined in order to read this desired rate. For example, y m (k) may be calculated as follows: 
     
       
         
           
             
               
                  
                 
                   x 
                   m 
                 
               
               
                  
                 t 
               
             
             = 
             
               
                 
                   - 
                   
                     
                       
                         
                           g 
                           ~ 
                         
                         m 
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                     
                       T 
                       i 
                     
                   
                 
                  
                 
                   
                     x 
                     m 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
               
               + 
               
                 
                   
                     u 
                     _ 
                   
                   c 
                 
                  
                 
                   ( 
                   k 
                   ) 
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   y 
                   m 
                 
                  
                 
                   ( 
                   k 
                   ) 
                 
               
               = 
               
                 
                   
                     
                       
                         ( 
                         
                           1 
                           + 
                           
                             g 
                              
                             
                               ( 
                               k 
                               ) 
                             
                           
                         
                         ) 
                       
                        
                       
                         
                           
                             g 
                             ~ 
                           
                           m 
                         
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                        
                       
                         ( 
                         
                           1 
                           - 
                           
                             
                               
                                 g 
                                 ~ 
                               
                               m 
                             
                              
                             
                               ( 
                               k 
                               ) 
                             
                           
                         
                         ) 
                       
                     
                     
                       T 
                       i 
                     
                   
                    
                   
                     
                       x 
                       m 
                     
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                 
                 + 
                 
                   
                     ( 
                     
                       1 
                       + 
                       
                         g 
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                     
                     ) 
                   
                    
                   
                     
                       
                         g 
                         ~ 
                       
                       m 
                     
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                    
                   
                     
                       
                         u 
                         _ 
                       
                       c 
                     
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                 
               
             
             ; 
           
         
       
       
         
           where 
         
       
       
         
           
             
               
                 
                   
                     g 
                     ~ 
                   
                   m 
                 
                  
                 
                   ( 
                   k 
                   ) 
                 
               
               = 
               
                 
                   
                     K 
                     m 
                   
                    
                   
                     ( 
                     
                       1 
                       + 
                       
                         g 
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                     
                     ) 
                   
                 
                 
                   1 
                   + 
                   
                     
                       K 
                       m 
                     
                      
                     
                       ( 
                       
                         1 
                         + 
                         
                           g 
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                       
                       ) 
                     
                   
                 
               
             
             , 
           
         
       
     
     and
         T i  is a design parameter.       

     Block  201 A indicates that the desired feedback response y m k) may be combined with actual feedback data (y(k)) to determine the “error” or difference (e m (k)) between these values. For example, e m (k) may be calculated by response controller  201 , controller gain module  204 , or any other module or device, as e m (k)=y(k)−y m (k). 
     Adjustment controller  202  calculates an adjustment factor (z(k)). The adjustment factor may be utilized to ensure that plant gain factor K p  and controller gain factor K c (k) remain related to one another. Adjustment controller  202  may calculate a factor z(k) used to adjust K c (k) such that this inverse proportionality holds. For example, this factor z(k) may be utilized by gain controller  203  along with the error (e m (k)) to calculate a new controller gain factor K c (k) using partial derivatives. In some embodiments, adjustment controller  202  may calculate factor z(k) by determining: 
     
       
         
           
             
               
                  
                 
                   x 
                   z 
                 
               
               
                  
                 t 
               
             
             = 
             
               
                 
                   [ 
                   
                     
                       
                         
                           - 
                           
                             
                               2 
                                
                               
                                   
                               
                                
                               
                                 
                                   
                                     g 
                                     ~ 
                                   
                                   m 
                                 
                                  
                                 
                                   ( 
                                   k 
                                   ) 
                                 
                               
                             
                             
                               T 
                               i 
                             
                           
                         
                       
                       
                         
                           - 
                           
                             
                               
                                 
                                   
                                     g 
                                     ~ 
                                   
                                   m 
                                 
                                  
                                 
                                   ( 
                                   k 
                                   ) 
                                 
                               
                               2 
                             
                             
                               T 
                               i 
                               2 
                             
                           
                         
                       
                     
                     
                       
                         1 
                       
                       
                         0 
                       
                     
                   
                   ] 
                 
                  
                 
                   
                     x 
                     z 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
               
               + 
               
                 
                   [ 
                   
                     
                       
                         1 
                       
                     
                     
                       
                         0 
                       
                     
                   
                   ] 
                 
                  
                 
                   
                     
                       u 
                       ~ 
                     
                     c 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               z 
                
               
                 ( 
                 k 
                 ) 
               
             
             = 
             
               
                 
                   - 
                   
                     
                       
                         
                           
                             
                               g 
                               ~ 
                             
                             m 
                           
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                         2 
                       
                       
                         K 
                         m 
                       
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 1 
                                 - 
                                 
                                   2 
                                    
                                   
                                       
                                   
                                    
                                   
                                     T 
                                     i 
                                   
                                    
                                   
                                     
                                       
                                         g 
                                         ~ 
                                       
                                       m 
                                     
                                      
                                     
                                       ( 
                                       k 
                                       ) 
                                     
                                   
                                 
                               
                               
                                 T 
                                 i 
                               
                             
                           
                           
                             
                               - 
                               
                                 
                                   
                                     
                                       
                                         g 
                                         ~ 
                                       
                                       m 
                                     
                                      
                                     
                                       ( 
                                       k 
                                       ) 
                                     
                                   
                                   2 
                                 
                                 
                                   T 
                                   i 
                                   2 
                                 
                               
                             
                           
                         
                       
                       ] 
                     
                   
                 
                  
                 
                   
                     x 
                     2 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
               
               + 
               
                 
                   
                     
                       
                         
                           g 
                           ~ 
                         
                         m 
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                     2 
                   
                   
                     K 
                     m 
                   
                 
                  
                 
                   
                     
                       
                         u 
                         ~ 
                       
                       c 
                     
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                   . 
                 
               
             
           
         
       
     
     Gain controller  203  utilizes error e m (k) and factor z(k) to calculate a new controller gain factor K c (k) to maintain the inverse relationship between controller gain factor K c (k) and plant gain factor K p . In some embodiments, the actual relationship between controller gain factor K c (k) and plant gain factor K p  (i.e., the K m  in the above example relationship 
     
       
         
           
             
               
                 
                   
                     K 
                     c 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
                 ≈ 
                 
                   
                     K 
                     m 
                   
                   
                     K 
                     p 
                   
                 
               
               , 
               
                 
                   K 
                   m 
                 
                 ≠ 
                 0 
               
             
             ) 
           
         
       
     
     may not be known but may be approximated using partial derivatives. For example, gain controller  203  may determine controller gain factor K c (k) based on the following relationship: 
     
       
         
           
             
               
                 
                    
                   
                     K 
                     c 
                   
                 
                 
                    
                   t 
                 
               
               = 
               
                 
                   - 
                   γ 
                 
                  
                 
                     
                 
                  
                 
                   
                     e 
                     m 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
                  
                 
                   
                     z 
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                   
                     
                       K 
                       c 
                     
                      
                     
                       ( 
                       k 
                       ) 
                     
                   
                 
               
             
             , 
           
         
       
     
     where γ is a design parameter. One of ordinary skill will understand that there are other methods for calculating K c (k). 
     Seasonality controller  204  is configured to generate a seasonally-adjusted reference signal u c (k) based on reference data ū c (k). This seasonally-adjusted signal u c (k) is used to redistribute the budget for a particular content promotion over a period of time based on a known supply function. One example of a known supply function relates to network traffic over time. Internet traffic patterns in many regions of the world vary based on the time of day, day of week, or the like. For example, traffic coming from computers in North America tends to be higher during the daylight hours than during the late night hours, because most people are asleep during the latter period. Seasonality controller  204  may account for this “seasonality” or variance in content promotion engagement using a harmonic equation. In some embodiments, the seasonally-adjusted reference signal is generated using a two-harmonic equation. For example, seasonality controller  204  receives factors α l,1 , α l,2 , φ l,1 , and φ l,2  (each representing various factors selected based on particular seasonality patterns and related to partition l), and generates a seasonally-adjusted reference signal as: 
     
       
         
           
             
               
                 
                   u 
                   c 
                 
                  
                 
                   ( 
                   k 
                   ) 
                 
               
               = 
               
                 
                   ( 
                   
                     
                       
                         α 
                         
                           l 
                           , 
                           1 
                         
                       
                        
                       
                         sin 
                         ( 
                         
                           
                             
                               2 
                                
                               
                                   
                               
                                
                               π 
                                
                               
                                   
                               
                                
                               k 
                             
                             24 
                           
                           + 
                           
                             φ 
                             
                               l 
                               , 
                               1 
                             
                           
                         
                         ) 
                       
                     
                     + 
                     
                       
                         α 
                         
                           l 
                           , 
                           2 
                         
                       
                        
                       
                         sin 
                         ( 
                         
                           
                             
                               4 
                                
                               
                                   
                               
                                
                               π 
                                
                               
                                   
                               
                                
                               k 
                             
                             24 
                           
                           + 
                           
                             φ 
                             
                               l 
                               , 
                               2 
                             
                           
                         
                         ) 
                       
                     
                   
                   ) 
                 
                  
                 
                   
                     
                       u 
                       _ 
                     
                     c 
                   
                    
                   
                     ( 
                     k 
                     ) 
                   
                 
               
             
             , 
           
         
       
     
     In the above equation, α l,1  and α l,2  relate to the amplitude of the signal, while φ l,1  and φ l,2  relate to the phase of the signal. In some embodiments, α l,1 , α l,2 , φ l,1 , and φ l,2  are chosen such that u c (k)≧0 for all time measurements k. Other periodic functions, including other truncated Fourier series expansions of periodic functions, are possible as well. Block  203 A indicates that the seasonally-adjusted signal u c (k) may be combined with feedback data y(k) to determine the “error” or difference (e(k)) between these values. For example, this may be calculated by seasonality controller  204 , price controller  205 , or any other module or device, as 
         e ( k )= u   c ( k )− y ( k ).
 
     Price controller  205  generates a bid price adjustment u p (k) based on received data. In some embodiments, price controller  205  may be implemented as a “PI controller” (also known as a “Proportional-Integral controller”) which attempts to minimize the difference between the reference data ū c (k) and y(k). Price controller module  205  determines a bid price control signal u p (k), using controller gain factor K c (k), error e(k), and other factors, including the change in time between the current update and the most recent update, previous measurements of bid price control signals, or the like. For example, price controller module  205  may receive error e(k) and controller gain factor K c (k), and calculate bid price adjustment signal u p (k) as: 
     
       
         
           
             
               
                 u 
                 p 
               
                
               
                 ( 
                 k 
                 ) 
               
             
             = 
             
               
                 
                   K 
                   c 
                 
                  
                 
                   ( 
                   k 
                   ) 
                 
               
                
               
                 
                   ( 
                   
                     
                       e 
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                     + 
                     
                       
                         1 
                         
                           T 
                           i 
                         
                       
                        
                       
                         
                           ∫ 
                           0 
                           t 
                         
                          
                         
                           
                             e 
                              
                             
                               ( 
                               τ 
                               ) 
                             
                           
                            
                           
                               
                           
                            
                           
                              
                             τ 
                           
                         
                       
                     
                   
                   ) 
                 
                 . 
               
             
           
         
       
     
     Uncertainty controller  206  is configured to generate a bid uncertainty adjustment u u (k). The bid uncertainty adjustment u u (k) may be calculated to shape the plant gain factor K p  as follows: 
         u   u ( k )= u   u   0   +K   u ( K   c ( k )− K   c   0 ) 2 ; where
         u u   0  and K c   0  represent baseline values for bid uncertainty adjustment u u (k) and K c (k), respectively; and   K u  represents a design parameter≠0.       

     In some embodiments, the bid price adjustment u p (k) and bid uncertainty adjustment u u (k) are combined by a module (such as Heisenberg perturbation module  106  in  FIG. 1 ) with the expected value and variance for the content promotion (θ i (k) and σ i (k), respectively) to generate a final bid price, bid uncertainty, and bid allocation for use in a market clearing process. 
     While some embodiments of the modules in control module  105  have been described above, other arrangements of the modules in control module  105  are also possible as well. 
       FIG. 3  illustrates an exemplary process of exploration, exploitation, and control for a set of candidate promotions. The exemplary process in  FIG. 3  may be executed by one or more systems, modules, or software programs, such as that illustrated in  FIG. 1  and  FIG. 2 . 
     The process in  FIG. 3  begins at step  301  for each candidate promotion i that has not previously been exploited. Parameters α i (k),β i (k) are initialized to initial values α i,0 ,β i,0  to begin the exploitation, exploration, and control process. These parameters are used to generate a probability density function representing engagement rate for a candidate promotion at time k. 
     However, if the candidate promotion i has previously been exploited (e.g., has already been displayed to one or more users), the process instead starts at step  303 , where estimator inputs, such as a time series of impression measurements ñ i,l (k) or engagement volume measurements ñ i,C (k), are filtered. In step  305 , the estimator state is propagated based on the filtered measurements to create parameters for generating a probability density function for the expected engagement rate for the candidate promotion. For example, the parameters α i (k),β i (k) can be generated based on γ Δ     k    (γ being a chosen factor less than 1.0, such as 0.99, and Δ k  being the difference between the current time and the time of the last measurement), initial pre-set parameters α i,0 ,β i,0 , earlier values of those parameters (α i (k−1),β i (k−1)), a filtered impression measurement ñ i,l (k), a filtered volume measurement ñ i,C (k), or the like. In some embodiments, one way of filtering the impression measurements ñ i,l (k) and the engagement volume measurements ñ i,C (k) is: 
     
       
         
           
             
               
                 
                   
                     n 
                     ~ 
                   
                   
                     i 
                     , 
                     I 
                   
                 
                  
                 
                   ( 
                   k 
                   ) 
                 
               
               = 
               
                 min 
                  
                 
                     
                 
                  
                 
                   ( 
                   
                     
                       
                         n 
                         
                           i 
                           , 
                           I 
                         
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                     , 
                     
                       
                         n 
                         
                           I 
                           , 
                           max 
                         
                       
                        
                       
                         Δ 
                         k 
                       
                     
                     , 
                     
                       
                         ρ 
                         
                           Δ 
                           k 
                         
                       
                        
                       
                         
                           β 
                           i 
                         
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                     
                   
                   ) 
                 
               
             
             ; 
           
         
       
       
         
           
             
               
                 
                   n 
                   ~ 
                 
                 
                   i 
                   , 
                   C 
                 
               
                
               
                 ( 
                 k 
                 ) 
               
             
             = 
             
               { 
               
                 
                   
                     
                       
                         
                           
                             
                               
                                 n 
                                 ~ 
                               
                               
                                 i 
                                 , 
                                 I 
                               
                             
                              
                             
                               ( 
                               k 
                               ) 
                             
                           
                           
                             
                               n 
                               
                                 i 
                                 , 
                                 I 
                               
                             
                              
                             
                               ( 
                               k 
                               ) 
                             
                           
                         
                          
                         
                           
                             n 
                             
                               i 
                               , 
                               C 
                             
                           
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                       
                       , 
                     
                   
                   
                     
                       
                         if 
                          
                         
                             
                         
                          
                         
                           
                             n 
                             
                               i 
                               , 
                               I 
                             
                           
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                       
                       &gt; 
                       
                         
                           
                             n 
                             ~ 
                           
                           
                             i 
                             , 
                             I 
                           
                         
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                     
                   
                 
                 
                   
                     
                       
                         n 
                         
                           i 
                           , 
                           C 
                         
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                   
                   
                     else 
                   
                 
               
             
           
         
       
     
     Parameters α i (k),β i (k) are then calculated based on the earlier values of the parameters and the filtered measurements, as: 
     
       
         
           
             
               [ 
               
                 
                   
                     
                       
                         α 
                         i 
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                   
                 
                 
                   
                     
                       
                         β 
                         i 
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                   
                 
               
               ] 
             
             = 
             
               
                 
                   ( 
                   
                     1 
                     - 
                     
                       γ 
                       
                         Δ 
                         k 
                       
                     
                   
                   ) 
                 
                  
                 
                   [ 
                   
                     
                       
                         
                           α 
                           
                             i 
                             , 
                             0 
                           
                         
                       
                     
                     
                       
                         
                           β 
                           
                             i 
                             , 
                             0 
                           
                         
                       
                     
                   
                   ] 
                 
               
               + 
               
                 
                   γ 
                   
                     Δ 
                     k 
                   
                 
                  
                 
                   
                     λ 
                     
                       Δ 
                       k 
                     
                   
                    
                   
                     [ 
                     
                       
                         
                           
                             
                               α 
                               i 
                             
                              
                             
                               ( 
                               
                                 k 
                                 - 
                                 1 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               β 
                               i 
                             
                              
                             
                               ( 
                               
                                 k 
                                 - 
                                 1 
                               
                               ) 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
               + 
               
                 [ 
                 
                   
                     
                       
                         
                           
                             n 
                             ~ 
                           
                           
                             i 
                             , 
                             C 
                           
                         
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                     
                   
                   
                     
                       
                         
                           
                             n 
                             ~ 
                           
                           
                             i 
                             , 
                             I 
                           
                         
                          
                         
                           ( 
                           k 
                           ) 
                         
                       
                     
                   
                 
                 ] 
               
             
           
         
       
     
     The process in  FIG. 3  continues to step  307 . Step  307  outputs an estimation of two parameters θ i (k) and σ i (k), indicating the estimated expected value and relative standard deviation of the engagement rate for the candidate promotion, respectively. In some embodiments, these values may be calculated based on the calculated parameters α i (k) and β i (k), as follows: 
     
       
         
           
             
               [ 
               
                 
                   
                     
                       
                         θ 
                         i 
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                   
                 
                 
                   
                     
                       
                         σ 
                         i 
                       
                        
                       
                         ( 
                         k 
                         ) 
                       
                     
                   
                 
               
               ] 
             
             = 
             
               
                 [ 
                 
                   
                     
                       
                         
                           
                             α 
                             i 
                           
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                         / 
                         
                           
                             β 
                             i 
                           
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                       
                     
                   
                   
                     
                       
                         1 
                         / 
                         
                           
                             
                               α 
                               i 
                             
                              
                             
                               ( 
                               k 
                               ) 
                             
                           
                         
                       
                     
                   
                 
                 ] 
               
               . 
             
           
         
       
     
     After calculating the expected value and relative standard deviation of the engagement rate, the process continues to steps  309 - 311 . Steps  309 - 311  are executed once for each partition l. In some embodiments, each partition l contains at least one candidate promotion i, and each candidate promotion i is a member of at most one partition l. In other embodiments, each candidate promotion i can be a member of one or more partitions. (Moreover, if a particular promotion i is not a member of any partition, the process continues to step  313  without generating control signal adjustments in steps  309  or  311 .) 
     In step  309 , a controller is initialized for each partition l that is “new” (e.g., having candidate promotions that have not yet been displayed to users). Parameters u l,p (k),u l,a (k), and u l,u (k), representing control signal adjustments for bid price, bid allocation, and bid uncertainty, respectively, are initialized to initial values (u l,p   0 ,u l,a   0 ,u l,u   0 ). These parameters are used in conjunction with the expected value and variance for the content promotion (θ i (k) and σ i (k), respectively) to generate a final bid price, bid uncertainty, and bid allocation for use in a market clearing process. 
     However, if content promotions in partition l have already been displayed to users, the process continues to step  311 . In step  311 , the control signal adjustments are updated for generating a final bid price. u l,p (k), u l,a (k), and u l,u (k) are calculated based on values such as n l,C (k) (engagement volume measurements for the partition l associated with the content promotion), u l,a (k−1) (allocation signal associated with the partition l during a previous time period), n l,C   ref (k) (a “reference,” or desired, daily value for engagement for the partition l), Δ k  (the difference between the current time and the last time that a measurement was made), δ l  (a factor whose component parts include n l,C   ref  (k) and a seasonally-adjusted reference signal for n l,C   ref (k)), and a variety of constant values. For example, these control signal adjustments may be calculated as explained above with reference to  FIG. 2 . 
     After determining the values for the control signal adjustments u l,p (k),u l,a (k), and u l,u (k) associated with a partition l (e.g., either through calculation as explained in step  311  or through initial values as explained in step  309 ), these control signal adjustments may be combined with the expected value and variance (θ i (k)and σ i (k), respectively) for each content promotion, to generate a bid price and bid uncertainty for the content promotion. For example, in some embodiments, the bid price b i,p (k) may be based on addition of the expected value and the bid price control signal adjustment (e.g., b i,p (k)=θ i (k)+u l,p (k)), the bid uncertainty b i,u (k) may be based on addition of the expected variance and the bid uncertainty control signal adjustment (e.g., b i,u (k)=σ i (k)+u l,u (k)), and the bid allocation may be based on a calculated bid allocation control signal adjustment or a pre-set bid allocation control signal adjustment (e.g., b i,a (k)=u l,u (k)). These values may be combined in a variety of ways in other embodiments—such as through multiplication, division, or the like. 
     In some embodiments, the control signal adjustments are only used in step  313  for generating bid price, bid allocation, and bid uncertainty if the content promotion belongs to one or more partitions. If a particular content promotion does not belong to a partition l, the bid price, bid allocation, and bid uncertainty be calculated without taking control signal adjustments into account (e.g., b i,p (k)=θ i (k), b i,a (k)=1, and b i,u (k)=σ i (k)). 
     After calculating the final bid price, bid allocation, and bid uncertainty for each candidate promotion, the process continues to step  315 . In step  315 , a process of market clearing is performed on each impression set request j. Market clearing involves, for example, generating a random bid for each eligible candidate promotion i. A candidate promotion is “eligible” if the candidate promotion is able to appear in a particular promotion slot that is part of an impression set request. Whether a candidate promotion is able to appear in a particular promotion slot may be based on, for example, the content of the content promotion and/or a web page containing the promotion slot. 
     Market clearing may also involve identifying the highest-bidding candidate promotion and generating a random number. If the random number is less than a factor of the bid allocation assigned to the highest-bidding candidate promotion (b i,a (k)), the highest-bidding candidate promotion is assigned to the promotion set request (e.g., for presentation in the promotion slot). This process may be repeated until no eligible candidate promotions remain or until all open promotion slots are filled with candidate promotions. The identified candidate promotions are then served (e.g., presented to users on web pages) in descending order of their bid prices. For example, the content promotions can be presented to a user as part of a web page requested by that user. 
     In step  317 , results from the presentation of the candidate promotions are gathered. For example, impression and/or engagement (e.g., click or other interaction) data are gathered. This data is then used as input to step  303 , to enable generation of an estimated state for the next bid calculated related to each eligible content promotion. 
       FIG. 4  illustrates graphs  401  and  402 , depicting a sample relationship between bid price, bid uncertainty, and impression volume resulting from each combination of bid price and bid uncertainty. In some embodiments, graphs  401  and  402  depict the data related to a particular content promotion that has been subject to a market clearing process. The bid price and bid uncertainty are the values used in the market clearing process, and the impression data depicted in graphs  401  and  402  relate to the actual number of impressions received for each combination of bid price and bid uncertainty in the market clearing process. 
     Graph  401  is a contour graph depicting the relationship between bid price (X-axis), bid uncertainty (Y-axis), and impressions related to each combination of bid price and bid uncertainty (contour lines  401 A,  401 B,  401 C, and  401 D). Line  403 A depicts the particular values of each of bid price and bid uncertainty used in a market clearing process, and line  403 B represents the particular values of each estimated expected value and estimated relative standard deviation. The difference (e.g., the distance) between lines  403 A and  403 B represents the bid price adjustment u p  and bid uncertainty adjustment u u . Contour lines  401 A,  401 B,  401 C, and  401 D represent impression estimates for a content promotion. The contour lines  401 A- 401 D may be based on the impressions depicted in line  403 , and may represent expected impressions. For example, contour line  401 C illustrates that the content promotion can expect to receive 1,230,000 impressions with a bid price of approximately 0.022 and a bid uncertainty no larger than approximately 0.75. Contour line  401 A, in contrast, shows a relationship between bid price and bid uncertainty, illustrating the content promotion can expect to receive 410,000 impressions with a bid price of approximately 0.014, but with a more widely varying bid uncertainty in order to win a constant number of impressions. Graph  402  is a surface graph depicting the same relationship as in graph  401 . 
       FIG. 5  illustrates an exemplary computer system  500  for implementing embodiments consistent with the present disclosure. Variations of computer system  500  may be used for implementing devices, algorithms, or other systems, described in this specification or in  FIGS. 1-4 . Such devices include those that would be understood or contemplated by those skilled in the art. Persons skilled in the art will also understand, from the present disclosure, that the components represented in  FIG. 5  may be duplicated, omitted, or modified. 
     As shown in  FIG. 5 , exemplary computer system  500  may include a central processing unit  501  (also referred to as an electronic processor) for managing and processing data, as well as operations, consistent with the present disclosure. Computer system  500  also includes storage device  503 . Storage device  503  comprises optical, magnetic, signal, and/or any other type of storage device. Computer system  500  may also include network adapter  505 . Network adapter  505  allows computer system  500  to connect to electronic networks, such as the Internet, a local area network, a wide area network, a cellular network, a wireless network, or any other type of network. Computer system  500  also includes power unit  506 , which may enable computer system  500  and its components to receive power and operate fully. 
     In some embodiments, computer system  500  may also include input device  502 , which receive input from users and/or modules or devices. Such modules or devices may include, but are not limited to, keyboards, mice, trackballs, trackpads, scanners, cameras, and other devices which connect via Universal Serial Bus (USB), serial, parallel, infrared, wireless, wired, or other connections. Computer system  500  also includes output device  504 , which transmit data to users and/or modules or devices. Such modules or devices may include, but are not limited to, computer monitors, televisions, screens, projectors, printers, plotters, and other recording/displaying devices which connect via wired or wireless connections. 
     In this disclosure, various embodiments have been described with reference to the accompanying drawings and embodiments. It will, however, be evident that various modifications and changes may be made thereto, and additional embodiments may be implemented, without departing from the present disclosure. The specification and drawings are accordingly to be regarded in an illustrative rather than restrictive sense. 
     For example, advantageous results may still be achieved if steps of the disclosed methods were performed in a different order and/or if components in the disclosed systems were combined in a different manner and/or replaced or supplemented by other components. Other implementations are also within the scope of the present disclosure. Further, the accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and together with the description, and are similarly not restrictive.