Optimized selection and delivery of content

Technologies for optimized selection of content for delivery to a user that both optimizes the expected return from the delivery of the content to the user and that enables exploration of delivery of new content to users are disclosed. Content is selected for delivery to a user based on an exploitation score that defines an estimate of the feedback expected from the delivery of the content to the user and an exploration score that varies inversely with the number of times that the content has been transmitted to all users. The use of the exploration score enables the exploration of delivery of new content to users. The content might be delivered via e-mail messages, a web site, or using another mechanism.

BACKGROUND

In large organizations it is not unusual for many different operating units to have an interest in sending electronic content to customers or potential customers. For example, the operator of an e-commerce web site or other type or retail business might have many different operating units that would like to send electronic mail (“email”) messages to customers or potential customers of the e-commerce web site. Each operating unit might, for instance, desire to send content to customers or potential customers advertising certain products or services of interest to a customer or potential customer.

Customers or potential customers of an organization, such as the e-commerce web site described above, typically do not want to receive a large volume of email or other types of messages from a single organization. As a result, an organization might limit the number of messages that are sent to each customer during a particular time period. For example, a limit of one email message per day per customer might be imposed across all of the various operating units within an organization. When such a restriction is imposed, however, it might be difficult to select the content that should be sent to each customer in the messages in order to maximize the potential return from the transmission of the messages while at the same time exploring the effectiveness of new content.

DETAILED DESCRIPTION

The following detailed description is directed to technologies for optimized selection and delivery of content. Through an implementation of the concepts and technologies disclosed herein, content can be selected for delivery to a customer or potential customer (which may be referred to herein simply as a “user”) in a manner that both optimizes the expected return from the transmission of the content, and that also permits exploration of the expected return of new content that has not been previously transmitted to users or that has only been transmitted to users a relatively small number of times.

In one particular configuration, a content selection module is provided that implements computer learning methods in order to identify content for transmission to users, such as in email messages or other types of electronic messages, that is likely to result in a desired user action, such as making a purchase or visiting an e-commerce web site. In selecting content for transmission to a user, it is generally desirable to present the most effective content to the user. The effectiveness of a particular instance of content can be a measure of whether a desired result is obtained from the user and/or whether a desired action is performed by the user. The desired action or result can be any action or result that may be desired from a user once the user has viewed the content. For example, and without limitation, a desired action may be the user selecting a portion of the content, or the user eventually purchasing an item referenced by an instance of content through a hyperlink or in another manner.

Selecting an instance of content for transmission to a user from a number of available instance of content might be thought of as variation of the multi-armed bandit problem. In statistics, a “multi-armed bandit” problem (referencing the “one-armed bandit” term that is commonly used for a slot machine) consists of determining which one of multiple “arms” or levers to select in each of a series of trials, where each lever provides a reward drawn from a distribution associated with that specific lever. The objective is generally to maximize the total reward earned through a sequence of pulls of the levers. Generally, no initial knowledge about the levers is available prior to the first trial. While various strategies have been developed to provide approximate solutions to versions of the multi-armed bandit problem, these solutions often have limited applicability to specific real world circumstances due to their reliance on certain constraints or assumptions regarding the underlying problem.

If a given content selection problem involves selecting an instance of content from among a set of instances of content (e.g. selecting one instance of content from a number of instances of content created by different operating units in an organization) for transmission to a user, such as a customer or potential customer, each instance of content available for transmission might be thought of as an “arm” in a multi-armed bandit problem. Selecting a particular instance of content for transmission to a user, therefore, may be thought of as “pulling” that arm. However, this content selection problem may differ from the traditional multi-armed bandit problem in various ways, such that known approximate solutions to the traditional multi-armed bandit problem may not be well suited to the content selection problem.

For example, content selection, depending on the specific instance or scenario, may differ from the traditional multi-armed bandit problem because feedback indicating whether a particular instance of content was successful in a given trial may not be immediately known. For example, an email message containing the selected content may be transmitted to a user. It may, however, be 12 or more hours before the user views the email message and takes action on the content of the message, such as selecting a hyperlink in the message or requesting to unsubscribe from any future messages. Additional aspects of the content selection problem are discussed in more detail below, along with approaches to solving the various unique aspects of this problem.

In order to address these considerations, and potentially others, a content selection module is provided in one particular configuration. The content selection module is configured to select instances of content for transmission to users, such as customers or potential customers of an e-commerce web site. The selected content might then be transmitted to users through various mechanisms such as, but not limited to, email messages, short messaging system (“SMS”) messages, instant messages, push notifications, and/or other types of electronic messaging systems. The selected content might also be provided in a web page or another type of content page. The selected content might also be provided to users in other ways in other configurations.

As described briefly above, the various instances of content might be authored by different operating units within an organization. For example, one operating unit might desire to send content to customers or potential customers advertising one product or service, while another operating unit desires to send content regarding another product or service. As also discussed above, various constraints might also be placed on the number of messages transmitted to a user in a particular time period, such as one message per day. In this scenario, two or more instances of candidate content might first be identified for a user based upon various attributes or context associated with the user. For example, the instances of candidate content might be identified based upon the user's age, hobbies, previous purchase history, geographic location, and/or other attributes.

In order to select one of the instances of candidate content for transmission to a user, an exploitation score might be computed for each of the instances of candidate content. The exploitation score defines an estimate of the feedback expected from the user following the delivery of the instance of content to the user. The exploitation score may be computed based upon historical feedback received following transmission of an instance of content to one or more users. For example, and without limitation, the exploitation score might indicate the percentage of users to which the content has been previously transmitted that subsequently made a purchase or took another desired action.

An exploration score might also be computed for each of the instances of candidate content. The exploration score is computed by a function in one particular configuration that varies inversely to the number of times that an instance of content has been transmitted to users. In this way, the exploration score will be very high when the instance of candidate content has not yet been transmitted to users or has only been transmitted to users a relatively low number of times. The exploration score will decrease to zero as the instance of candidate content is transmitted to a larger number of users. In this way, a ranking score associated with each instance of candidate content can be artificially inflated while the effectiveness of the content is explored.

In one configuration, a ranking score is also computed for each of the instances of candidate content. For example, and without limitation, the ranking score might be computed by summing the exploitation score and the exploration score for a particular instance of candidate content. Other types of mathematical operations might be utilized on the exploitation score and the exploration score in order to compute the ranking score for a particular instance of candidate content. The instances of candidate content might then be ranked based upon their associated ranking scores.

Once the instances of candidate content for a user have been ranked, one of the instances of candidate content may be selected for transmission to a user. For example, and without limitation, the highest ranked instance of candidate content might be selected for transmission to a user. Other mechanisms might also be utilized to select a particular instance of candidate content for transmission to a user from the ranked instances of candidate content. The selected instance of candidate content may then be transmitted to the user utilizing one or more of the mechanisms described above.

Following transmission of the content to the user, feedback may be obtained from the user based upon the content transmitted to the user. For example, and without limitation, positive feedback might be considered to have been received from the user if the user makes a purchase based on the content. Negative feedback might also be received from the user. For example, and without limitation, the user might request to unsubscribe from any further content. In either case, the exploitation score may be updated based upon the obtained feedback in some configurations. If the obtained feedback is negative, the exploitation score might be penalized dynamically based on one or more attributes. For example, and without limitation, the exploitation score might be penalized in an amount that varies based upon the volume of purchases previously made by the user. Other attributes might also be utilized to dynamically penalize the exploitation score.

In another configuration, instances of candidate content are selected for transmission to a user based upon a distribution over the possible values of the exploitation score for the instances of candidate content. In this configuration, a distribution over the possible values of the exploitation score is computed for each of the instances of candidate content. Sampling is performed over the distributions in order to obtain a ranking score for the instances of candidate content. The ranking scores are then ranked and utilize to select one or more instances of candidate content for transmission to a user in the manner described above. Feedback obtained from a user following transmission of an instance of candidate content to the user can then be utilized to update the distribution over the possible values of the exploitation score.

It should be appreciated that the technologies presented herein are primarily described in terms of a configuration in which the effectiveness of a given instance of content is determined based on the likelihood of a user purchasing an item after the user is shown the content. However, any metric could be used to measure effectiveness or success rate of an instance of content (i.e. the exploitation score), and the technologies disclosed herein could consider data other than units or volume purchased, such as the number of times a user selected an item or other content, the total amount of money spent by the user after viewing an instance of content, the number of clicks made by a user, a user signing up or registering with a web site or other type of service, or another metric. Additional details regarding these and other aspects of the various technologies disclosed herein will be provided below with regard toFIGS. 1-4.

It should be appreciated that the configurations disclosed herein might be utilized with any type of computer, computing system, device, merchant or e-commerce site, application program, operating system, or other type of system or component. It should also be appreciated that the subject matter presented herein may be implemented as a computer process, a computer-controlled apparatus, a computing system, or an article of manufacture, such as a computer-readable storage medium. These and various other features will become apparent from a reading of the following disclosure and a review of the associated drawings.

While the subject matter described herein is presented in the general context of program modules that execute on one or more computing devices, those skilled in the art will recognize that other implementations may be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.

Those skilled in the art will appreciate that the subject matter described herein may be practiced on or in conjunction with other computer system configurations beyond those described below, including multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, handheld computers, personal digital assistants, tablet computers, electronic book readers, wireless telephone devices, special-purposed hardware devices, network appliances, or the like. The technologies described herein may also be practiced in distributed computing environments, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

In the following detailed description, references are made to the accompanying drawings that form a part hereof, and that show, by way of illustration, specific configurations or examples. The drawings herein are not drawn to scale. Like numerals represent like elements throughout the several figures.

FIG. 1is a system diagram illustrating various aspects of a mechanism disclosed herein for optimizing the selection of content for delivery to users in one particular configuration. In the illustrated configuration, a content planning server102is provided for selecting content to be transmitted to users. The content planning server102may include a content selection module104for selecting content to be transmitted to users, such as in email messages. The illustrated environment also includes an e-commerce system124that facilitates electronic browsing and/or purchasing of goods and services using various user devices, such as a user computing device120. Those skilled in the art will recognize that the user computing device120may be one of any number of computing devices that are capable of communicating over a network including, but not limited to, a laptop, personal computer, personal digital assistant (PDA), table or slate computing device, smartphone, feature phone, electronic book reader, digital media player, and the like.

The e-commerce system124may be connected to or in communication with an item data store126that stores information associated with items available for browse and/or purchase. Item data stored in the item data store126may include any information related to an item, such as an item available for purchase, that may be of interest to a user or may be useful for classifying or recommending an item. For example, item data may include, but is not limited to, price, availability, title, item identifier, item feedback (e.g., user reviews, ratings, etc.), item image, item description, item attributes, etc. While the item data store126is depicted inFIG. 1as being local to e-commerce system124, those skilled in the art will appreciate that the item data store126may be remote to the e-commerce system124, and/or may be a network-based service itself.

The content planning server102may be connected to or in communication with a feedback data store128. The feedback data store128may store information regarding purchases and/or other types of actions taken by a user following provision of content to a user. For example, and without limitation, the feedback data store128might store information associated with completed purchases, such as information identifying the items included in an order, a session identifier, information identifying the user or customer, shipping information, etc. The feedback data store128might also store information associated with users' browsing or viewing activities, such as information regarding the pages or files that a user viewed and the content selected by a user in a given session.

The content planning server102may additionally be connected to or in communication with a candidate content data store112that stores information associated with candidate content. For example, and without limitation, the candidate content data store112might store instances of candidate content106A-106N. As discussed above, the instances of candidate content106A-106N might be content that is suitable for transmission in email messages or other types of electronic communications. The instances of candidate content106A-106N might include different content advertising different products or services that is created by different operating units within an organization, for example. As will be described in greater detail below, exploitation scores108A-108N and exploration scores110A-110N might be computed for each of the instances of candidate content106A-106N, respectively, and utilized to select one of the instances of candidate content106A-106N for transmission to a particular user.

It should be appreciated that the configuration of data stores shown inFIG. 1is illustrative and that other configurations might be utilized. For example, and without limitation, the candidate content data store112, the item data store126and the feedback data store128might be local or remote to the content planning server102, might be local or remote to the e-commerce system124, and/or may be network-based services themselves.

In the environment shown inFIG. 1, the user computing device120may communicate with the e-commerce system124via a communication network118, such as the Internet or other communications link. Those skilled in the art will appreciate that the network118may be any wired network, wireless network or combination thereof. In addition, the network118may be a personal area network, local area network, wide area network, cable network, satellite network, cellular telephone network, etc. or combination thereof. Protocols and components for communicating via the Internet or any of the other aforementioned types of communication networks are well known to those skilled in the art of computer communications and, thus, need not be described in more detail herein.

The system depicted inFIG. 1operates in a distributed computer environment comprising several computer systems that are interconnected using one or more computer networks. However, it will be appreciated by those skilled in the art that the disclosed system could equally operate within a computer system having a fewer or greater number of components than are illustrated inFIG. 1. Thus, the depiction of system inFIG. 1should be taken as illustrative and not limiting to the present disclosure. For example, the system could implement various web services components and/or peer-to-peer network configurations to implement at least a portion of the processes.

In brief, the e-commerce system124is generally responsible for providing front-end communication with various user devices, such as user computing device120, via network108. The front-end communication provided by the e-commerce system124may include generating text and/or graphics, possibly organized as a user interface using hypertext transfer (“HTTP”) or other protocols in response to information inquiries received from the various user devices.

The e-commerce system124may obtain information on available goods and services (which might be referred to herein as “items”) from item data store126, as is done in conventional electronic commerce systems. In one configuration, the item data store126includes information on items available from a plurality of sellers (as opposed to storing information for only a single vendor). In certain configurations, the e-commerce system124may also access item data from other data sources, either internal or external to the e-commerce system124. Accordingly, the e-commerce system124may obtain item information for items offered for sale by a plurality of sellers. A user may then purchase items from a plurality of sellers in a single transaction or order placed with the e-commerce system124. In other configurations, the user may purchase items from a single vendor in a single transaction or order placed with the e-commerce system124.

As described below in reference toFIG. 4, the content planning server102may include a processing unit, a memory unit, an optional display, and a network interface, all of which may communicate with one another by way of a communication bus. The e-commerce system124may be similarly arranged. A network interface included in the content planning server102is preferably connected to one or more computer networks that connect to other components. A processing unit may thus receive information and instructions from other computing components, such as the e-commerce system124, item data store126, and/or the feedback data store128via the network interface.

As shown inFIG. 1, and described briefly above, the content planning server102is configured with a content selection module104in one particular configuration. The content selection module104is configured to select instances of content106for transmission to users, such as customers or potential customers of an e-commerce web site. The selected instance of content106might then be transmitted to users through various mechanisms such as, but not limited to, email messages, SMS messages, instant messages, and/or other types of electronic messaging systems. For example, and without limitation, in the example shown inFIG. 1, a message114with selected content116is transmitted to the user computing device120via the network110. The selected content116might also be provided in a web page or another type of content page. The selected content116might also be provided to users in other ways in other configurations.

As also described briefly above, the various instances of content106A-106N might be authored by different operating units within an organization. For example, one operating unit might desire to send an instance of content106A to customers or potential customers advertising one product or service, while another operating unit desires to send an instance of content106B regarding another product or service. As additionally discussed above, various constraints might also be placed on the number of messages transmitted to a user in a particular time period, such as one message per day. In this scenario, two or more instances of candidate content106might first be identified for a particular user based upon various attributes or context associated with the user. For example, the candidate instances of content106for the particular user might be identified based upon the user's age, hobbies, previous purchase history, geographic location, and/or other attributes.

In order to select one of the instances of candidate content106for transmission to a particular user, an exploitation score108might be computed for each of the instances of candidate content106. The exploitation score108for an instance of candidate content106defines an estimate of the feedback expected from the user following the delivery of the instance of content106to the user. The exploitation score108might be computed based upon historical feedback received following transmission of an instance of content106to one or more users. For example, and without limitation, the exploitation score108for a particular instance of content106might indicate the percentage of users to which the instance of content106has been previously transmitted that subsequently made a purchase or took another desired action based on the instance of candidate content106.

An exploration score110might also be computed for each of the instances of candidate content106. The exploration score100for a particular instance of candidate content106is computed by a function in one particular configuration that varies inversely to the number of times that the instance of content106has been transmitted to users. In this way, the exploration score110will be very high when the instance of candidate content106has not yet been transmitted to users or has only been transmitted to users a relatively low number of times. The exploration score110for an instance of candidate content106will decrease to zero as the content106is transmitted to a larger number of users. In this way, a ranking score associated with each instance of candidate content106can be artificially inflated while the effectiveness of the content106is explored.

In one configuration, a ranking score is also computed for each of the instances of candidate content106. For example, and without limitation, the ranking score might be computed by summing the exploitation score108and the exploration score110for a particular instance of candidate content106. Other types of mathematical operations might be utilized on the exploitation score108and the exploration score110in order to compute the ranking score for a particular instance of candidate content106. The instances of candidate content106might then be ranked based upon their associated ranking scores.

Once the instances of candidate content106for a particular user have been ranked, one of the instances of candidate content106may be selected for transmission to a user. For example, and without limitation, the highest ranked instance of candidate content106might be selected for transmission to a user. Other mechanisms might also be utilized to select a particular instance of candidate content106for transmission to a user from the ranked instances of candidate content106. The selected instance of candidate content106(i.e. the selected content116inFIG. 1) may then be transmitted to the user computing device120via the network118utilizing one or more of the mechanisms described above. Additional details regarding the selection of an instance of content106for transmission to a user will be provided below with regard toFIG. 2A.

It should be appreciated that, in some configurations, more than one instance of candidate content106might be selected for transmission to a user. For example, and without limitation, the top N instances of candidate content106might be selected from the ranked instances of candidate content. In another configuration, a function may be maximized over the N instances of candidate content106in the ranking. Other mechanisms might also be utilized to select multiple instances of candidate content106for transmission to a user.

Following transmission of the selected content116to the user, feedback122might be obtained from the user based upon the content transmitted to the user. For example, and without limitation, positive feedback122might be considered to have been received from the user if the user makes a purchase from the e-commerce system124based on the content. Negative feedback122might also be received from the user. For example, and without limitation, the user might request to unsubscribe from any further content from the content planning server102or might request that fewer messages be transmitted. In this regard, it should be appreciated that negative feedback might be explicit, such as in the case where the user requests to unsubscribe from any further content, or implicit, such as in the case where the user does not take the desired action (e.g. the user does not make a purchase or visit an e-commerce web site within some period of time following transmission of a message114).

In either case, the exploitation score108for the content transmitted to the user is updated based upon the obtained feedback122in some configurations. If the obtained feedback122is negative, the exploitation score108for the transmitted content might be penalized dynamically based on one or more attributes. For example, and without limitation, the exploitation score108for the transmitted content might be penalized in an amount that varies based upon the volume of purchases previously made by the user. Other attributes might also be utilized to dynamically penalize the exploitation score. Additional details regarding the process of dynamically penalizing the exploitation scores108will be described below with regard toFIG. 3.

Turning now toFIG. 2A, additional details will be provided regarding the operation of the content selection module104for selecting instances of content for transmission to users. In particular,FIG. 2Ais a flow diagram showing one illustrative routine200that illustrates aspects of the operation of the content selection module104for selecting content106for delivery to users based on an exploitation score108and an exploration score110.

The routine200begins at operation202, where two or more instances of candidate content106are identified as candidates for transmission to a particular user based upon various attributes or context associated with the user. As discussed above, for example, the candidate instances of content106for the particular user might be identified based upon the user's age, hobbies, previous purchase history, geographic location, and/or other attributes.

From operation202, the routine200proceeds to operation204, where an exploitation score108is computed for each of the instances of candidate content106for the user. As discussed above, the exploitation score108for an instance of candidate content106defines an estimate of the feedback expected from the user following the delivery of the instance of content106to the user. The routine200then proceeds from operation204to operation206.

At operation206, an exploration score110is computed for each of the instances of candidate content106. As also discussed above, the exploration score100for a particular instance of candidate content106is computed by a function in one particular configuration that varies inversely to the number of times that the instance of content106has been transmitted to users. Once the exploration scores110have been computed, the routine200proceeds from operation206to operation208.

At operation208, a ranking score computed for each of the instances of candidate content106. For example, and without limitation, the ranking score might be computed by summing the exploitation score108and the exploration score110for a particular instance of candidate content106. Other types of mathematical operations might be utilized on the exploitation score108and the exploration score110in order to compute the ranking score for a particular instance of candidate content106. The instances of candidate content106are then ranked based upon their associated ranking scores at operation210.

At operation212, one or more of the instances of candidate content106may be selected for transmission to the user based upon the ranking performed at operation210. For example, and without limitation, the highest ranked instance, or N instances, of candidate content106might be selected for transmission to a user. Other mechanisms might also be utilized to select one or more instances of candidate content106for transmission to a user from the ranked instances of candidate content106. The selected instance, or instances, of candidate content106(i.e. the selected content116inFIG. 1) is then transmitted to the user computing device120via the network118utilizing one or more of the mechanisms described above at operation214. At operation216, the selected content116that was transmitted to the user at operation214may be removed from the set of candidate content106. In this way, the same content106will not be transmitted to the user twice.

From operation216, the routine200proceeds to operation218, where feedback122may be received from the user and stored in the feedback data store128. For example, and without limitation, positive feedback122might be considered to have been received from the user if the user makes a purchase from the e-commerce system124based on the content. Implicit or explicit negative feedback122might also be received from the user. For example, and without limitation, the user might request to unsubscribe from any further content from the content planning server102.

In either case, the exploitation score108for the content transmitted to the user is updated based upon the obtained feedback122in some configurations. If the obtained feedback122is negative, the exploitation score108for the transmitted content might be penalized dynamically based on one or more attributes. For example, and without limitation, the exploitation score108for the transmitted content might be penalized in an amount that varies based upon the volume of purchases previously made by the user. Other attributes might also be utilized to dynamically penalize the exploitation score. Additional details regarding the process of dynamically penalizing the exploitation scores108will be described below with regard toFIG. 3. From operation218, the routine200proceeds back to operation202, where the process described above might be repeated, for instance after a certain period of time (e.g. 24 hours) has elapsed since the previous iteration of the routine200.

FIG. 2Bis a flow diagram showing a routine250that illustrates aspects of the operation of the content selection module104in which one or more instances of content106are selected for transmission to a user by sampling a distribution of possible values of the exploitation score108. The routine250begins at operation252, where two or more instances of candidate content106are identified as candidates for transmission to a particular user based upon various attributes or context associated with the user. As discussed above, for example, the candidate instances of content106for the particular user might be identified based upon the user's age, hobbies, previous purchase history, geographic location, and/or other attributes.

From operation252, the routine250proceeds to operation254, where the content selection module104computes a distribution over the possible values of the exploitation score for the instances of candidate content106identified at operation252. The routine250then proceeds from operation254to operation256, where the content selection module104samples from the distributions computed at operation254to obtain a ranking score for each of the instances of candidate content identified at operation252. in one particular implementation, Thompson Sampling is utilized to sample from the distributions. Other sampling methodologies might also be utilized in other configurations.

The routine250then proceeds from operation256to operation258, where the instances of candidate content106are ranked based upon their associated ranking scores. The routine250then proceeds to operation260, where one or more of the instances of candidate content106may be selected for transmission to the user based upon the ranking performed at operation258. For example, and without limitation, the highest ranked instance of candidate content106might be selected for transmission to a user. Other mechanisms might also be utilized to select one or more instances of candidate content106for transmission to a user from the ranked instances of candidate content106. The selected instance of candidate content106(i.e. the selected content116inFIG. 1) is then transmitted to the user computing device120via the network118utilizing one or more of the mechanisms described above at operation262. At operation264, the selected content116that was transmitted to the user at operation262may be removed from the set of candidate content106. In this way, the same content106will not be transmitted to the user twice.

From operation264, the routine200proceeds to operation266, where feedback122may be received from the user and stored in the feedback data store128as discussed above. The distribution over the possible values of the exploitation score108for the content transmitted to the user is then updated based upon the obtained feedback122in some configurations. From operation266, the routine200proceeds back to operation252, where the process described above might be repeated, for instance after a certain period of time (e.g. 24 hours) has elapsed since the previous iteration of the routine250.

FIG. 3is a flow diagram showing an illustrative routine300that illustrates aspects of one mechanism for dynamically penalizing an exploitation score108for content106in response to receiving negative feedback for the content106. The routine300begins at operation302, where feedback122is collected in the manner described above with regard to operation218of the routine200. The routine300then proceeds from operation302to operation304, where a determination is made as to whether the collected feedback122is positive or negative. If the feedback122is positive, the routine300proceeds from operation304to operation308, where the exploitation score for the content106is updated in the manner described above.

If, however, the received feedback122is negative, the routine300proceeds from operation304to operation306. At operation306, the exploitation score108for the transmitted content is penalized dynamically based on one or more attributes. For example, and without limitation, the exploitation score108for the transmitted content might be penalized in an amount that varies based upon the volume of purchases previously made by the user. Other attributes might also be utilized to dynamically penalize the exploitation score. From operation306, the routine300proceeds back to operation302, where additional feedback122might be received and processed in the manner described above.

FIG. 4shows an example computer architecture for a computer400capable of executing the software components described herein for optimized selection of content for delivery to a user. The computer architecture shown inFIG. 4illustrates a conventional server computer, workstation, desktop computer, laptop, electronic book reader, digital wireless phone, tablet computer, network appliance, set-top box, or other computing device. The computer architecture shown inFIG. 4may be utilized to implement a computer that is capable of executing the various software components described above.

The CPUs402perform operations by transitioning from one discrete, physical state to the next through the manipulation of switching elements that differentiate between and change these states. Switching elements may generally include electronic circuits that maintain one of two binary states, such as flip-flops, and electronic circuits that provide an output state based on the logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements may be combined to create more complex logic circuits, including registers, adders-subtractors, arithmetic logic units, floating-point units, or the like.

The chipset404provides an interface between the CPUs402and the remainder of the components and devices on the baseboard. The chipset404may provide an interface to a random access memory (“RAM”)406, used as the main memory in the computer400. The chipset404may further provide an interface to a computer-readable storage medium such as a read-only memory (“ROM”)408or non-volatile RAM (“NVRAM”) for storing basic routines that that help to startup the computer400and to transfer information between the various components and devices. The ROM408or NVRAM may also store other software components necessary for the operation of the computer400in accordance with the configurations described herein.

In one configuration, the computer400operates in a networked environment using logical connections to remote computing devices and computer systems through a network, such as a local-area network (“LAN”), a wide-area network (“WAN”), the Internet, or any other networking topology known in the art that connects the computer400to remote computers. The chipset404includes functionality for providing network connectivity through a network interface controller (“NIC”)410, such as a gigabit Ethernet adapter. It should be appreciated that multiple NICs410may be present in the computer400, connecting the computer400to other types of networks and remote computer systems.

The computer400may be connected to a mass storage device412that provides non-volatile storage for the computer. The mass storage device412may store system programs, application programs, other program modules, and data, which have been described in greater detail herein. The mass storage device412may be connected to the computer400through a storage controller414connected to the chipset404. The mass storage device412may consist of one or more physical storage units. The storage controller414may interface with the physical storage units through a serial attached SCSI (“SAS”) interface, a serial advanced technology attachment (“SATA”) interface, a FIBRE CHANNEL (“FC”) interface, or other standard interface for physically connecting and transferring data between computers and physical storage devices.

The computer400may store data on the mass storage device412by transforming the physical state of the physical storage units to reflect the information being stored. The specific transformation of physical state may depend on various factors, in different implementations of this description. Examples of such factors may include, but are not limited to, the technology used to implement the physical storage units, whether the mass storage device412is characterized as primary or secondary storage, or the like.

For example, the computer400may store information to the mass storage device412by issuing instructions through the storage controller414to alter the magnetic characteristics of a particular location within a magnetic disk drive unit, the reflective or refractive characteristics of a particular location in an optical storage unit, or the electrical characteristics of a particular capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this description. The computer400may further read information from the mass storage device412by detecting the physical states or characteristics of one or more particular locations within the physical storage units.

In addition to the mass storage device412described above, the computer400might have access to other computer-readable media to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable media can be any available media that may be accessed by the computer400, including computer-readable storage media and communications media. Communications media includes transitory signals. Computer-readable storage media includes volatile and non-volatile, removable and non-removable storage media implemented in any method or technology. For example, computer-readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically-erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technology, compact disc ROM (“CD-ROM”), digital versatile disk (“DVD”), high definition DVD (“HD-DVD”), BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information. Computer-readable storage media does not include transitory signals.

The mass storage device412may store an operating system416utilized to control the operation of the computer400. In one configuration, the operating system comprises the LINUX operating system. In another configuration, the operating system comprises the WINDOWS® SERVER operating system from MICROSOFT Corporation of Redmond, Wash. In yet another configuration, the operating system may comprise the UNIX operating system. It should be appreciated that other operating systems may also be utilized. The mass storage device412may store other system or application programs and data utilized by the computer400. For instance, the mass storage device may store the content selection module104, which was described in detail above.

In one configuration, the mass storage device412or other computer-readable storage media is encoded with computer-executable instructions which, when loaded into the computer400, transform the computer from a general-purpose computing system into a special-purpose computer capable of implementing the technologies described herein. These computer-executable instructions transform the computer400by specifying how the CPUs402transition between states, as described above. According to one configuration, the computer400has access to computer-readable storage media storing computer-executable instructions that, when executed by the computer, perform the various routines and operations described herein.

The computer400may also include an input/output controller418for receiving and processing input from a number of input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus, or other type of input device. Similarly, the input/output controller418may provide output to a display device, such as a computer monitor, a flat-panel display, a digital projector, a printer, a plotter, or other type of output device. It will be appreciated that the computer400may not include all of the components shown inFIG. 4, may include other components that are not explicitly shown inFIG. 4, or may utilize an architecture completely different than that shown inFIG. 4.

The subject matter described above is provided by way of illustration only and should not be construed as limiting. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. Various modifications and changes may be made to the subject matter described herein without following the example configurations and applications illustrated and described, and without departing from the true spirit and scope of the present invention, which is set forth in the following claims.