Audience size estimation and complex segment logic

Selection of a trait may be received. A complex segment rule may be created that is usable to evaluate one or more qualification events. For example, the segment rule may be usable to evaluate a combined recency and frequency of the one or more qualification events. The qualification events may be based on collected network data associated with the plurality of visitors with each qualification event corresponding to a separate qualification of visitor according to the trait. The qualification events may be evaluated together according to the segment rule. For example, the combined recency and frequency of the one or more qualification events may be evaluated according to the segment rule. Evaluating the segment rule may include estimating a segment population size in real-time.

BACKGROUND

Goods and services providers often employ various forms of marketing to drive consumer demand for products and services. Marketing includes various techniques to expose to target audiences to brands, products, services, and so forth. For example, marketing often includes providing promotions (e.g., advertisements) to an audience to encourage them to purchase a product or service. In some instances, promotions are provided through media outlets, such as television, radio, and the internet via television commercials, radio commercials and webpage advertisements. In the context of websites, marketing may include advertisements for a website and products associated with that website to encourage persons to visit and/or use the website, purchase products and services offered via the website, and/or otherwise interact with the website.

In marketing or other applications, data may be managed. User data management is the collection and analysis of user website interaction data. It includes collecting information about how individual users interact on website.

SUMMARY

Methods and apparatus for real-time audience size estimation and implementing complex segment logic are disclosed. Selection of a trait may be received with the trait being descriptive of visitors of network content (e.g., behavior exhibited, information disclosed while on the network content, etc.). A complex segment rule may be created that is usable to evaluate one or more qualification events. For example, the segment rule may be usable to evaluate a combined recency and frequency of the one or more qualification events. The qualification events may be based on collected network data associated with the plurality of website interactions. Each of the qualification events may correspond to a separate qualification of one of the visitors according to the trait. In one embodiment, the qualification events may be evaluated together according to the segment rule. For example, the combined recency and frequency of the one or more qualification events may be evaluated according to the segment rule. In one embodiment, evaluating the segment rule may include estimating a segment population size in real-time.

While the disclosure is described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that the disclosure is not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit the disclosure to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present disclosure. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include”, “including”, and “includes” mean including, but not limited to.

DETAILED DESCRIPTION OF EMBODIMENTS

“First,” “Second,” etc. As used herein, these terms are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, for a segment analysis module evaluating the qualification of a visitor according to a segment rule that includes first and second traits, the terms “first” and “second” traits can be used to refer to any two traits that are part of the segment rule. In other words, the “first” and “second” advertisements are not limited to logical traits 0 and 1.

INTRODUCTION

This specification presents an illustrative segment analysis and online advertising system, as well as multiple illustrative segment analysis modules that may implement one or more of the disclosed real-time and back end (also referred to as processed) segment qualification techniques as well as audience size estimation and/or complex segment logic techniques. The specification then discloses techniques for performing real-time and back end segment qualification, audience size estimation, and/or implementing complex segment logic. Various examples and applications are disclosed. Some of these techniques may be implemented, for example, by a segment analysis module or computer system.

While much of the specification is described in terms of online marketing, similar techniques may apply in customer segmentation, content customization, and/or variable pricing of products.

Some embodiments may include a means for real-time and back end segment qualification and/or a means for audience size estimation and/or a means for implementing complex segment logic shared between back end and real-time systems. For example, a segment analysis module (e.g., segment analysis module120/218/313ofFIGS. 1/2/3) may determine that a visitor qualifies in a segment (by a real-time component, back end component, or both) and, in one embodiment, store an indication of the visitor's segment qualification in the real-time component, as described herein. As another example, the segment analysis module may receive selection of one or more traits or pixel qualification, create a segment rule that is usable to evaluate a combined recency and frequency of one or more qualification events, and evaluate the combined recency and frequency of the qualification events according to the segment rule. The segment analysis module may in some embodiments be implemented by a non-transitory, computer-readable storage medium and one or more processors (e.g., CPUs and/or GPUs) of a computing apparatus. The computer-readable storage medium may store program instructions executable by the one or more processors to cause the computing apparatus to perform determining that a visitor qualifies in a segment (by a real-time component, back end component, or both) and storing an indication of the visitor's segment qualification in the real-time component, as described herein. The computer-readable storage medium may store program instructions executable by the one or more processors to cause the computing apparatus to perform receiving selection of a trait, creating a segment rule that is usable to evaluate a combined recency and frequency of one or more qualification events, and evaluating the combined recency and frequency of the qualification events according to the segment rule. Other embodiments of the segment analysis module may be at least partially implemented by hardware circuitry and/or firmware stored, for example, in a non-volatile memory.

In some embodiments, these techniques may be used in measuring the size of a segment audience, and in qualifying visitors in a segment. The segment audience size and visitor qualifications can then be used in an advertising campaign, email campaign, content customization, or for analytics purposes, among other example applications. Although certain embodiments and applications are described in terms of segment qualification, audience size estimation, and online advertising, it should be noted that the same or similar principles may be applied in other fields.

Moreover, although certain embodiments are described with respect to a webpage and/or website, it will be appreciated that the techniques disclosed herein may be employed with other forms of network content sites, such as documents with a traversable tree-like hierarchy (e.g., XML, HTML, etc.).

Turning now to the figures,FIG. 1illustrates an example segment analysis system100configured to support real-time and back end segment qualification, audience size estimation (e.g., using combined recency and frequency), and/or complex segment logic, in accordance with one or more embodiments. In the illustrated embodiment, segment analysis system100may include three primary entities/users of the system: advertisers/agencies102, publishers/customers112, and end consumers118. End consumers118may also referred to as visitors, such as visitors of a website or other network content.

Advertisers/agencies102may be the buyers of ad impression opportunities for online advertising. They may also be the originators of media buys, campaigns, and creatives. In the example shown, advertisers/agencies102may be communicatively coupled (e.g., over a network) to demand side platform (DSP)104and data management platform (DMP)108. Coupling to DMP108may be via advertising website106. In one embodiment, advertisers/agencies102may provide product content to advertising website106, where the product content may be presented for display and/or to generate a display. Advertiser website106may be network content owned by or otherwise associated with advertisers/agencies. Advertiser website106may collect data regarding visitors to the website and provide such data to DMP108.

DMP108may include segment analysis module120. Segment analysis module120may collect, aggregate, store, combines, and/or provide insights on audience behavioral and demographic data for advertiser(s), publisher(s), and agency customers. For example, segment analysis module120may determine real-time and back end segment qualification, perform audience size estimation (e.g., using combined recency and frequency), and implement complex segment logic. DMP108may allow customers to easily ingest data from multiple online and/or offline data sources. As shown, DMP108may receive offline audience data (also referred to as network content data) from advertisers/agencies102and/or publishers112. Additionally, audience data may be collected at advertising website106and delivered to DMP108. In some embodiments, DMP108may additionally or alternatively collect audience data from publisher website114. Not shown inFIG. 1, but as described herein, DMP108may also receive data from a third party data provider. DMP108may provide audience segment information (e.g., information indicative of segment qualification) to one or more of DSP104, supply side platform (SSP)110, and onsite ad server116.

DSP104may be an advertising campaign management application which allows advertisers to manage their campaign/creative bidding rules, use audience data at scale and/or bid on available display advertising inventory. In one embodiment, advertisers/agencies102may provide ad campaigns, creative, and bid rules to DSP104. In some embodiments, DSP104may be integrated with SSP110and perform ad exchanges via real-time bidding (RTB) server-to-server pipes. As shown, DSP104and SSP110are separately illustrated. In various embodiments, DSP104may receive RTB advertisements from SSP110.

In one embodiment, SSP110is a platform that aggregates publisher advertisement inventory supply and allows publishers to leverage audience data for revenue optimization. As noted above, SSP110may be integrated with DSP104via RTB interfaces. In some embodiments, SSP110may receive a get ad request from publisher website114and provide an advertisement, or facilitate providing of an advertisement, to publisher website114to present to end consumer/visitor118.

Onsite ad server116may manage guaranteed advertising buys and advertisement insertions onto publisher pages. Examples include: DoubleClick for Publishers (DFP) and Open Ad Stream (OAS). Onsite ad server116may receive a get ad request from publisher website114and provide an advertisement, or facilitate providing of an advertisement, to publisher website114to present to end consumer/visitor118.

Publisher(s)112may include supplier(s) of advertisement inventory (e.g., available ad slots on pages). As described herein, publisher(s)112may send audience data in real-time to DMP108as end consumers/visitors browse publisher website114. As noted above, publisher(s)112may also provide offline audience data to DMP108. Such offline audience data may be provided in bulk feed form, in one embodiment.

Publisher website114may be network content, such as an automobile shopping website, a webmail website, etc. Publisher website114may be executable by a client device of end consumer/visitor118. For example, in some embodiments, the client device may include an application (e.g., internet web-browser application) that can be used to generate a request for content, to render content, and/or to communicate requests to various devices on the network. For example, upon selection of a website link on a webpage displayed to consumer/visitor118or upon receiving a URL by the browser application, the browser application may submit a request for the corresponding webpage/content to a content server (not shown), and content server may provide corresponding content, including an HTML file, that is executed by the browser application to render the requested website for display to the user. In some instances, execution of the HTML file may cause the corresponding webpage/content, such as publisher website114to provide real-time audience data regarding the web browsing of the visitor to segment analysis module120. Note that while a webpage is discussed as an example of the network content available for use with the embodiments described herein, one of skill in the art will readily realize that other forms of content, such as audio, image, or video files, may be used without departing from the scope and content herein disclosed. Likewise, while references herein to HTML and the HTTP protocol are discussed as an example of the languages and protocols available for use with the embodiments described herein, one of skill in the art will readily realize that other forms of languages and protocols, such as XML or FTP may be used without departing from the scope and content herein disclosed.

Each of DSP104, advertiser website106, DMP108, SSP110, onsite ad server116, and publisher website114may be communicatively coupled to one another via a network (not shown). The network may include any channel for providing communication between each of the entities of system100. In some embodiments, the network includes an electronic communication network, such as the internet, a local area network (LAN), a cellular communications network, or the like. The network may include a single network or combination of networks that facilitate communication between each of the entities (e.g., advertiser website106, DMP108, SSP110, onsite ad server116, and publisher website114) of system100. In some embodiments, various components of system100may be collocated (e.g., DSP104and SSP110) as part of the same computing device or devices. In one embodiment, one or more of the components of system100may be geographically diverse (e.g., located on different computing devices connected over the network). In some embodiments, various components of system100may be cloud based.

Note that in various embodiments, some components of system100(e.g., DMP108, DSP104, SSP110) may be implemented on multiple computing devices. For example, in one embodiment, DMP108may be implemented as a distributed system on a number of cloud nodes and on a number of servers.

While segment analysis module120is shown inFIG. 1as a component of DMP108, one of skill in the art will readily realize, in light of having read the present disclosure, that segment analysis module120may be embodied in a separate system with access to data received by DMP108, such as offline and/or real-time data, via the network.

Audience data, whether real-time or offline audience data (e.g., first party or from a third party data provider), may include a variety of information, such as cookies, hits, page views, visits, sessions, downloads, first visits, first sessions, visitors, unique visitors, unique users, repeat visitors, new visitors, impressions, singletons, bounce rates, exit percentages, visibility time, session duration, page view duration, time on page, active time, engagement time, page depth, page views per session, frequency, session per unique, click path, click, site overlay, behavioral traits, user intents, user interests, demographic data, etc. The various data may describe usage and visitation patterns for websites (e.g., publisher website114, advertiser website106, etc.) and/or individual webpages within the website. The various data may include information relating to the activity and interactions of one or more users/visitors with a given website or webpage. For example, audience data may include historic and/or current website browsing information for one or more website visitors, including, but not limited to identification of links selected, identification of web pages viewed, related content topics, and other data that may help gauge user interactions with webpages/websites.

In some embodiments, audience data may include information indicative of a location. For example, audience data may include location data indicative of a geographic location of the client device of consumer/visitor118. In some embodiments, location data may be correlated with corresponding user activity. In some embodiments, location data includes geographic location information. For example, location data may include an indication of the geographic coordinates (e.g., latitude and longitude coordinates), IP address or the like or a user or a device. In some embodiments, audience data may include demographic information indicative of the user.

In some embodiments, audience data is accumulated over time to generate a set of audience data (e.g., offline audience data) that is representative of activity and interactions of one or more users with a given website or webpage. In one embodiment, such an accumulation may be performed by various publishers/customers with respect to audience data generated through websites/network content that the publishers/customers own. In one embodiment, such an accumulation may be performed by third party data providers.

The data (e.g., real-time and/or offline) may be used to qualify visitors in a segment, according to the disclosed techniques, and/or to perform real-time audience size estimation for a segment. Such a qualification of a visitor may be usable to select an advertisement to display to consumer/visitor118. Audience size estimation may be usable to evaluate a worth of a segment. For example, if the segment is sufficiently large, then it may be worth commencing a new advertising campaign or continuing an existing one. Conversely, if the segment is small, then it may not be worth commencing a new advertising campaign directed toward that segment or it may be determined that a new outreach campaign should be commenced to increase the size of the segment.

In some embodiments, segment analysis module120may include computer executable code (e.g., executable software modules) stored on a computer readable storage medium that is executable by a computer to provide associated processing. For example, segment analysis module120may process real-time and/or offline audience data to perform the techniques described herein.

In some embodiments, a customer/publisher/content provider112may log-in to a website or some other user interface/network portal, for example, hosted by DMP108, and may interact with audience data to create, modify, and/or apply a segment rule. The segment rule may then be usable to determine an audience size and/or qualify visitors in real-time or as an offline back end qualification. Such determinations may be usable to select and place an online advertisement for one or more visitors.

FIG. 2depicts a module that may implement real-time segment processing and back end segment processing, according to some embodiments. Segment analysis module218(which may be the same module as segment analysis module120ofFIG. 1) may, for example, implement one or more of the techniques described herein atFIGS. 4-5. For simplicity of explanation, the segment analysis module ofFIG. 2is described as implementing the techniques ofFIG. 4and the segment analysis module ofFIG. 3is described as implementing the techniques ofFIG. 5. Note that the segment analysis modules ofFIGS. 1-3may refer to the same segment analysis module and may implement any or all of the disclosed techniques.

Segment analysis module218may receive, via UI/API portal220from publisher/customer202, input that defines a segment rule. Such input may be provided via touchscreen, mouse, keyboard, or other suitable device. As described herein, the input may include one or more traits, Boolean operator(s), recency/frequency requirements, and/or destination rules, among other inputs. A trait may correspond to a single data collection event from received audience data that is descriptive of a visitor of network content. Example traits may include male, expensive camera, high end car, laptop, etc. For example, the expensive camera trait may correspond to data collection events indicative of browsing for a camera over $500. Based on the received input, a segment rule may be created. The inputs and/or segment rule may persist in customer/control data database222. Customer/control data database222may provide customer segment and destination rules as an asynchronous feed to data collection server(s)226of edge servers230and segment rule processor224. Segment rule processor224may interface with both real-time component228and back end component232such that the same segment rule may be used by either or both components, as described herein. In the illustrated embodiment, segment rule processor224may provide URL destinations, based on segments determined by back end component232, to real-time component228. Data collection server(s)226may receive and send destinations from and to real-time component228. Data collection server(s)226may also send an asynchronous data aggregation feed of traits and real-time determined segments to back end component232for storage in a data store of back end component232. Data collection server(s)226may also receive first party data via HTTP data collection requests from client browsers210and212, which may be executed on client devices of consumers/visitors206. Not shown, segment analysis module218may also receive third party data (e.g., from data partners324ofFIG. 3). Third party data is illustrated inFIG. 3but applies equally to the illustration ofFIG. 2. Note that other components not shown in both ofFIGS. 2 and 3may exist in both figures for simplicity of drawing and explanation, are omitted.

Segment analysis module218may then perform the techniques described herein atFIGS. 4-5on the received data (e.g., first party from the customer's website and/or third party data) based on the segment rule. For example, either or both of real-time component228and back end component232may determine that a visitor qualifies in a segment according to the same segment rule, as described herein. Segment analysis module218may generate, as output, a destination call to network content (e.g., customer B's website in the illustrated example) such that an HTTP destination URL call may be requested from destination-data collection server216of destination servers214. Segment analysis module218may also generate, as output, a report regarding segment audience size, qualifications, or other reports, which may be displayable via UI/API portal220and/or stored to a storage medium (not shown), such as system memory, a disk drive, DVD, CD, etc.

FIG. 3depicts a module that may implement audience size estimation (e.g., real-time) using combined recency and frequency, according to some embodiments. Segment analysis module312may, for example, implement one or more of the techniques described herein atFIGS. 4-5. As noted above, for simplicity of explanation, the segment analysis module ofFIG. 3is described as implementing the techniques ofFIG. 5. Note that the segment analysis modules ofFIGS. 1-3may refer to the same segment analysis module and may implement any or all of the disclosed techniques.

Segment analysis module312may receive input, via UI/API portal314, from publisher/customer302. Such input may be provided via touchscreen, mouse, keyboard, or other suitable device. As described herein, the input may include one or more traits, Boolean operator(s), recency/frequency requirements, and/or destination rules, time ranges, among other inputs. Based on such input, a segment rule may be created and/or an audience may be defined. In an audience size determination application, a synchronous query for audience size may be sent from UI/API portal314to audience size estimator316and a real-time determination of the audience size may be provided from audience size estimator316to UI/API Portal314. A similar synchronous query and result may also take place between audience size estimator316and audience indexes/database218.

Back end component322may receive, via an asynchronous data aggregation feed from data collection servers320, first party data via HTTP data collection requests from client browser310, which may be executed on a client device of consumers/visitors306. Back end component322may also receive third party data from data provider servers326of data partners324. Third party data may be received as a bulk asynchronous feed. Back end component322may then provide visitor/trait data from the first and third party data in an asynchronous feed to audience indexes/database318, upon which the data may be used to determine audience size, as described herein.

Segment analysis module312may then perform the techniques described herein atFIGS. 4-5on the received data (e.g., first party from the customer's website and/or third party data) based on the segment rule. For example, real-time audience size estimation may be performed according to the segment rule, which may include combined recency and frequency, as described herein. Segment analysis module312may generate, as output, a destination call to network content such that an HTTP destination URL call may be requested. Segment analysis module312may also generate, as output, a report regarding segment audience size, qualifications, or other reports, which may be displayable via UI/API portal314and/or stored to a storage medium (not shown), such as system memory, a disk drive, DVD, CD, etc. Examples of such reports are shown inFIGS. 14-18.

Turning now toFIG. 4, one embodiment of real-time and back end segment qualification is illustrated. While the blocks are shown in a particular order for ease of understanding, other orders may be used. In some embodiments, the method ofFIG. 4may include additional (or fewer) blocks than shown. Blocks400-420may be performed automatically or may receive user input. In one embodiment, the segment analysis module ofFIGS. 1-3may implement the method ofFIG. 4.

As shown at400, it may be determined that a visitor qualifies in a segment according to a segment rule. Determining that the visitor qualifies in the segment may include evaluating network content data. Network content data may include audience data, as described herein, such as real-time audience data and/or offline audience data. In some embodiments, at least some of the network content data may be received from a third party data provider. For example, various network content sites may provide data about various visitors to a third party data provider who may aggregate that data and provide it to the segment analysis module.

In one embodiment, at least some of the network content data that is received from the third party data provider may be matched with at least some other of the network content data that is associated with the visitor's visit to the first network content. The result of the matching may be matched data. Determining that the visitor qualifies in the segment may be based on the matched data. As an example of matched data, a visitor may have logged into network content under a profile associated with the visitor (e.g., logged into a social networking website, shopping website, forum, etc.) or may have entered data into an online form. Data created during the visit and logging in under a profile or entering information into an online forum may include demographic information (e.g., name, age, gender, occupation, etc. Data created during that visit may also include a visitor identifier (e.g., IP address, MAC address, etc.) associated with the visitor. The data from the third party provider may also include the same visitor identifier thereby allowing data having the same visitor identifier to be matched. As an example, it may be known that visitor A is male because he logged in under his profile to a social networking site. Third party provider may have data from months ago that includes the visitor's IP address that matches the visitor's IP address from where the visitor logged in to the social networking website. Therefore, the data from the third party provider may then be determined as corresponding to a male visitor.

In various embodiments, the same segment rule may be used in both the real-time component and the back end component. For example, the segment rule may be usable by the real-time component, during the visitor's visit to first network content (e.g., the website of a first customer), to determine that the visitor qualifies in the segment. The segment rule may also be usable by the back end component to determine that the visitor qualifies in the segment. In some embodiments, determining that the visitor qualifies in the segment may be performed by the real-time component, back end component, or both. In one embodiment, determining that the visitor qualifies in the segment may be performed by the back end component after the visitor has completed visiting the first network content (e.g., visiting second network content or after the visitor's browser is closed).

Evaluating network content data by the backend component may include performing a full table scan on an hbase cluster (or comparable data store) that stores billions of visitor profiles. Each profile includes network content data associated with one of the visitors. Moreover, evaluating network content data (by the real-time and/or back end) may include evaluating the network content data for all of the active segment rules. Note that many segment rules may exist at once but for ease of explanation, however, much of the description focuses on a single segment rule. In various embodiments, evaluating network content data may be performed periodically (e.g., once an hour, once a day, once a week, etc.) by the back end component. Periodic evaluations may result in updated visitor qualifications based on a new, updated, or changed segment rule, or based on new data that allows a visitor to be qualified under the segment rule.

In various embodiments, at least some of the network content data, which is evaluated according to the segment rule, may be associated with the visitor's visit to first network content.

In some embodiments, determining that a visitor qualifies in the segment may be performed after or in response to a change to a previous segment rule. For example, the previous segment rule may have defined a male, high end car segment as a visitor who is male and has visited a high end car (e.g., MSRP>$50k) website (or distinct webpages) five times in the previous three days. The updated segment rule may lower the threshold of high end car to MSRP>40k. Accordingly, qualification of the visitor in the segment may be determined automatically in response to the update to the segment rule or it may require user input to determine qualification under the updated segment rule.

In some embodiments, the segment rule may be created after the visitor has completed visiting the first network content. For example, similar to the example above, a segment rule for a male, high end car segment may be created after the visitor has already visited a high end car website the requisite number of times to qualify in the segment. Moreover, the segment rule may be created after the visitor has left the first network content such that the visitor is not actively browsing the first network content. Accordingly, the visitor may be qualified, for example, by the back end component, after such a visit to the network content is complete by using the newly created segment rule and offline data (e.g., first party from the customer who owns the first network content or from a third party data provider, etc.).

In some embodiments, a visitor may qualify in a segment based entirely on third party provided data. Such embodiments may permit a customer to target an advertisement to a visitor who has never visited a website associated with the customer. In other embodiments, first party data (e.g., from the visitor's visit to a website), third party data, or some combination of first and third party data may be used to qualify the visitor in a segment.

In one embodiment, the same segment rule and same code libraries may be shared across both the real-time component as well as the back end component application. The code libraries may exist on each of the data collection servers of the real-time components and also on a plurality of nodes of the back end components. In one embodiment, the back end component application may be a Map-Reduce application implemented in Java. The same core logic may be used in both the real-time and back end components. The data storage layer for the real-time component, though, may be different than the data storage layer for the back end component. In the case of the real-time component, the data storage layer may be a combination of visitor cookie data as well as behavioral trait data stored in the real-time profile cache server (PCS) machines. In one embodiment, the data storage layer for the back end component may be a distributed data store implemented in the cloud, such as an hbase cluster.

At410, an indication of the visitor's segment qualification may be stored in the real-time component. In an embodiment where determining that the visitor qualifies in the segment is performed by the back end component, storing the indication may include the back end component providing the indication to the real-time component. Note that an indication of each segment that the visitor qualifies for may be stored in the real-time component. For example, a visitor may qualify for multiple segments, each of which may be represented in the real-time component by an indication of the qualification.

As illustrated at420, an instruction to display an advertisement for the visitor may be generated based on the visitor's qualification in the segment. The instruction may be a destination call that is a URL into another system that sends a unique segment identifier and allows the other system to associate it to the visitor based on cookies in their own domain. The URL destination delivery may happen within a safe/protected IFRAME to ensure it does not impact perceived page performed on the customer website and to ensure a high degree of security to the customer's own domain and the document object model (DOM) of its pages. The destination may be delivered via server-to-server destinations (e.g., DSPs104) or by having the real-time destination sent to the real-time component, to be delivered the next time the visitor visits a website of the customer owning the segment (or any other customer's website). For example, in one embodiment, based on the stored indication of the visitor's segment qualification, the real-time component may deliver a destination call associated with the segment to second network content during the visitor's visit to the second network content. The destination call may be usable to request and display the advertisement associated with the segment.

In one embodiment, the second network content may be different network content than the first network content. For example, the first network content may be one webpage of a website of a customer (e.g., the home webpage of an automobile shopping website) and the second network content may be a different webpage of the same website (e.g., an about us webpage of the automobile shopping website). In such an instance, the advertisement may be delivered while the visitor is on the same website, but a different page of the same website.

In one embodiment, the second network content is network content associated with a different customer than the first network content. For example, the first network content may be a website of one customer (e.g., an automobile shopping website owned by company/customer A) and the second network content may be a website of a different customer (e.g., a webmail website owned by company/customer B). In such an instance, an advertisement associated with customer A, which may be based on the user's qualification in a segment created by customer A, may be delivered after the visitor has left the website owned by company/customer A (e.g., while the visitor is visiting the website owned by company/customer B). In fact, the advertisement associated with customer A may be delivered to the visitor even if the visitor never again returns to the website owned by customer A.

In some embodiments, after display of the advertisement (e.g., in a browser or some other interface of the visitor), the real-time component may provide an indication of the display of the advertisement to the back end component. The back end component may then store the indication of the display of the advertisement. The stored indication of the display of the advertisement may be used in selecting which advertisement to present to a visitor. For example, each advertisement may be presented to each visitor only a limited number of times (e.g., once, five times, etc.). By maintaining an indication of which advertisements have been displayed to a particular visitor, a variety of advertisements pertaining to the segment(s) for which the visitor qualifies may be presented.

In one embodiment, the real-time component may apply the segment rule to the visitor and determine that the visitor does not qualify in the segment with such a determination being based on less data than the network content data. The back end component, based on the network content data, may determine, however, that the visitor qualifies in the segment. Such a determination by the back end component may qualify the visitor in the segment regardless of the determination by the real-time component that the visitor does not qualify. As a simple example, real-time component may not have the storage capability to store much old data (e.g., older than one month) whereas the back end component may have that storage capability (e.g., over a year's worth of data). Therefore, the back end component may qualify the visitor based on older data that is available to the back end component but not to the real-time component. Therefore, the determination, by the back end component, that the visitor qualifies may be based on the network content data whereas the determination, by the real-time component, that the visitor does not qualify may be based on less data than the network content. Consider an example regarding a seasonal event, such as tax season. A visitor may not visit any tax preparation related websites outside of February each year. Therefore, in September, the real-time component may not have access to any data that may qualify a visitor as interested in tax preparation whereas the back end component may have access to data from February that could qualify the visitor.

Turning now toFIG. 5, one embodiment of audience size estimation and/or implementing complex segment logic (e.g., combined recency and frequency, n-level deep nested Boolean expressions, etc.) is illustrated. While the blocks are shown in a particular order for ease of understanding, other orders may be used. In some embodiments, the method ofFIG. 5may include additional (or fewer) blocks than shown. Blocks500-520may be performed automatically or may receive user input. The method ofFIG. 5may be used in conjunction with the method ofFIG. 4. Accordingly, a combination of some or all of the blocks ofFIGS. 4-5may be used in some embodiments. In one embodiment, the segment analysis module ofFIGS. 1-3may implement the method ofFIG. 5.

As shown at500, a selection of a trait may be received. As described herein, the trait may be descriptive of visitors of network content. Example traits may include male, car buyer, between ages 40 and 50, interested in laptops, etc. Selection of a trait may be received via a user interface or portal, for example, from a customer associated with network content. Example user interfaces are shown inFIGS. 7-18, and are described in more detail below. In some embodiments, selection of other traits that are also descriptive of visitors of network content may be received. For example, selection of any number (e.g., three, five, seven, etc.) of traits may be received.

At510, a segment rule may be created that is usable to evaluate a combined recency and frequency of one or more qualification events together. The qualification events may indicate qualification of a visitor according to a trait. The one or more qualification events may be based on collected network content data (e.g., audience data) associated with a plurality of visitors. Each of the one or more qualification events may correspond to qualification of a separate portion of the collected network content data according to the trait.

In contrast to a system that separately evaluates recency and separately evaluates frequency, evaluating a combined recency and frequency of one or more qualification events together may include evaluating a number of qualification events over a defined period of time (e.g., the visitor qualified x times over the past y days). A system that separately evaluates recency and frequency may evaluate a most recent event and separately evaluate a frequency counter of the event (e.g., the visitor qualified x times total and qualified once in the past y days). A simple example illustrates the difference. Consider a scenario in which visitor A has visited a high end car website once a month for the past five months with the most recent occurrence being two days ago. At block510, a segment rule may be created that defines a high end car buyer segment as someone who has visited a high end car website five times in the past three days. Visitor A, who has visited such a website once a month for the past five months with the most recent visit being two days ago would not qualify according to the five times in the past three days combined recency and frequency segment rule. However, in a system that considers the recency and frequency separately, visitor A would qualify under frequency as having visited a high end car website five times total and would also qualify under recency because they visited a high end car website two days ago. Marketers are often interested in the intensity with which visitors are associated with a trait. Evaluating the combined recency and frequency may reflect the intensity whereas separate recency and separate frequency does not. In the example above, visitor A is not associated with very much intensity and therefore may be viewed, by marketers, as a casual browser of the high end car website rather than a serious consumer of high end cars.

In one embodiment, each of the one or more qualification events may be associated with a time of occurrence of the respective qualification event. Each time of occurrence may be stored with an indication of the respective qualification event, as opposed to a system that only stores the most recent timestamp of a qualification event. Such times of occurrence may be usable in evaluating the combined recency and frequency according to the segment rule.

In some embodiments, an indication of each of the one or more qualification events may be stored (e.g., in a data store). In some embodiments, the data store may be local to the real-time component, such as in a data store that is quickly accessible (e.g., a data cache). In some embodiments, the indication may be stored in the back end component in addition to or instead of storing the indication local to the real-time component.

In an embodiment in which multiple traits are selected, a selection of an operator for the segment rule may be received. The operator may be for a Boolean expression that includes the trait and another trait. For example, the segment rule may include one or more instances of the operators AND, OR, NOT AND, or NOT OR. As a simple example, a segment rule may be male AND high end car buyer. Note that the above discussed combined recency and frequency may be applied across one or more of the multiple traits. Moreover, a segment rule having a Boolean expression may have such a Boolean expression in addition to or instead of combined recency and frequency. Therefore, in some embodiment, a segment rule may not use the described combined recency and frequency but instead, my use one or more of the described Boolean expression. As another simple example, consider a segment rule that is defined as high end car buyer AND high end boat buyer. The combined recency and frequency can be applied to either high end car buyer, high end boat buyer, or both traits. In some embodiments, the segment rule may be an n-level deep nested Boolean expression. Such an example is illustrated inFIG. 17, at item1702.

In one embodiment, the complex segment logic may be implemented as a domain specific language (DSL). One example implementation is in Java, using Another Tool for Language Recognition (ANLTR). The DSL rules may support: Boolean expressions that are next N-levels deep (e.g., no limitation on nesting depth); operators that include AND, OR, NOT; a combined recency and frequency (R/F) operator, which can be applied to one or a list of traits; R/F operators ==, >=, <=; and optional inclusion of recency in the R/F expression.

As illustrated at520, the combined recency and frequency of the one or more qualification events may be evaluated according to the segment rule. Evaluating the qualification events according to the segment rule may include reading the qualification events from the data store, which may include reading a respective time of occurrence of the respective qualification events.

In one embodiment, evaluating the qualification events according to the segment rule may include estimating the segment population size based on the stored indications of each of the one or more qualification events. The segment population size may indicate the number of unique visitors that qualify under a segment according to the segment rule. The segment population size may be performed for one or more time ranges. For example, the segment population size may be for the last 7 days, last 30 days, last 60 days, etc. Estimating the segment population size for a time range may include evaluating the combined recency and frequency of each qualification event (e.g., including a time of occurrence associated with each respective qualification event) that is within the time range.

In one embodiment, estimating the segment population size may include executing a search query on a data store of the back end component that searches millions of visitor profiles (from first and third party data) nearly immediately. The search service may be referred to as a distributed search cluster service with the result being a real-time estimation of the segment population size. In one embodiment, the search service may be implemented as a SOLR cluster (or comparable open source search tool) of the segment analysis module. For example, all the visitor profiles and traits may be stored in a SOLR cluster. Upon receiving an audience definition/segment rule (e.g., via a UI/portal), a query to search the SOLR cluster may be executed. The SOLR cluster may then return the number of visitors qualifying under the audience definition. Other embodiments may use a traditional database.

The segment population size for the time period (e.g., for one time period) and other respective segment population sizes (e.g., for other time periods) may be provided for display. The segment population size and each of the other respective segment population sizes may each correspond to a respective time range of a plurality of time ranges. Example displays of such segment population sizes are shown inFIGS. 14-18. The UI/portal may provide a calculate/recalculate segment population input option. Upon selecting such an option, a query may be generated to the backend, as described herein, which may then return the segment size estimation. Note that if the segment rule is defined as three occurrences (e.g., qualification events) in the last seven days, then the segment size estimation over the last 30 days corresponds to qualification of visitors over any 7 consecutive day period in the last 30 days. For example, the last 30 days segment population may include a visitor who qualified based on data from days 1-7 and another visitor who qualified based on data from days 21-27.

In one embodiment, a future segment population size may be estimated based on the estimated segment population size. For example, the system may determine the estimated segment population size has a trend of a 10% increase for each 60 day period. Therefore, the system may estimate that the future segment population size may continue to increase by 10% for subsequent 60 day periods.

In various embodiments, an advertisement to be provided to the visitor of network content may be determined. Note that which specific advertisement is chosen may be selected by an ad server (e.g., onsite ad server116). In other instances, the segment analysis module may select the advertisement or simply provide an instruction that includes which segments a visitor qualifies for. That instruction may be accompanied with which advertisements a particular visitor has already seen such that any advertisement viewing limits may be respected. The evaluating of block520may result in the visitor qualifying for a segment associated with a particular advertisement.

The disclosed techniques may have the speed advantages of real-time segment qualification along with the data storage scale advantages of back end segment qualification. The back end segment qualification additionally permits qualification of visitors even without ever seeing that visitor again. Moreover, by using real-time qualification in conjunction with backend qualification, there may be less discrepancy rates as opposed to simply using real-time qualification and waiting for the visitor to stick around. Further, the disclosed techniques may support arbitrarily complex rule definitions and permit the ability to qualify visitors into segments in real-time. The disclosed techniques may also support arbitrarily deeply nested Boolean expressions. Additionally, the disclosed techniques may provide real-time feedback to a customer on segment population size as the segment rule is being defined, something that otherwise may not be achieved with a traditional relational database system.

FIG. 6illustrates an example block diagram of a system configured to implement the methods ofFIGS. 4-5, according to some embodiments. Note that certain portions ofFIG. 6are similar to portions ofFIGS. 2 and 3. For those portions, description similar to that ofFIGS. 2 and 3applies toFIG. 6but, for brevity, is not necessarily repeated. Solid lines between components ofFIG. 6may represent customer driven data, dashed lines may represented visitor driven data, and the dotted line to custom reports608may represent reporting data. Beginning at the top ofFIG. 6, visitors612may visitor a customer website610. Customer website610may be presented in a browser application of a client device of visitor612. A JavaScript (e.g., data integration library (DIL)) hosted by the customer may be loaded, which may call an IFRAME that is hosted (e.g., on an Akamai server) that may be used for real-time destinations (e.g., where to send a URL to get an advertisement).

Data about the visitor's visit and/or about the visitor may be collected through customer website610and provided to edge data centers618, which maintain the traits and segments. The DCS may store trait qualification activity to PCS, which may be a low latency edge cache database (e.g., implemented on top of a NoSQL database technology). The DCS may read times of trait qualifications of visitors from the PCS in real-time to determine segment qualification. Note that back end ingested trait data may also be stored at edge data centers718to take advantage of real-time speed with greater flexibility in segment qualification. Log files that include information about the visitor and what the visitor did during the visit may then be sent to cluster (visitor database)640, which is implemented in the cloud in the illustrated embodiment. Sending of the log files may be periodic (e.g., every 10 minutes). Cluster (visitor database)640may store a large number of visitor profiles (e.g., more than 6 billion). Once that data is written to cluster (visitor database)640, which may be part of the back end component, it may be written out to analytics cluster (raw logs)638. Analytics cluster (raw logs)638allows for an association between visitor and trait. Periodically (e.g., once a day, twice a day, etc.), analytics cluster638may then write its stored data to SOLR632to update the SOLR indexes. That data may include a trait/time combination for each visitor for each qualification event. Estimates of segment size may then be performed according to the techniques disclosed herein. Such estimates may be provided as customer reports608and/or as data for display at UI portal and APIs604for customer602.

From the viewpoint of customer602, UI Portal and APIs604may call a customer API that interacts with the SOLR cluster to generate queries on the dataset and return results and a confidence interval based on the data set result size. The result is that customers can receive nearly instant feedback even while creating extremely complex segments. As described herein, customer602may define a segment rule by inputting traits, destination rules, R/F operators, among other inputs. That rule is stored and persisted in control database620. Via configurator622, tag code, traits, segments, and destinations are shared between the real-time component (including edge data centers618) and the back end component (including cluster640and analytics cluster638). TIM Tags, Destination Publishing, and Data Collection Scripts may then be pushed through a cloud-based server616to the DIL/TIM container of customer site610. The destination IFRAME may be used for performing ID sync calls via ID sync650. For example, the destination IFRAME may be used when trying to match a visitor ID with an ID from third party data for server to server file transfers. The IFRAME is a separate frame that can have a different source from the current domain that the visitor's browser is in. The IFRAME may be hidden from the visitor such that the visitor never sees it. Therefore, data transfers and data synchronization may be performed without impacting the end user/visitor. Additionally, the IFRAME is boxed off from a security standpoint. Whatever happens in the IFRAME cannot impact the customer domain. Using the IFRAME and the call to the DCS, a URL destination can be sent to send information to a third party.

Data provider partner654represents a third party data provider. Third party data may be matched to other data via ID sync650. Third party data (and first party data from edge data centers618) may be processed by inbound feed converter644before being stored in the backend (e.g., cluster640). Outbound feed converter642of the back end component may likewise process data from the backend before sending it to SFTP publishers646and HTTP/HTTPS feed submitter648. Such data from the back end may be provided to data provider partner654.

As shown in the right side ofFIG. 6, an instruction may be provided to ad networks/demand side platforms652indicative of a visitor's segment qualification and/or indicative of an advertisement to present for display to that visitor.

FIGS. 7-18illustrate screenshots of an example graphical user interface for which various embodiments may be performed.FIG. 7illustrates interface700for managing various segments according to a list view. Context-specific tool bar702of interface700may present a variety of context-specific actions. For example, Create a New Segment, Add Selected to Destination, Create Model with Selected, Delete Selected, are among other possible actions may be presented in context-specific tool bar702. Some actions may appear based on specific interactions. For example, Create Model with Selected may only appear after selection of a segment. Search704permits segment-specific searches to be performed. The search/filter element may be specific to searching segments. Entering a character in the search input may immediately filter the segment list to segments that contain the term or sequence of characters. Folder browser706allows users to navigate folders and/or sub-folders that contain segments. Folders that contain children can be toggled open by clicking the folder icon. Activating a folder filter may activate a segment list view, an example of which is shown inFIG. 7, to display the segments contained within the selected folder. The list of segments708may be sortable, filterable, and/or include tabular data. Segment elements may be sorted, for example, by clicking the table header of the column. Segment hyperlinks710may be selectable (e.g., via mouse click, touchscreen, or other selection) to take the user to a segment detail page/summary. Segment description712may appear in a bubble, as shown, when hovering (e.g., with an indicator of a mouse) over the segment row. Segment actions714may be displayed upon hover over and may include actions such as edit segment, pause/activate segment, clone segment, and/or remove segment, among others. Hovering over an action icon may cause display of a tool tip describing the action. Segment selection716may permit users to select segments, for example using checkbox elements, to remove the selected segments from the list view.

FIG. 8shows a snapshot of an example display800of the basic information options for creation of a new segment. Basic information tab802may be open by default. Segment storage804may permit users to select a folder to store a new segment.FIG. 8illustrates collapsed accordions206for both the Traits and Destinations Mapping tabs. The collapsed accordions may be expanded and likewise, expanded accordions (e.g., basic information) may be collapsed. Expanding/collapsing may be toggled, in one embodiment, via selection of the double carrot to the left of the tab name. Save/cancel808may be selected inFIG. 8to apply any previous input(s), such as segment name, description, status, data source, integration code, etc. as shown in basic information tab802.

FIG. 9shows a snapshot of an example display900of the expanded trait tab/accordion, which may be used to create a new segment. The default active tab902may be a what-you-see-is-what-you-get (WYSIWYG) view. Code view tab904may permit a user to see the code that is generated from the WYSIWYG view and or permit a user to define a segment and/or traits using a code view. For example, a user may also simply type or otherwise input a code expression into the segment expression code view. An example of the code view is shown inFIG. 17. Displayed trait906may be composed of a hyperlinked trait name and the last 30 day unique visitors (e.g., 24,054 in the example shown). Clicking a trait hyperlink may open a modal window, or other display, that displays read-only basic information regarding the trait. Such a display that displays read-only basic information regarding the trait is shown inFIG. 11. Added traits may be automatically grouped together but can be separated by operators (e.g., Boolean operators). Icon908(shown as a clock symbol) may indicate if recency and frequency rules have been added to a trait or a group of traits. Selection of icon908may permit the frequency and/or recency rule to be modified. Operators910may allow traits or groups to be separate by operators such as AND, OR, AND NOT, and/or OR NOT. Hovering between two traits may display operator selection interface912. Selecting an operator may separate the traits or groups of traits. Hovering over a trait may expose trait tools914, such as edit, remove, and/or drag-and-drop tools. As one example, a trait may be dragged vertically into the desired position according to the drag-and-drop tool. As another example, a trait may be edited by clicking the edit icon or a trait may be removed from the list by clicking the remove icon. Total size916may be calculated as the sum of the 30-day unique visitors of all of the traits in a group. Add trait918permits input to enter a rule into the interface. Typing in the Search by Trait Name field of add trait918may display existing traits that can be selected and added to the segment. As shown, the user here is searching for “Cam” with results being “Camera”, “Camera Shopper”, “Canon Camera” and “High-End Camera Shopper.” New traits are added to the bottom of the other traits that are already shown and may be automatically grouped with the last trait in the list. Interface900may provide indication922that is indicative of a changed segment. Such an indication may prompt a user to select Recalculate Size to recalculate the size of a segment, as described herein. The example display ofFIG. 9may also permit selection of Browse All Traits920, which may open a modal interface containing a filterable, searchable list of traits, as shown inFIG. 10.

FIG. 10illustrates an example of the modal interface upon selection of Browse All Traits920ofFIG. 9. Trait-specific search1002is a search/filter element specific to traits. Entering a character in the search input may filter the trait list to traits that contain the term or sequence of characters. Folder browser1004permits navigation of folders and sub-folders that contain traits. Folders containing children can be toggled open by clicking the folder icon. Activating a folder may cause the trait list view to display the traits contained within the selected folder. Trait selection1006is indicated by the checkbox next to a given trait. Traits may be selected by clicking anywhere on the trait row and is not limited to the actual checkbox. In some embodiments, trait data1008may not permit trait name hyperlinks or trait actions in the model interface ofFIG. 10. Action buttons1010may permit selected traits to be added to a segment by clicking Add Selected Traits to Segment. Clicking Cancel closes the interface, ignoring whatever selections were made, and returns the user to the segment builder display ofFIGS. 7-9.

Selecting a hyperlinked trait name in the segment builder (e.g., hyperlinked trait name710ofFIG. 7) may open another modal interface1100, as shown inFIG. 11. Modal interface1100displays read-only basic information for the selected trait with name1102. The display of the information is read-only in that no actions may be taken regarding the trait or segment in from the interface ofFIG. 11. As shown, various information may be presented in the interface ofFIG. 11including a trait ID (e.g., 85495), name (e.g., Trait Name), description, type, data source, integration code, stored location, data category, unique visitors for various time ranges (e.g., 7 day, 30 day, 60 day, etc.), and/or comments regarding the trait.

As shown in example interface1200ofFIG. 12, selecting the recency-frequency icon1202for a given trait or group of traits may present a recency frequency interface1204over the selected trait or group of traits for which the recency frequency operator is to be applied. Recency frequency selectors1206may include frequency operators, numbers, and/or time selectors (e.g., hours, days, weeks, etc.). For example, as shown in the example ofFIG. 12, a frequency of greater than 5 times within the past 2 days is indicated. Action buttons1208may include a reset button that may reset the recency and frequency interface settings to a default setting. Other action buttons (not shown) may include a save button within recency frequency interface1204(in addition to the Save/Cancel buttons at the bottom of interface1200). Estimated historic segment size1210is displayed below the trait list. If a change is made to a segment, notification1212may be displayed (e.g., as shown, “This segment has changed”) along with an action button to Recalculate Size.

FIG. 13shows example interface1300that illustrates expansion of the Destinations Mapping tab1302for an example segment builder application. The destinations are displayed in table format in the illustrated example. In various embodiments, destinations may include a URL, key value pair, and/or ID, as well as a start and end date. Destination name1304may be a selectable hyperlink that, upon selection, opens a modal interface containing read-only basic information about the destination. Actions1306and1308permit a destination to be removed from or added to a segment. Browse All Destinations1310is selectable to open a modal interface that includes a list of destinations.

FIG. 14illustrates a summary view of a segment. Toolbar options1402may include Create a New Segment, Edit Segment, Clone Segment, Delete Segment, and/or Share/Print. Basic information entered during segment creation may be displayed in the basic information panel1404, which may include read-only segment storage path1406. The summary view may also include a graphical display1408that illustrates the number of unique segment visitors at various times (which may represent various length time intervals, such as a 7-day, 30-day, 60-day, etc. time interval). The traits and operators that are used to create the segment may be displayed in trait panel1410. Trait panel1410may include icon1412that indicates if recency and frequency rules have been added to a trait or a group of traits. Hovering over icon1412(e.g., as shown by the finger pointer) may display a read-only (e.g., not editable within trait panel1410) summary of the recency-frequency. Selecting a trait name1414may take the user to the trait summary interface. Segment traits may be edited by selecting button1416, which may redirect the user directly to the traits edit interface. Destinations1420are displayed and can be edited by selecting Edit Destinations.

FIGS. 15-17illustrate an example creation of a segment and use of the created segment, according to various embodiments. As shown inFIG. 15, an example interface1500is shown, which illustrates the basic information and trait tabs expanded. The segment in the example is named Male Jersey Shore Viewers, is selected as an active segment, with a data source chosen. Data sources may include real-time data sources, offline data sources (e.g., first party offline data, or data from third party data providers), and/or other data sources. As shown, the segment rule includes AND'ing together the trait Male with the traits Video-Generic Video View-Show Title-Jersey Shore and Video-Full Episode Start-Show Title-Jersey Shore. InFIG. 15, because the segment is being created, the size of the segment has not yet been calculated. As shown at1502, Recalculate Size button1502is shown, which may cause the audience size (e.g., last 7 days, 30 days, 60 days, real-time, historic, etc.) of unique visitors that qualify according to the segment rule to be calculated.

FIG. 16continues upon the example ofFIG. 15and illustrates the calculation of the audience size as shown in example interface1600. As shown, the last 30 days of unique visitors that qualify for each of the traits is shown as well as the estimated historic segment size for the last 7, 30, and 60 days. Note that in other embodiments, other time ranges may be displayed in addition to, or instead of, the example time ranges shown inFIG. 16.

FIG. 17illustrates an example of using code view of interface1700to define a trait and/or segment, as opposed to the basic view shown inFIG. 15. As shown, code view1702illustrates a segment definition using various terms and Boolean operators to define various traits. The segment rule (61405T OR 62727T) AND (67874T OR 71056T) AND NOT (61404T OR 61794T OR (62726T AND 63562T)) is an example of a segment rule and traits defined according to a code view representation. Such a segment rule may represent an equivalent to the rule shown inFIG. 15or it may be a completely different rule. Note that toggling between the basic view and code view1702may toggle the display of the trait and segment rule such that the when basic view is selected, a plain English representation of the segment rule represented by code view1702may be shown. Moreover, for a segment rule defined in the basic view, by selecting code view, the segment rule may be display according to the code view. Code view may include a validate expression button1704that is usable to compile and/or otherwise validate that the segment rule and traits are valid. Save/cancel buttons1706are usable to save/cancel the code view segment rule. Note that if validate expression1704is performed, if the segment rule is validated successfully, the rule may automatically be saved. The other portions of interface1700may function in a similar manner as the interface shown inFIGS. 15-16.

FIG. 18illustrates one embodiment of an interface1800configured to display real-time and total segment population numbers for a segment rule that includes by one or more traits. As shown, the estimated historic segment size over the last 30 days indicates 4736 unique visitors that qualify for the segment whereas the real-time segment population indicates 6525 unique visitors over the last 30 days. The total segment population is 8370 unique visitors over the last 30 days. As shown in the example, numbers for the last 7 days and last 60 days are also displayed. Note that in other embodiments, other time ranges may be displayed in addition to, or instead of, the example time ranges shown inFIG. 18. Note that the total segment population minus the real-time segment population yields the number of visitors qualified via the back end component.

Example System

In the illustrated embodiment, computer system1900includes one or more processors1910coupled to a system memory1920via an input/output (I/O) interface1930. Computer system1900further includes a network interface1940coupled to I/O interface1930, and one or more input/output devices1950, such as cursor control device1960, keyboard1970, and display(s)1980. In some embodiments, it is contemplated that embodiments may be implemented using a single instance of computer system1900, while, in other embodiments, multiple such systems, or multiple nodes making up computer system1900, may be configured to host different portions or instances of embodiments. For example, in one embodiment some elements may be implemented via one or more nodes of computer system1900that are distinct from those nodes implementing other elements.

In various embodiments, computer system1900may be a uniprocessor system including one processor1910, or a multiprocessor system including several processors1910(e.g., two, four, eight, or another suitable number). Processors1910may be any suitable processor capable of executing instructions. For example, in various embodiments, processors1910may be general-purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. In multiprocessor systems, each of processors1910may commonly, but not necessarily, implement the same ISA.

System memory1920may be configured to store program instructions and/or data accessible by processor1910. In various embodiments, system memory1920may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory, or any other type of memory. In the illustrated embodiment, program instructions and data implementing desired functions, such as the techniques described above are shown stored within system memory1920as program instructions1925and data storage1935, respectively. In other embodiments, program instructions and/or data may be received, sent or stored upon different types of computer-accessible media or on similar media separate from system memory1920or computer system1900. Generally speaking, a computer-accessible medium may include storage media or memory media such as magnetic or optical media, e.g., disk or CD/DVD-ROM coupled to computer system1900via I/O interface1930. Program instructions and data stored via a computer-accessible medium may be transmitted by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a network and/or a wireless link, such as may be implemented via network interface1940.

In one embodiment, I/O interface1930may be configured to coordinate I/O traffic between processor1910, system memory1920, and any peripheral devices in the device, including network interface1940or other peripheral interfaces, such as input/output devices1950. In some embodiments, I/O interface1930may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory1920) into a format suitable for use by another component (e.g., processor1910). In some embodiments, I/O interface1930may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example. In some embodiments, the function of I/O interface1930may be split into two or more separate components, such as a north bridge and a south bridge, for example. In addition, in some embodiments some or all of the functionality of I/O interface1930, such as an interface to system memory1920, may be incorporated directly into processor1910.

Network interface1940may be configured to allow data to be exchanged between computer system1900and other devices attached to a network, such as other computer systems, or between nodes of computer system1900. In various embodiments, network interface1940may support communication via wired or wireless general data networks, such as any suitable type of Ethernet network, for example; via telecommunications/telephony networks such as analog voice networks or digital fiber communications networks; via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and/or protocol.

Input/output devices1950may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or retrieving data by one or more computer system1900. Multiple input/output devices1950may be present in computer system1900or may be distributed on various nodes of computer system1900. In some embodiments, similar input/output devices may be separate from computer system1900and may interact with one or more nodes of computer system1900through a wired or wireless connection, such as over network interface1940.

As shown inFIG. 19, memory1920may include program instructions1925, configured to implement embodiments of a segment analysis module as described herein, and data storage1935, comprising various data accessible by program instructions1925. In one embodiment, program instructions1925may include software elements of embodiments of a segment analysis module as illustrated in the above Figures. Data storage1935may include data that may be used in embodiments. In other embodiments, other or different software elements and data may be included.