Patent ID: 9031861
Filing Date: 2015-05-12
Classification: G06Q

Abstract:
1. A method comprising: identifying a corpus of campaigns, each campaign including constraints, delivery goals and selection criteria for the delivery of one or more associated content items; determining, by one or more processors, a candidate campaign from the corpus of campaigns including evaluating constraints associated with delivery of the one or more content items for a first campaign and determining that the constraints include a budget constraint that is sufficient so as to enable the first campaign to be included in an experiment while still enabling the first campaign to meet its respective delivery goals; identifying a content item to be presented to users in response to requests for content, the content item including a link to a resource and being part of the first campaign associated with a content item sponsor, the campaign being associated with a geographic region; determining a division of the geographic region including identifying a plurality of different geographic locations in the geographic region wherein each geographic location is sufficiently small so as to not consume more than a predetermined percentage of the impressions of the content item for the first campaign; selecting one or more of the plurality of different geographic locations to be experimental geographic locations; selecting one or more of the plurality of different geographic locations to be control geographic locations; pausing presentation of the content item in the one or more experimental geographic locations for a time period, while continuing presentation of the content item in the one or more control geographic locations; and measuring content item effectiveness of the content item including comparing conversions associated with the resource during the time period for the one or more experimental geographic locations and conversions associated with the resource during the time period for the one or more control geographic locations.