Determining advertising effectiveness outside of a social networking system

A social networking system or other user registration site builds a log of exposures by users to advertisements outside of the user registration site to determine their effectiveness. For each user exposed to an advertisement, a log entry is created indicating that the user has been exposed to the advertisement. Tracking pixels are embedded into advertisements that, when accessed, enable the social networking system or user registration site to log access to the advertisement by the user. From the log files, the user registration site identifies exposed users and selects unexposed users with similar demographics and/or behavior information to generate a control group. The two groups can be surveyed about the advertisement to determine its effectiveness. The user exposure information also can be used to retarget advertisements, to measure advertising effectiveness on connections of exposed users, and to measure actions of exposed users.

BACKGROUND

This invention relates generally to social networking, and more particularly to determining advertising effectiveness among social network users for advertisements outside of a social networking system.

To determine the effectiveness of advertising, typically polls are issued to groups of people exposed to an advertisement or an advertising campaign, and to a control group of people not exposed to the same advertisement or campaign, who may be similar in other respects. However, in the context of users of a social networking system and advertising exposure outside of the social networking system, often there is not a readily available way to determine the exposed and control groups.

In a typical social networking system, such as a social networking website, users set up their user profiles and then establish connections with other users of the social networking system. The users often provide information about themselves expressly to the social networking system, such as demographic information and/or a list of the users' interests. Users may also provide information about themselves implicitly to the social networking system, through their actions on the system and interactions with other users. In this way, a social networking system can obtain a rich set of social information about its users, which may be used in a great many ways to enhance a user's experience online.

Users of a social networking system may be exposed to various online advertisements. If they occur within the social networking system, these exposures may be tracked by the social networking system. The social information about the users maintained by the social networking system may provide a rich set of data about the users and their exposures. However, advertisers typically cannot leverage this social information in the contact of advertising impressions that occur outside of the social networking system, and thus cannot use the information to determine the effectiveness of advertising.

SUMMARY

To assess effectiveness of advertisements outside the domain of a social networking system or other user registration site, the social networking system or registration site builds a log of exposures by users to advertisements outside of the social networking system or user registration site. For each user exposed to an advertisement, a log entry is created indicating that the user has been exposed to the advertisement. To build the log, tracking pixels are embedded into advertisements on web pages provided by third parties. When a user accesses an advertisement including a tracking pixel, the tracking pixel enables the social networking system or user registration site to log access to the advertisement by the user.

In one embodiment, the tracking pixel accesses a cookie on the user's device, from which it retrieves a user identifier and sends it to the social networking system or user registration site along with identification of the advertisement viewed. According to another embodiment, upon the user accessing the advertisement, the tracking pixel initiates a request from the user device for the associated image from the social networking system or user registration site, and sends a social networking system cookie or user registration site cookie stored on the user's device along with the request. From the request the social networking system or user registration site determines the associated user. The social networking system or user registration site logs access to the advertisement by the user. From the log files for various users who have accessed the advertisement, the social networking system or user registration site determines exposed users, and users not exposed who have similar demographics and/or behavior information to generate a control group. The two groups are surveyed about the advertisement to determine its effectiveness.

The user exposure information may also be used to retarget advertisements based on whether a user has been exposed to the advertisement, or based on the number of impressions for the user. For example, an advertiser can use the advertisement exposure information to determine whether to serve another advertisement to a particular user, based on whether multiple exposures to the same advertisement are desired or whether a second advertisement should be provided to a user exposed to a first advertisement.

The advertising exposure information may also be used for measuring advertisement effectiveness on connections of exposed users, e.g., by discussion of the advertisement between the user and the user's connections outside of the social networking system or user registration site.

The data also can be used to measure actions of exposed users within the social networking system or user registration site that relate to the advertising to which the user was exposed, e.g., “liking” or commenting on a product that is the subject of an advertisement to which the user was exposed. These actions provide another possible measure of the effectiveness of the advertisement beyond just a survey.

DETAILED DESCRIPTION

Overview of a Social Networking System Architecture

FIG. 1is a network diagram of one embodiment of a system100for providing interactions between advertising content and users of a social networking system130, or user registration site. Although the example used herein is one of a social networking system, the system100could instead include any other type of user registration site. The system100includes one or more user devices110, one or more third-party websites120, the social networking system130and a website140. For purposes of illustration, the embodiment of the system100shown byFIG. 1includes a single third-party website120and a single user device110. However, in other embodiments, the system100may include more user devices110and/or more third-party websites120. In certain embodiments, the social networking system130is operated by the social network provider, whereas the third-party websites120are separate from the social networking system130in that they may be operated by different entities. In various embodiments, however, the social networking system130and the third-party websites120operate in conjunction to provide social networking services to users of the social networking system130. In this sense, the social networking system130provides a platform, or backbone, which other systems, such as third-party websites120, may use to provide social networking services and functionalities to users across the Internet.

The user device110comprises one or more computing devices that can receive input from a user and can transmit and receive data via the network140. For example, the user device110may be a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDAs) or any other device including computing functionality and data communication capabilities. The user device110is configured to communicate with the third-party website120and the social networking system130via the network140, which may comprise any combination of local area and/or wide area networks, using both wired and wireless communication systems.

In one embodiment, the user device110displays content from the third-party website120or from the social networking system130by processing a markup language document116received from the third-party website120or from the social networking system130using a browser application112. The markup language document116identifies content and one or more instructions describing formatting or presentation of the content. By executing the instructions included in the markup language document116, the browser application112displays the identified content using the format or presentation described by the markup language document116. For example, the markup language document116includes instructions for generating and displaying a web page including a tracking pixel that can read a cookie114provided by the social networking system130or can redirect the browser112to the social networking system130. In various embodiments, the markup language document116comprises a data file including extensible markup language (XML) data, extensible hypertext markup language (XHTML) data or other markup language data.

In one embodiment, the user device110also includes a cookie114including data indicating an identifier of a user account associated with the user of the device110, identifying the client device110itself, and/or whether the user is logged into the social networking system130. The cookie114indicates whether the user of the computing device110is involved in an active session where the user device110exchanges data with the social networking system130, allowing modification of the data communicated from the social networking system130to the user device110. Use of the cookie114in exchanging data between the user device110, the social networking system130and/or the third-party website120is further described below in conjunction withFIG. 3.

The third-party website120comprises one or more web servers including one or more web page advertisements122, which are communicated to the user device110using the network140. The third-party website120is separate from the social networking system130. For example, the third-party website120is associated with a first domain while the social networking website is associated with a separate social networking domain. A web page advertisement122included in the third-party website120comprises a markup language document identifying content and including instructions specifying formatting or presentation of the identified content, as described above. In one embodiment, a web page advertisement122includes a tracking pixel124comprising instructions that, when executed by a browser application112of a user device110, retrieve the user account identifier from the cookie114or redirects the browser112to the social networking system130, using the client device identifier to determine an associated user identification from the social networking system130. The user account identifier, whether provided to the social networking system directly or derived from the social networking system cookie114, provides a mechanism for the social networking system130to associate the web page advertisement122with the user. Hence, the tracking pixel124allows a web page advertisement122from the third party website120to access a user identifier associated with a user of the client device110for associating the advertising content with the user account identifier by the social networking system130, which then stores the association in a log.

The social networking system130comprises one or more computing devices storing a social network, or mapping of a social graph, comprising a plurality of users and providing users of the social network with the ability to communicate and interact with other users of the social network. The social networking system130is further described below in conjunction withFIG. 2. In use, users of the social networking system130add connections to a number of other users of the social networking system130to whom they desire to be connected. As used herein, the term “friend” or the term “a connection” refers to any other user of the social networking system130to whom a user has formed a connection, association, or relationship via the social networking system130. As indicated above, the system100alternatively could use a user registration site of a different type to provide the functionality described herein using the example of a social networking system.

In addition to establishing and maintaining connections between users and allowing interactions between users, the social networking system130provides users with the ability to take actions on various types of items supported by the social networking system130. These items may include groups or networks (where “networks” here refer not to physical communication networks, but rather social networks of people, entities, and concepts) to which users of the social networking system may belong, events or calendar entries in which a user might be interested, computer-based applications that a user may use via the social networking system130, transactions that allow users to buy or sell items via the service, and interactions with advertisements that a user may perform on or off the social networking system. These are just a few examples of the items upon which a user may act on a social networking system, and many others are possible. A user may interact with anything that is capable of being represented in the social networking system130or in a third-party website120, separate from the social networking system130, coupled to the social networking system130via a network140.

The social networking system130is also capable of connecting a variety of entities. For example, the social networking system130enables users to interact with each other as well as third-party websites120or other entities through an API or other communication channels.

The social networking system130also includes user-generated content, which enhances a user's interactions with the social networking system130. User-generated content may include anything a user can add, upload, send, or “post,” to the social networking system130. For example, a user communicates posts to the social networking system130from a user device110. Posts may include data such as status updates or other textual data, location information, photos, videos, links, music or other similar data, content and/or media. Content may also be added to the social networking system130by a third-party through a “communication channel,” such as a newsfeed or stream. Content “items” represent single pieces of content that are represented as objects in the social networking system130. In this way, users of the social networking system130are encouraged to communicate with each other by posting text and content items of various types of media through various communication channels, increasing the interaction of users with each other and increasing the frequency with which users interact within the social networking system130.

FIG. 2is a diagram of one embodiment of a social networking system130. The embodiment of a social networking website130shown byFIG. 2includes a web server210, an action logger215, an API request server220, a control group generator225, an action log230, an ad exposure log234, an authorization server235, a user account store240, a connection store245, and a demographics and behavioral information store250. In other embodiments, the social networking website130may include additional, fewer, or different modules for various applications. Conventional components such as network interfaces, security mechanisms, load balancers, failover servers, management and network operations consoles, and the like are not shown so as to not obscure the details of the system.

As described above in conjunction withFIG. 1, the social networking system130comprises a computing system that allows users to communicate or otherwise interact with each other and access content as described herein. The social networking system130stores user profiles describing the users of a social network. The user profiles include biographic, demographic, and other types of descriptive information, such as work experience, educational history, hobbies or preferences, interests, location, and the like. The social networking system130further stores data describing one or more connections between different users. The connection information may indicate users who have similar or common work experience, group memberships, hobbies, educational history, or are in any way related or share common attributes. Additionally, the social networking system130includes user-defined connections between different users, allowing users to specify their relationships with other users. For example, user-defined connections allow users to generate relationships with other users that parallel the users' real-life relationships, such as friends, co-workers, partners, and so forth. Users may select from predefined types of connections, or define their own connection types as needed.

The web server210links the social networking system to one or more client devices110and/or one or more third-party websites130via the network140. The web server210serves web pages, as well as other web-related content, such as Java, Flash, XML, and so forth. The web server210may include a mail server or other messaging functionality for receiving and routing messages between the social networking system130and one or more client devices110. The messages can be instant messages, queued messages (e.g., email), text and SMS messages, or any other suitable messaging format.

The Application Programming Interface (API) request server220allows one or more third-party websites120to access information from the social networking system130by calling one or more APIs. The API request server220also may allow third-party websites120to send information to social networking website by calling APIs. For example, a third-party website120sends an API request to the social networking system130via the network140and the API request server220receives the API request. The API request server220processes the request by calling an API associated with the API request to generate an appropriate response, which the API request server220communicates to the third-party website120via the network140. For example, responsive to an API request, the API request server220collects data associated with a user and communicates the collected data to the third-party website120. The data communicated includes survey result data according to one embodiment. In some instances, the API request server220may create and provide information for a third-party website120to include a tracking pixel124as described herein within a web page advertisement122provided on the third-party website120.

The action logger215is capable of receiving communications from the web server210about user actions on and/or off the social networking system130. The action logger215populates the action log230with information about user actions, allowing the social networking system130to track various actions taken by its users within the social networking system130and outside of the social networking system130. Any action that a particular user takes with respect to another user is associated with each user's profile, through information maintained in the action log230or in a similar database or other data repository. Examples of actions taken by a user within the social network130that are identified and stored may include, for example, adding a connection to another user, sending a message to another user, reading a message from another user, viewing content associated with another user, attending an event posted by another user or other actions interacting with another user. When a user takes an action within the social networking system130, the action is recorded in an action log230. In one embodiment, the social networking system maintains the action log230as a database of entries. When an action is taken within the social networking system130, an entry for the action is added to the action log230.

Additionally, user actions may be associated with an entity outside of the core social networking system130, such as a third-party website120that is separate from the social networking system website130. For example, the action logger215receives data describing a user's interaction with a third party website120from the web server210. Examples of actions where a user interacts with a third-party website120includes a user expressing an interest in a third-party website120or another entity, a user posting a comment to the social networking system130that discusses a third-party website120, or a web page122within the third-party website120, a user posting to the social networking system130a Uniform Resource Locator (URL) or other identifier associated with a third-party website120, a user attending an event associated with a third-party website120or any other action by a user that is related to a third-party website120. Thus, the action log240may include actions describing interactions between a social networking system user and a third-party website120that is separate from the social networking system130.

Some actions taken with respect to a third-party website120separate from the social networking system130are logged separately according to one embodiment. For example, an ad exposure log234includes information associated with user exposure to a web page advertisement122provided by a third-party120. When a user accesses a web page advertisement122and that exposure is reported to the social networking system130, the ad exposure log234records the user identifier, the advertisement identifier, and the time and date of the access. In addition, other information may be logged such as an identifier of the ad placement, an identifier of the ad creative, a friend count for the user, whether the user “liked” an associated brand page, user interests and interest categories, etc. Alternatively, the ad exposure log234may be included within the action log230. The ad exposure log234may be accessed by the control group generator225and other aspects of the social networking system130for using the data stored therein.

The privacy setting specification may be provided at different levels of granularity. For example, the privacy setting may identify specific information to be shared with other users. For example, the privacy setting identifies a work phone number or a specific set of related information, such as, personal information including profile photo, home phone number, and status. Alternatively, the privacy setting may apply to all the information associated with the user. The specification of the set of entities that can access particular information can also be specified at various levels of granularity. Various sets of entities with which information can be shared may include, for example, all friends of the user, all friends of friends, all applications, all third-party websites120, specific third-party websites120, or all external systems. One embodiment allows the specification of the set of entries comprise an enumeration of entities, for example, the user may select a list of third-party websites120that are allowed to access certain information. Another embodiment allows the specification to comprise a set of entities along with exceptions that are not allowed to access the information. For example, a user may allow all third-party websites120to access the user's work information but specify a list of third-party websites120that are not allowed to access the work information. Certain embodiments call the list of exceptions that are not allowed to access certain information a block list. External systems, such as third-party websites120, belonging to a block list specified by a user are blocked from accessing the information specified in the privacy setting. Note that the various combinations of granularity of specification of information and the granularity of specification of entities with which information is shared are possible, i.e., all personal information may be shared with friends whereas all work information may be shared with friends of friends.

The authorization server235contains logic to determine if certain information associated with a user can be accessed by a user's friends, third-party websites120and/or other applications and entities. For example, a third-party website120that attempts to access a user's comment about a URL associated with the third-party website120must get authorization from the authorization server235to access the user's work phone number. Based on the user's privacy settings, the authorization server235determines if another user, a third-party website120, an application or another entity is allowed to access information associated with the user, including information about actions taken by the user. For example, the authorization server235uses a users privacy setting to determine if the user's comment about a URL associated with the third-party website120can be accessed by the third-party website120. This enables a user's privacy setting to specify which other users, or other entities, are allowed to receive data about the user's actions or other data associated with the user.

The control group generator225is a means for building advertising control groups within the social networking system130according to one embodiment. The control group generator225is configured to interact with other aspects of the social networking system130, such as the ad exposure log234and the demographic and behavioral information store250. The control group generator225uses information in the log files to determine which users have been exposed to a particular web page advertisement122, and can access demographic, behavioral, and/or information for exposed users, or can use the information to determine users who are similar to each other based upon their demographic, behavioral, and/or social information. The control group generator225can select subsets of users from all users, from exposed users, from users connected to the user, and from sets of users determined to be candidates for a control group of users not exposed to a particular web page advertisement122, either through random selection or by applying rules to determine the subset. In this way, the control group generator225can determine and candidate control group, and ultimately a control group of users, who were not exposed to a web page advertisement122that other users with similar characteristics were exposed to, to determine advertising effectiveness. For example, the control group generator225may select users who have connections to an exposed user as the candidate control group, for estimating any effect on those connections based on the user viewing the advertisement, e.g., by discussion of the advertisement between the user and the user's connections outside of the social networking system130.

The control group generator225also may generate surveys to issue to exposed and unexposed users.

The surveys or the user impression information itself can be used, for example, for retargeting the advertisement based on a user's past impressions for the advertisement, for richer reporting about an advertising campaign, or other modifications of the original advertisement.

Additionally, the social networking system130maintains data about objects with which a user may interact using the social networking system130. To maintain this data, the user account store240and the connection store245store instances of the corresponding type of objects maintained by the social networking system130. Each object type has information fields that are suitable for storing information appropriate to the type of object. For example, the user account store240contains data structures with fields suitable for describing a user's profile. When a new object of a particular type is created, the social networking system130initializes a new data structure of the corresponding type, assigns a unique object identifier to it, and begins to add data to the object as needed. This might occur, for example, when a user becomes a user of the social networking system130, the social networking system130generates a new instance of a user profile in the user account store240, assigns a unique identifier to the user profile, and begins to populate the fields of the user profile with information provided by the user.

The connection store245includes data structures suitable for describing a user's connections to other users, connections to third-party websites120or connections to other entities. The connection stores245may also associate a connection type with a user's connections, which may be used in conjunction with the user's privacy setting, further described above, to regulate access to information about the user. In addition, the connection store245may be accessed by other aspects of the social networking system130, e.g., the control group generator225, to obtain information about a user's connections for determining any effect of an advertisement on the connections of a user exposed to the advertisement.

The demographic and behavioral information store250includes data structures suitable for describing a user's demographic data, behavioral data, and other social data, which may be mined from the other logs230,234and stores240,245of the social networking system130. Demographic data typically includes data about the user, such as age, gender, location, etc., e.g., as included in the user's profile. Behavioral data typically includes information about the user's activities within the social networking system130, such as specific actions (posts, likes, comments, etc.), activity levels, usage statistics, etc. Other social data comprises information about the user from within the social networking system130that isn't demographic or behavioral, such as interests or affinities, whether a user is trusted by the social networking system130, etc. The demographic and behavioral information store250data can be accessed by the control group creator225or other aspects of the social networking system130.

Determining Effectiveness of Advertisements External to the Social Networking System

In order to assess effectiveness of advertisements outside the domain of the social networking system130or user registration site, tracking pixels124are embedded into advertisements on web pages122provided by a third party120. The information for embedding the tracking pixel124is received by the third party120from the social networking system130user registration site. Each time a client device110accesses a web page advertisement122including a tracking pixel124, the tracking pixel124enables the social networking system130or user registration site to log access to the web page advertisement122by the user associated with the device110. In one embodiment, the tracking pixel124accesses a cookie114stored on the browser112by the social networking system130or user registration site, from which is retrieves a user identifier associated with the user of the client device110and sends the identifier to the social networking system130or user registration site along with identification of the web page advertisement122. The social networking system130or user registration site logs access to the web page advertisement122by the user associated with the user identifier. According to another embodiment, upon download of the web page advertisement122, the tracking pixel124requests the image associated with the tracking pixel124from the social networking system130or user registration site, and sends with the request a cookie114stored on the browser112by the social networking system130or user registration site; the social networking system130or user registration site responds with the image to complete rendering of the web. The social networking system130or user registration site determines the user account associated with the cookie114, and logs access by the user to the web page advertisement122. From the log files, the social networking system130or user registration site determines users exposed to the web page advertisement122, and those not exposed, but with similar demographics and/or behavior information. The social networking system130or user registration site can use these two groups to provide surveys about the advertisement, with the unexposed group acting as a control group. The surveys may be provided by the social networking system130or user registration site, or by the third party120.

FIG. 3Aare interaction diagrams of embodiments of a process300a,300bfor a social networking system130building log files tracking access to a web page advertisement122by users of a user registration site, e.g., the social networking system130. In the embodiment shown byFIG. 3A, the third-party website120is separate from the social networking system130. For example, the third-party website120may be a separate and distinct domain from the domain of the social networking system130. Initially, the social networking system130maintains305user accounts for a plurality of users of the social networking system130, and user demographic and behavioral information, as discussed in conjunction withFIG. 2. Upon access to the social networking system130by the user via a client device110, the social networking system130sends310a cookie114to the device110, which includes one or more parameters, e.g., a user identification associated with the user account. The cookie114may be sent upon the client device110first accessing the social networking system130, or may be sent upon the first access by the client device110during a user session.

In one embodiment, the existence of cookie114in the browser112indicates that the user of the computing device110is a user of the social networking system130(e.g., whether the user has a valid user account with the social networking system130). If the user of the computing device110is a user of the social networking system130, the cookie114may contain information indicating whether the user is logged into the social networking system130(e.g., whether the user has a current valid session with the social networking system130). Thus, the cookie114may be used by the social networking system130to authenticate the user session. In other embodiments, the social networking system130may determine only whether the user is a user of the social networking system130, e.g., by checking for an existence of a cookie, without determining whether the user is logged into the social networking system130. Although cookies are described herein for authenticating the user and/or the user session, any other methods of user or session identification or authentication may be used (such as recognizing a physical token).

At a later time (as indicated by wavy lines), a user device110requests315a web page advertisement122from a third-party website120. For example, a user of the client device110enters a uniform resource locator (URL) or other identifier associated with the web page advertisement122into a browser application112operating on the user device110, e.g., by typing in the URL or selecting a link to the page122. The browser application112identifies the third-party website120associated with the received URL or other identifier and requests315the web page advertisement122associated with the received URL or other identifier from the identified third-party website120.

After receiving the request for the web page advertisement122, the third-party website120generates320the requested web page advertisement122. The third-party website120includes in the web page advertisement122a tracking pixel124for providing tracking information about users of the social networking system130. The tracking pixel124is an embedded object that is an invisible single pixel gif which, when executed by the browser application112of the client device110, retrieves information from the cookie114for providing to the social networking system130. For example, the tracking pixel124may embed using a <img src=“GOTO” width=“1” height=“1”/> tag, where GOTO represents a URL or instructions, or both, and 1×1 is the size of the tracking pixel124(i.e., it is invisible to the human eye). The third-party website120generates320a markup language document116describing the content and formatting of the web page advertisement122, which includes the tracking pixel124, and which is sent325to the client device110for rendering330by the web browser112. The tracking pixel124reads335the cookie114and sends340an identifier associated with the user of the client device110to the social networking system130along with identification of the web page advertisement122. The social networking system130then creates370a log entry for the identifier, indicating that the user associated with the identifier has viewed the web page advertisement122.

In the embodiment shown inFIG. 3B, steps305through325are the same as described in conjunction withFIG. 3A. However, the tracking pixel124is different. In this example, the tracking pixel124is an embedded object that is an invisible single pixel gif, for which the associated image is hosted on the social networking system130. For example, the tracking pixel124includes a URL, or other web page identifier, associated with the social networking system130and the location of the image associated with the tracking pixel124within the social networking system130. Thus, when the web page advertisement112is rendered330by the browser112, a request350is sent to the social networking system130for the image, and the cookie114previously stored by the browser112is sent to the social networking system130unchanged as part of the request, along with an identifier of the web page advertisement122. The social networking system130responds355to the request with the (invisible) pixel image, and the browser112completes the rendering330. From the cookie114, the social networking system130determines360the associated user, and logs370an entry indicating that the user has accessed the web page advertisement122.

Thus, for each user exposed to the web page advertisement122, a log entry is created identifying the web page advertisement122and the user. The user is identified by a user account in the social networking system130, which is either received directly from the client device110(e.g., because the tracking pixel124has read it from the cookie114;FIG. 3A) or is ascertained from the cookie114received from the client device110(FIG. 3B). The user associated with the client device110is the user logged into the social networking system130at the time the web page advertisement122is accessed, or if no user is logged in, the user most recently logged into the social networking system130via the client device110.

Next, the social networking system130builds a control group of users of the social networking system130not exposed to the web page advertisement122from the log files. An exposed subset of users of the social networking system130is determined based on the log entries of advertising exposures, and a candidate control group of users is generated by comparison of the demographic and behavior information between selected users of the social networking system130and the exposed subset of users. Any users from the candidate control group exposed to the web page advertisement122are eliminated, yielding a control group of users not exposed to the web page advertisement122.

FIG. 4is an interaction diagram of one embodiment of a process400for building an advertising control group within the social networking system130or user registration site. Initially, the social networking system130maintains405demographic and behavior information for each user, e.g., as described in conjunction with the D/B info store250, and maintains log files of exposures to web page advertisements122, e.g., in ad exposure log234. Sometime later, as indicated by wavy lines, the social networking system130, e.g., via control group generator225, determines users exposed to the advertisement by retrieving415exposed user information from the log files. From the exposed users, the social networking system130selects420a subset of the exposed users, e.g., randomly or by any other method. The social networking system130then retrieves425user demographics and behavioral information for the subset of exposed users. Alternatively, steps420and425could be reversed: the social networking system130could retrieve425the demographics and behavioral information for all exposed users, and then select420a subset of the users for comparison. In some cases the subset may be the whole.

Next, the demographics and behavioral information for the subset of exposed users is used to determine a candidate group of unexposed users with similar information. First, using the stored demographic and behavior information, the social networking system130retrieves430a list of users with similar information to those exposed to the advertisement, from which a candidate control group can be determined435. However, because the comparison is to users of the social networking system130generally, the candidate group might include some users who have in fact been exposed to the advertisement. Thus, the social networking system130then retrieves440information about whether any of the candidate control group have been exposed to the advertisement, who are removed from the group to create445the control group of unexposed users. The exposed and control groups then can be surveyed to determine the effectiveness of the advertisement.

In another embodiment, the exposure information can be used to retarget advertisements based on whether a user has already been exposed to the advertisement, or based on the number of impressions for the user. For example, an advertiser can use the advertisement exposure information to determine whether to serve another advertisement to a particular user of the plurality of users. The determination can be based on various intentions of the advertiser, such as whether the advertiser wants to send the advertisement again to an exposed user, to send the advertisement only to unexposed users, or to provide a second related advertisement only to users who have been exposed to a first advertisement.

The advertising exposure information can also be used for measuring advertisement effectiveness on connections of exposed users, which may happen by discussion of the advertisement between the user and the user's connections within or outside of the social networking system. For example, in one embodiment, the system determines exposed users, as discussed above, and then surveys the effects on the friends of those exposed users, thereby measuring the indirect effects of an advertisement.

In another embodiment, instead of using surveys, the effect of the advertisements can be measured in terms of the actions of the exposed users within the social networking system that relate to the advertising to which the user was exposed. Such action may include, for example, “liking” or commenting on a product that is the subject of an advertisement to which the user was exposed. These actions provide another possible measure of the effectiveness of the advertisement.

In addition, the surveys or the user impression information itself can be used to provide richer reporting about an advertising campaign, or as a basis for other modifications of the original advertisement.

SUMMARY

Embodiments of the invention may also relate to a product that is produced by a computing process described herein. Such a product may comprise information resulting from a computing process, where the information is stored on a non-transitory, tangible computer-readable storage medium and may include any embodiment of a computer program product or other data combination described herein. For example, in one embodiment the computer-readable storage medium contains a markup language document for being rendered by a web browser application executing on a computer system. In this embodiment, the markup language document comprises information items encoded in a markup language comprising instructions for rendering information from a third-party website in a web browser application, as well as instructions to a web browser application to retrieve information associated with a user of a social networking system, as described herein.