Capturing a cluster effect with targeted digital-content exposures

This disclosure covers methods, non-transitory computer readable media, and systems that determine a cluster effect representing the impact that a user's digital-content exposure has on other users' conversion actions. The disclosed methods, non-transitory computer readable media, and systems detect the downloads, purchases, or other forms of consumption of a featured item that result from users within the same cluster viewing digital content featuring the item. In some embodiments, the methods, non-transitory computer readable media, and systems apply the cluster effect by, for example, generating a conversion report representing the cluster effect or by providing tools that exploit the cluster effect in distributing digital content.

BACKGROUND

Distributing digital content poses a unique set of challenges to tracking the connection between users' exposure to digital content and conversion actions. Such conversion actions may include the download, purchase, streaming or other consumption of an item featured in digital content. In contrast to movement in the physical world, a network user may navigate from one website to another to purchase or otherwise consume an item. Some software application interfaces offer the same navigation functionality.

Exposure to digital content is not, however, always or even often an isolated experience. A user may converse with other users concerning digital content offline. A user may also share digital content electronically sending a link (e.g., a “URL”) to another user, for example. These conversations and digital-sharing events can prompt other users to purchase or otherwise consume a featured item as part of an indirect conversion action. These indirect conversion actions have proven difficult (if not impossible) for conventional conversion-tracking techniques to detect.

To track the connection between exposure to digital content and conversion actions, certain websites, applications, and other digital media have implemented some network-specific techniques. For example, some web servers use cookies for a user's web browser to track a user navigating from one digital touchpoint to another digital touchpoint. The web server may, for instance, use a cookie to assign a unique identifier to the user's browser when the browser requests a digital advertisement and then subsequently track the browser's navigation to other websites. As another example, some websites insert a transparent pixel tag into a web page and track a user's navigation from webpage to webpage. Such pixel tracking enables digital advertisers to track a user's navigation and consumption of advertised items. In some cases, application servers track in-application actions, such as purchases or reactions to posts, or track device identifiers as a user receives a digital advertisement and subsequently downloads an application or makes a purchase.

But these and other conventional conversion-tracking techniques have significant limits that fail to capture actions outside of a computer network as well as digital-sharing events that indirectly affect conversion. For instance, conventional cookies, pixel tracking, and in-application tracking do not detect conversations between network users concerning digital content. Nor do such conventional conversion-tracking techniques detect other interactions outside of the network that prompt users to consume or purchase an item featured in digital content.

Similarly, conventional conversion-tracking techniques do not track the connection between digital content displayed on a device that a user in turn shows to other users—resulting in another user's conversion action. Such conventional techniques cannot track the connection between a digital message, post, video, or other social-networking events that indirectly result in another user's conversion action. In sum, conventional conversion-tracking techniques are often user-specific and fail to capture the impact and reach that outside-network actions and digital-sharing activity have on conversion actions.

SUMMARY

This disclosure describes one or more embodiments of methods, non-transitory computer readable media, and systems that solve the foregoing problems in addition to providing other benefits. While this summary refers to systems for simplicity, the summary also applies to certain disclosed methods and non-transitory computer readable media. To solve the foregoing and other problems, the disclosed systems determine a cluster effect representing the impact that a user's digital-content exposure has on other users' conversion actions. The disclosed systems detect the downloads, purchases, or other forms of consumption of a featured item that result from users within the same cluster viewing digital content featuring the item. The systems also optionally apply the cluster effect by, for example, generating a conversion report representing the cluster effect or by providing tools that exploit the cluster effect in distributing digital content.

In some embodiments, for instance, the systems generate clusters of users, where each cluster has one or more users. The systems then separate the clusters of users into two groups—that is, a first group of clusters and a second group of clusters. The systems subsequently determine incremental lifts for different trial groups. For example, the systems determine a first incremental lift between a first test group and a first control group, where each such group comprises clusters of users from the first group of clusters. The systems also determine a second incremental lift between a second test group and a second control group, where each such group comprises individual users from the second group of clusters. By comparing the first incremental lift to the second incremental lift, the systems determine a cluster effect.

The disclosed systems overcome the limits of conventional conversion-tracking techniques. By determining an incremental lift for trial groups organized by either clusters of users or by individual users, the disclosed systems detect any differences in conversion that a digital-content delivery may have on multiple users within a cluster. Relatedly, by determining a cluster effect representing the difference between compared incremental lifts, the disclosed systems indirectly detect the influence that a user's outside-network actions may have on other users' purchases or consumption. In determining such a cluster effect, the disclosed systems indirectly detect the impact a digital message, post, digital video, or other digital-sharing activity has on converting digital-content delivery into a purchase or other form of digital consumption.

DETAILED DESCRIPTION

This disclosure describes a social networking system that determines a cluster effect representing the impact that users' digital-content exposure has on other users' conversion actions. The disclosed social networking system detects downloads, purchases, or other forms of consumption of a featured item that result from users within the same cluster viewing digital content featuring the item. The social networking system optionally applies the cluster effect by, for example, generating a conversion report representing the cluster effect or by providing tools that exploit the cluster effect in distributing digital content. Such tools optionally adjust the distribution of digital content to users within target clusters based on the cluster effect.

In some embodiments, for instance, the social networking system generates clusters of users, where each cluster has one or more users. The social networking system then separates the clusters of users into two groups—that is, a first group of clusters and a second group of clusters. The social networking system subsequently determines incremental lifts for different trial groups. For example, the social networking system determines a first incremental lift between a first test group and a first control group, where each such group comprises clusters of users from the first group of clusters. The social networking system also determines a second incremental lift between a second test group and a second control group, where each such group comprises individual users from the second group of clusters. By comparing the first incremental lift to the second incremental lift, the social networking system determines a cluster effect.

To generate clusters of users, the social networking system optionally applies a clustering algorithm to identify clusters based on common attributes shared among users. The clustering algorithm may, for example, identify clusters of users based on designated relationships within the social networking system, commonly used Internet Protocol addresses, common device identifiers, or some other common attribute that indicates a cluster. Such common attributes indicate to the social networking system that certain users are part of a cluster, such as a household, a group of classmates, or a group of coworkers.

The clusters may include differing numbers of users. In some cases, a cluster includes multiple users that share common attributes. In other cases, a cluster includes a single user with whom the clustering algorithm has not grouped other users. Regardless of the number of users in a cluster, the clustering algorithm optionally creates such clusters by assigning each user a cluster identifier. In some such embodiments, the social networking system uses both the cluster identifier and a user identifier assigned to each user to sort users into groups. Accordingly, each user may have both a user identifier and a cluster identifier. Whereas users in the same cluster share a common cluster identifier, each user has a different user identifier.

After generating clusters of users, in some embodiments, the social networking system creates trial groups to facilitate determining incremental lift. In some cases, the incremental lift represents the difference between (a) the consumption of a featured item by users to whom the social networking system delivered digital content (e.g., a digital advertisement) and (b) the consumption of the featured item by users to whom the social networking system did not deliver the digital content. Consumption by users who viewed the digital content represents a conversion action, that is, a consumption that results from or is linked to exposure to digital content.

To create the trial groups, the social networking system sorts users into two groups by clusters. For example, the social networking system randomly sorts clusters of users into a first group and a second group for analysis. As part of that analysis, the social networking system sorts clusters from the first group into a first test group and a first control group. By contrast, the social networking system sorts users (not clusters) from the second group into a second test group and a second control group. The social networking system then delivers digital content to users in the test groups, but not to users in the control groups.

In some embodiments, the social networking system creates trial groups by sorting users according to cluster identifiers or by user identifiers. For example, in certain embodiments, the social networking system sorts users according to their cluster identifiers to form the first group and the second group, as well as to form the first test group and the first control group. By contrast, the social networking system optionally sorts users according to their user identifiers to form the second test group and the second control group.

As noted above, the social networking system uses the trial groups to determine an incremental lift for both users sorted as clusters and users sorted as users. The social networking system then compares the incremental lifts to determine a cluster effect. For example, the social networking system may measure the incremental lift between the first test group and the first control group in terms of the percentage of incremental buyers of a product in the first test group compared to buyers of the product in the first control group. Similarly, the social networking system may measure the incremental lift between the second test group and the second control group in terms of the percentage of incremental buyers of a product in the second test group compared to buyers of the product in the second control group. As explained below, the social networking system may use many other measures of incremental lift.

Regardless of the measure for incremental lift, the social networking system uses incremental lifts to determine a cluster effect. For example, the social networking system determines a cluster effect as the divergence between compared incremental lifts. That cluster effect represents the impact users with digital-content exposure have on other users in the same cluster. For example, the social networking system may determine a cluster effect as the difference between (a) the percentage of incremental buyers for the first test-and-control groups and (b) the percentage of incremental buyers for the second test-and-control groups. Again, the social networking system may use several other measures to determine a cluster effect, and the foregoing examples are merely illustrative.

In addition to determining a cluster effect, in some embodiments, the disclosed social networking system evaluates different clustering algorithms' capture of a cluster effect. As explained below, the social networking system applies different clustering algorithms to users to create both clusters of users (from one clustering algorithm) and alternative clusters of users (from another clustering algorithm). To compare the different algorithms, the social networking system sorts super clusters into trial groups, where the super clusters may include overlapping conglomerations of both clusters and alternative clusters. The social networking system then compares incremental lift attributed to clusters formed by the one clustering algorithm to incremental lift attributed to the alternative clusters formed by the other clustering algorithm. By performing this comparison, the social networking system determines how different clustering algorithms capture a cluster effect and which clustering algorithms better represent clusters, such as by identifying those algorithms that more accurately represent households, a group of coworkers, or a group of classmates.

Beyond determining a cluster effect and evaluating clustering algorithms, the disclosed social networking system also applies the determined cluster effect. For example, in some embodiments, the social networking system generates a conversion report representing the cluster effect in various metrics. Additionally, or alternatively, the social networking system provides target-cluster options that exploit the cluster effect in distributing digital content. Such tools optionally adjust the distribution of digital content to users within target clusters by location, demographic, institution, or some other variable detected as part of the cluster effect.

As noted above, the disclosed social networking system overcomes the limits of conventional conversion-tracking techniques. Pixel tracking, in-application tracking, and other conventional conversion-tracking techniques do not track the connection between a user's out-of-network conversation or digital-sharing event and another user's conversion action. By determining an incremental lift for trial groups organized by either clusters or by individual users, the disclosed social networking system detects a cluster effect, such as differences in conversion that a digital-content delivery may have on multiple users within a cluster. In addition to detecting the cluster effect, the disclosed social networking system exploits the effect by providing target-cluster options.

In addition to failing to detect a cluster effect, pixel tracking, in-application tracking, and other conventional conversion-tracking techniques do not track the connection between digital content on a single device, on the one hand, and another user's conversion action as a result of that digital content, on the other hand. By determining a cluster effect representing the divergence between compared incremental lifts, the disclosed system detects the influence digital content on one device may have on multiple users in terms of conversion actions. Relatedly, by determining and applying the cluster effect, the social networking system further detects and exploits the influence an in-person conversation or social-networking activity has on converting digital-content delivery into a purchase or other form of digital consumption.

Turning now to the figures,FIG. 1illustrates a block diagram of one embodiment of a system environment100in which a social networking system104operates. As illustrated inFIG. 1, the system environment100includes server device(s)102, which include the social networking system104. The system environment100further includes client devices108a,108b, through108n. As depicted inFIG. 1, each of the client devices108a-108nhave an associated user—with a user112aassociated with the client device108a, a user112bassociated with the client device108b, and a user112nassociated with the client device108n. The users112a-112nmay be humans, organizations, organization members, or any other entities. AlthoughFIG. 1illustrates a particular number of client devices108a-108nand a respective number of associated users112a-112n, the system environment100may include any number of client devices and any number of users. Indeed, in some embodiments, the client device108a-108nand the users112a-112ncorrespond to thousands, millions, or billions of users.

As illustrated inFIG. 1, the client devices108a-108ncan communicate with the server device(s)102, including the social networking system104, over a network106. This disclosure describes additional details related to the social networking system104below with reference toFIGS. 8-9. In addition, the network106may represent a network or a collection of networks, such as the Internet, a corporate intranet, a local area network (“LAN”), or a combination of two or more such networks. The network106may be any suitable network over which the client devices108a-108n(or other components) access the social networking system104(or vice versa), as described further below with reference toFIG. 8.

As described below, the server device(s)102can enable the various functions, features, processes, methods, and systems described herein using, for example, the social networking system104. Additionally, or alternatively, the server device(s)102coordinate with the client devices108a-108nto perform or provide the various functions, features, processes, methods, and systems described in more detail below. AlthoughFIG. 1illustrates a particular arrangement of the server device(s)102, social networking system104, network106, and client devices108a-108n, various additional arrangements are possible. For example, the social networking system104and the server device(s)102may directly communicate with the client devices108a-108nand thus bypass the network106.

Generally, the client devices108a-108ncan include any one of various types of client devices. For example, the client devices108a-108ncan include a mobile device (e.g., a smart phone), tablet, laptop computer, desktop computer, television, or any other type of computing device as further explained below with reference toFIG. 8. Additionally, the server device(s)102can include one or more computing devices including those explained below with reference toFIG. 8. The server device(s)102, social networking system104, network106, and client devices108a-108nmay communicate using any communication platforms and technologies suitable for transporting data and/or communication signals, including any known communication technologies, devices, media, and protocols supportive of data communications, examples of which are described below with reference toFIG. 9.

As shown inFIG. 1, the server device(s)102provide the client devices108a-108naccess to the social networking system104through the network106. In one or more embodiments, by accessing the social networking system104, the client devices108a-108nsend or receive one or more digital content (e.g., posts, instant messages, comments) to or from the social networking system104, including digital advertisements that feature a product. The social networking system104can provide, for example, a website that enables one of the users112a-112nto receive, post, purchase, send, or perform other actions with respect to digital content within the social networking system104.

Alternatively, the client devices108a-108ncommunicate with the social networking system104via a dedicated application on the client devices108a-108n. In particular, and as further shown inFIG. 1, each of the client devices108a-108nhas an associated social networking application—with a social networking application110aassociated with the client device108a, a social networking application110bassociated with the client device108b, and a social networking application110nassociated with the client device108n. In some instances, the social networking system104provides data packets comprising the social networking applications110a-110nto the client devices108a-108n, respectively (e.g., by providing data to a mobile device that, when executed, creates a software application on the client devices108a-108n).

In some embodiments, the social networking application110a, for instance, comprises a web browser, applet, or other software application (e.g., a native application) available to the client device108a. The client device108amay launch the social networking application110ato facilitate interacting with the social networking system104. For example, the social networking application110amay coordinate communications between the client device108aand the server device(s)102that, for example, access webpages of the social networking system104or send digital content within the system104. To facilitate user interaction with the social networking system104, the social networking application110aprovides a graphical user interface of the social networking system104; receives indications of interactions of the user112awith the graphical user interface; and performs various requests, queries, or responses to other user input. Although this paragraph uses the social networking application110aas an example, the social networking applications110b-110nmay comprise the same components and perform the same functions as those included and performed by the social networking application110a.

As an overview of the social networking system104, the system104generates clusters to determine a cluster effect based on the actions and interactions performed—and profile information entered—by the users112a-112n. For example, in some embodiments, the social networking system104generates clusters of users as part of determining a cluster effect. As used in this disclosure, the term “cluster” refers to a grouping of one or more users of the social networking system104. In some instances, a cluster includes a single user, such as a single user who lacks certain common attributes with other users as determined by a clustering algorithm. As shown inFIG. 1, for example, a dotted box surrounding the user112nindicates that the user112nis a single user within a cluster. In some instances, however, a cluster includes multiple users having one or more common attributes. As shown inFIG. 1, for example, a dotted box surrounding the users112aand112bindicates that the users112aand112bare part of a cluster.

Common attributes may include, but are not limited to, an affinity coefficient, a designated relationship, a commonly used Internet Protocol (“IP”) address, a common physical address, a common device identifier, tagged users within one or more shared images, a common employer, or a common educational institution. Such information may be entered as profile information for one or more of the users112a-112n. Accordingly, and by way of example, a cluster may include a group of alumni from the same educational institution, classmates from the same class or educational institution, club members from the same club, coworkers who share a same employer or joint venture, social-network friends who are associated by affinity coefficient, household members who are part of a household or live at a same location, or neighbors who live within proximity of each other (e.g., with physical addresses on the same street or same neighborhood).

The term “cluster effect” refers to the impact that one or more users' exposure to digital content has on conversion actions by other users in the same cluster. The term “conversion action” in turn refers to a targeted action a user performs in response to digital content. The targeted action may come in response to a digital advertisement or product placement within digital content. For example, a cluster effect includes the purchase or consumption of a product by a first household member as a consequence of the first household member showing a second household member a digital advertisement. Alternatively, the first household member may have sent a direct message to the second user, posted or streamed a digital video that the second user viewed, spoken to the second user, or performed some other activity that prompted the second user to purchase or consume the product.

As used in this disclosure, a cluster effect includes various forms of conversion actions, including, but not limited to, downloading a software application, viewing digital content, voting in an election (e.g., for a particular candidate), purchasing a product, or subscribing to a service (e.g., subscribing to access digital content, download a podcast, or receive a food basket). Accordingly, a cluster effect accounts for a conversion action by a first cluster member as a consequence of showing or otherwise sharing with a second cluster member digital content intended to promote the conversion action.

To determine such a cluster effect, the social networking system104creates various groups based on clusters of the users112a-112n. For example, the social networking system104sorts clusters of the users112a-112ninto a first group and a second group. The social networking system104then sorts clusters from the first group into a first test group and a first control group. By contrast, the social networking system104sorts users from the second group into a second test group and a second control group. In other words, the social networking system104sorts users from the first group by cluster and users from the second group by user.

As used in this disclosure, the term “test group” refers to a group of users who receive certain digital content from the social networking system104. In other words, the digital content serves as the variable that the social networking system104tests. For example, the social networking system104may add a digital advertisement into a newsfeed viewed by users in a test group.

In contrast to a test group, the term “control group” refers to a group of users who do not receive certain digital content or at least do not receive certain digital content from the social networking system104. In other words, users in a control group serve as a benchmark against which the effectiveness of digital content is measured by the social networking system104. Relatedly, the term “trial group” refers to a group separated from a sample of users. A trial group includes, but is not limited to, a first group, second group, first test group, first control group, second test group, or second control group.

Continuing the overview, after creating the test groups and control groups, the social networking system104determines an incremental lift between respective test and control groups. As used in this disclosure, the term “incremental lift” refers to the difference between the consumption or purchase by a test group of a featured item and the consumption or purchase by a control group of the featured item. For example, in some embodiments, an incremental lift represents the difference between (a) downloads by a test group of an advertised software application and (b) downloads by a control group of an advertised software application. As explained below in connection withFIG. 3A, the social networking system104measures incremental lift in a variety of forms.

Regardless of the form of measurement, the social networking system104compares incremental lifts to determine a cluster effect. For example, the social networking system104may compare (1) a first incremental lift between a first test group and a first control group to (2) a second incremental lift between a second test group and a second control group to determine a cluster effect. In some such embodiments, the cluster effect represents the divergence between the first incremental lift and the second incremental lift. In one such measurement, the social networking system104determines a cluster effect measured in terms of a percentage representing an incremental number of users who consumed or purchased an item as shown in the first incremental lift above or below the second incremental lift.

Turning now toFIG. 2, this figure illustrates a conceptual diagram of a cluster effect. As shown inFIG. 2, this particular cluster effect represents the impact that certain users' exposure to a digital advertisement has on other users within the same clusters as measured in conversion actions. Specifically, the cluster effect depicted inFIG. 2represents the additional purchases of an advertised product by some users that result from certain other users' exposure to a digital advertisement.

As shown inFIG. 2, the social networking system104groups users204a-204rinto clusters202a-202h. The clusters202a,202b,202f,202g, and202heach include multiple users. In particular, the cluster202aincludes the users204a,204b, and204c; the cluster202bincludes the users204dand204e; the cluster202fincludes the users204iand204j; the cluster202gincludes the users204k,204l,204m, and204n; and the cluster202hincludes the users204o,204p,204q, and2044. By contrast, the clusters202c,202d, and202eeach include a single user. In particular, the clusters202c,202d, and202erespectively include the users204f,204g, and204h.

For illustrative purposes, the clusters202a-202hrepresent different households. Accordingly, the clusters202a-202hrepresent an estimate by the social networking system104of which users form a household. Other forms of clusters could be used as well to demonstrate clusters, such as social-network friends, neighbors, or organizational members.

As further shown inFIG. 2, the social networking system104delivers digital advertisements206a,206b,206c,206d,206e,206f,206g, and206hto the users204b,204d,204e,204f,204g,204h,204jand204o, respectively. The digital advertisements206a-206hrepresent different instances of the same digital advertisement that advertises a product. The social networking system104delivers the digital advertisements206a-206has a post within a newsfeed to client devices associated with the users204b,204d,204e,204f,204g,204h,204jand204o. The digital advertisements206a-206hmay or may not prompt a user to purchase the advertised product. For example, the digital advertisements206band206gprompt the users204dand204hto respectively make purchases208band208dof the advertised product. By contrast, the digital advertisements206d,206e, and206fdo not prompt the users204f,204g, and204jto make purchases of the advertised product.

Other purchases inFIG. 2illustrate the cluster effect. The digital advertisements206aand206hindirectly prompt the users204cand204rto make purchases208aand208eof the advertised product. As illustrated inFIG. 2, the social networking system104delivers the digital advertisements206aand206hto the users204band204o. The users204band204oin turn perform some activity—such as speaking with a household member, sending a direct message, or posting or streaming a digital video—that prompts the users204cand204rto make the purchases208aand208e.

As further shown inFIG. 2, the purchases208aand208eare the result of a cluster effect. In particular, the purchases208aand208erepresent incremental purchases that one cluster member's exposure to digital content prompted another cluster member to make. As noted above, conventional conversion-tracking techniques cannot detect such a cluster effect. But the disclosed social networking system104performs an innovative technique for detecting such a cluster effect.

Turning now toFIGS. 3A and 3B, these figures provide an overview of embodiments of the social networking system104that determine a cluster effect representing the impact of users' exposure to digital content on other users' conversion actions. Specifically,FIG. 3Aillustrates a sequence of acts302-326that the social networking system104performs to determine and apply a cluster effect, such as identifying a sample of users, applying a clustering algorithm, determining a cluster effect, and generating cluster effect tools.FIG. 3Billustrates a sequence of acts328-354that the social networking system104performs to compare different clustering algorithms, such as identifying a sample of users, generating super clusters of users, and determining a cluster-effect differential.

Various components of the system environment100perform the acts302-326shown inFIG. 3Aor the acts328-354shown inFIG. 3B. In some embodiments, for example, the social networking system104comprises computer-executable instructions that cause the server device(s)102to perform one or more of the acts302-326or the acts328-354. Rather than repeatedly describe the instructions within the social networking system104as causing the server device(s)102to perform certain acts, this disclosure primarily describes the social networking system104as performing the acts302-326or the acts328-354as a shorthand for those relationships.

Turning back now toFIG. 3A, as shown, the social networking system104performs the act of identifying a sample of users. The social networking system104selects a sample of users using a variety of methods. In some embodiments, the social networking system104randomly identifies users until it has identified a statistically significant number of users to create multiple test-and-control groups. Additionally, or alternatively, the social networking system104identifiers users from within the social networking system104that comprise a representative sample of a target population (e.g., a representative demographic of a population in a target location).

When identifying a sample of users, the social networking system104optionally uses a target variance as a benchmark for identifying an adequate number of users to estimate a cluster effect. Additionally, or alternatively, the social networking system104optionally uses a confidence level to estimate the accuracy of results derived from the sample of users. Regardless of whether a target variance or confidence level is used, the social networking system104may use any well-known statistical sample equation to identify a sample of users within the system104, such as equations for estimation of a proportion, estimation of a mean, and stratified sample size.

In addition to identifying a sample of users, the social networking system104performs the act304of identifying common attributes from among the sample. The social networking system104identifies these common attributes from actions taken and information entered in the social networking system104by the users112a-112n. For example, in some embodiments, the social networking system104identifies one or more of designated relationships, common physical addresses, common employers, and/or common educational institutions in profile information associated with the sample of users. Additionally, or alternatively, the social networking system104identifies an IP address or device identifier used by two or more of users from the sample to access the social networking system104. The social networking system104also optionally identifies images, posts, or comments in which two or more of the users from the sample have been tagged or identifies groups which two or more of the users from the sample have joined.

In some embodiments, the social networking system104determines an affinity coefficient for some (or all) of the in the sample relative to each other as part of identifying common attributes. The social networking system104determines the affinity coefficient based on a social graph. In some embodiments, the affinity coefficient represents a strength of a relationship between a first user and a second user (e.g., users112aand112b). The affinity coefficient may also represent a probability or function that measures a probability that a first user and a second user are part of a same cluster (e.g., a group of social-network friends). In other words, the affinity coefficient may represent a strength of relationship between a user and a proposed cluster.

In some embodiments, the social networking system104generates or utilizes an affinity coefficient by quantifying an affinity between a first user and a second user-both of whom are represented as objects within a social graph. Specifically, within the social graph, the social networking system104represents each of the users112aand112bas a node. The social networking system104likewise represents various other users and objects associated with the users112aand112bas nodes within the social graph and connections among the user nodes and object nodes as edges within the social graph. The social graph may represent any of the common attributes noted above as a node. By using these nodes and edges, the social networking system104generates an affinity coefficient for any two users or between any user and a proposed cluster. As explained below, the social networking system104may group users within a cluster when they have an affinity coefficient above a threshold. This disclosure describes affinity coefficients in more detail with reference toFIG. 10below.

As further shown inFIG. 3A, the social networking system104performs the act306of applying a clustering algorithm to users of the social networking system104to generate clusters of users. In some embodiments, the social networking system104applies the clustering algorithm to the sample of users identified as part of the act302. By contrast, in some embodiments, the social networking system104applies the clustering algorithm as part of identifying the sample of users. In some such embodiments, the social networking system104applies the clustering algorithm to generate clusters to ensure that the sample of users does not split up users who are part of a same cluster—within or without the sample of users. To illustrate this relationship between identifying a sample of users and applying a clustering algorithm,FIG. 3Aincludes a dotted arrow from the act306to the act302.

When generating the clusters of users, the social networking system104may apply a variety of clustering algorithms. For example, in some embodiments, the social networking system104applies an affinity-propagation algorithm, a centroid-based clustering algorithm, a distribution-based clustering algorithm, or a density-based clustering algorithm. In particular, in some embodiments, the social networking system104applies any one of the methods or algorithms described by Jure Leskovec, Anand Rajaraman, and Jeffrey David Ullman, Ch. 10, “Mining Social-Network Graphs” in Mining of Massive Datasets 325-383 (2014) (hereinafter “Leskovec”), which is incorporated by reference in its entirety. In some such embodiments, the social networking system104uses a clustering algorithm that includes hash functions. For example, the social networking system104optionally applies MapReduce or the Flajolet-Martin technique when generating clusters of users, as described in Leskovec.

Additionally, or alternatively, in some embodiments, the social networking system104applies the clustering algorithm described by Nina Mishra, Robert Schreiber, Isabelle Stanton, and Robert E. Tarjan, “Finding Strongly Knit Clusters in Social Networks,” Internet Mathematics Vol. 5, No. 1-2: 153-172 (2009) (hereinafter “Mishra”), which is incorporated by reference in its entirety. Whether applying Mishra's clustering algorithm or some other clustering algorithm, the social networking system104optionally uses affinity coefficients as part of the clustering algorithm to generate clusters of users. In some such embodiments, the social networking system104determines an affinity coefficient for a user relative to a proposed cluster. For example, in some embodiments, the social networking system104proposes a set of possible clusters based on common attributes detected within the social networking system104. The social networking system104groups users into clusters based on an overall affinity coefficient among users (from a sample of users) to generate clusters of users. In one implementation, the social networking system104uses affinities as described in Mishra.

Regardless of the clustering algorithm, in some embodiments, the social networking system104generates clusters by assigning each user a cluster identifier. In other words, the social networking system104assigns a cluster identifier to each user representing the cluster in which the clustering algorithm determines the user belongs. The term “cluster identifier” refers to a code, combination, number, or other identifier for a particular cluster. A cluster identifier differs from a user identifier. The term “user identifier” refers to a code, combination, number, or other identifier for a particular user. As explained below, the social networking system104optionally uses both cluster identifiers and user identifiers to sort users into groups. In some embodiments, the social networking system104assigns each of the users112a-112na user identifier (e.g., assigning each user a unique user identifier upon opening an account in the social networking system104). Accordingly, after applying a clustering algorithm, each user may have both a user identifier and a cluster identifier.FIGS. 4A and 4Bprovide examples of cluster identifiers and user identifiers.

As further shown inFIG. 3A, after applying a clustering algorithm, the social networking system104performs the act308of assigning clusters of users to a first group and the act310of assigning clusters of users to a second group. For example, in some embodiments, the social networking system104randomly assigns clusters to the first group and the second group of clusters. To make that random assignment, the social networking system104optionally uses cluster identifiers to randomly sort the clusters of users into the first group or the second group. Accordingly, the social networking system104may randomly assign clusters of users to groups according to cluster identifiers.

Alternatively, in some embodiments, the social networking system104divides the clusters of users into the first and the second groups. For example, the social networking system104may divide the clusters of users such that a certain portion of the clusters are in the first group and a certain portion of the clusters are in the second group (e.g., evenly dividing the clusters of users into the first and second groups).

After sorting the clusters of users into the first and second groups, the social networking system104further sorts the users from each of the first and second groups into test-and-control groups. As shown inFIG. 3A, for instance, the social networking system104performs the act312of assigning some clusters to a first test group and the act314of assigning some clusters to a first control group. For example, the social networking system104may randomly assign particular clusters from the first group to the first test group or to the first control group according to cluster identifiers. In other words, the social networking system104randomly sorts the cluster identifiers assigned to each user having the same cluster identifier into either the first test group or the first control group. By sorting users according to cluster identifier, the social networking system104ensures that it does not split up users within a same cluster into a different group. Accordingly, when performing the acts312and314, the social networking system104avoids splitting one cluster member in a test group and another cluster member in a control group (or vice versa).

Alternatively, the social networking system104divides the clusters from the first group into a certain portion of clusters for the first test group and a certain portion of clusters for the first control group. For example, the social networking system104may divide the clusters from the first group evenly into the first test group and the first control group. Similarly, the social networking system104may divide the clusters from the first group into some other predetermined portions (e.g., two thirds in the first test group and one third in the second test group).

As further shown inFIG. 3A, the social networking system104performs the act316of assigning some users to a second test group and the act318of assigning some users to a second control group. For example, the social networking system104may randomly assign users from the second group to the second test group or to the second control group according to user identifiers. In other words, the social networking system104randomly sorts the user identifiers assigned to each user into either the second test group or the second control group. By sorting users according to user identifier, the social networking system104potentially splits users belonging to a same cluster into different groups (e.g., by splitting one cluster member in a test group and another cluster member in a control group).

Alternatively, the social networking system104divides the users from the second group into a certain portion of users for the second test group and a certain portion of users for the second control group. For example, the social networking system104may divide the users from the second group evenly into the second test group and the second control group or, alternatively, into some other predetermined portions.

After sorting users into test-and-control groups, the social networking system104performs the act320of determining a first incremental lift and the act322of determining a second incremental lift. As suggested above, the social networking system104delivers digital content to the first test group and to the second test group, but not to the first control group and not to the second control group. After delivering digital content to the test groups, the social networking system104determines which users within each of the test-and-control groups performed a conversion action, such as by downloading an advertised software application, purchasing an advertised product, streaming advertised digital content, or performing some other conversion action. This determination enables the social networking system104to in turn determine an incremental lift.

The social networking system104uses a variety of tools to determine which users performed a conversion action. For example, in some embodiments, the social networking system104generates a unique audience pixel for a website to securely track a user who visits a website and also visits webpages of the social networking system104. In some such embodiments, the social networking system104generates the unique audience pixel for a website administrator to add to the header code of a particular webpage for subsequent tracking. Additionally, or alternatively, the social networking system104generates a conversion pixel for addition to the HTML of a target webpage. For example the social networking system104generates a snippet of code for adding to the HTML of a purchase-confirmation webpage for an advertised item or a streaming-webpage for an advertised video. When a conversion action occurs, the conversion pixel sends a signal to the social networking system104indicating that the conversion action has occurred.

Additionally, in some embodiments, the social networking system104receives point-of-sale or point-of-transaction data from a third party who advertised a particular product to determine which users performed a conversion action. For example, the social networking system104optionally receives information from the third party concerning people who downloaded a software application, viewed digital content, purchased a product, subscribed to a service, or performed some other conversion action. The social networking system104then matches the information concerning people who performed a conversion action with profile or other information for users within the test-and-control groups.

While this disclosure describes audience pixels, conversion pixels, and point-of-transaction data, the social networking system104may determine which users performed conversion actions using any suitable method. Regardless of the method used, the social networking system104gathers information concerning the conversion actions. For example, the social networking system104gathers one or more of an amount spent by a user from the sample for an individual conversion action, a number of users from the sample who performed a conversion action, a total amount spent for conversion actions performed by users within the sample, and a total number of conversion actions performed by users within the sample. Additionally, the social networking system104optionally gathers such data and quantifies the data for each test and control group.

Using data concerning users' conversion actions, the social networking system104determines a first incremental lift between users in the first test group and the first control group. Similarly, the social networking system104determines a second incremental lift between users in the second test group and the second control group. The social networking system104determines incremental lift in a variety of measurements.

For example, in some instances, the social networking system104determines incremental lift in terms of (a) an incremental amount spent per user in a test group above or below the same measurement type in a control group, (b) a percentage representing an incremental amount spent per user in a test group above or below the same measurement type in a control group, (c) an incremental number of users who consumed or purchased an item in a test group above or below the same measurement type in a control group, (d) a percentage representing an incremental number of users who consumed or purchased an item in a test group above or below the same measurement type in a control group, (e) an incremental amount of sales generated by a test group above or below the same measurement type in a control group, and (f) a percentage representing an incremental amount of sales generated by a test group above or below the same measurement type in a control group. For simplicity, this disclosure refers to the foregoing measurements as (a) incremental-spend-amount lift, (b) average-spend lift, (c) incremental-consumption lift, (d) conversion lift, (e) incremental-sales lift, and (f) total-sales lift.FIG. 6below provides an example of incremental lift determined in measurements (a) through (f). While this disclosure describes and refers to incremental lift in terms of measurements (a) through (f), the social networking system104additionally or alternatively determines incremental lift in other measurements (e.g., an incremental number of users who voted in an election or for a particular candidate, an incremental number of subscriptions for a service).

As further shown inFIG. 3A, after determining first and second incremental lifts, the social networking system104performs the act324of determining a cluster effect. To determine the cluster effect, the social networking system104compares the first incremental lift to the second incremental lift. In some embodiments, the social networking system104determines the cluster effect by determining a divergence between the first incremental lift and the second incremental lift.

Because the first incremental lift captures incremental lift between test-and-control groups separated by clusters, the first incremental lift includes incremental consumption or purchase caused by a cluster effect. Conversely, because the second incremental lift captures incremental lift between test-and-control groups separated by users, the second incremental lift does not include incremental consumption or purchase caused by a cluster effect. Accordingly, by comparing the first incremental lift to the second incremental lift, the social networking system104determines a measurement of the cluster effect.

For example, in some embodiments, the social networking system104determines a cluster effect measured in terms of conversion lifts represented by the first incremental lift and the second incremental lift. By comparing the two conversion lifts, the social networking system104determines a percentage representing an incremental number of users who consumed or purchased an item as shown in the first incremental lift above or below the second incremental lift. While this example highlights conversion lift, the social networking system104may use incremental lifts in any of the measurements (a) through (f)—or other suitable measurements—to determine a cluster effect. As described below,FIG. 6provides additional examples.

As further shown inFIG. 3A, after determining a cluster effect, the social networking system104performs the act326of generating cluster effect tools. As suggested above, in some embodiments, the social networking system104provides cluster effect tools as one or both of conversion reports and target-cluster options. For example, the social networking system104generates a conversion report that includes a representation of the cluster effect and/or target-cluster options. After generation, the social networking system104provides the conversion report to a client device for presentation within a graphical user interface.

The conversion report may represent the cluster effect in terms of comparison of any of the measurements (a) through (f). Additionally, the conversion report optionally represents the cluster effect in more specific portions of a cluster. For instance, the conversion report may include a cluster effect for a particular age group, gender, location, relationship status, region, religion, or other demographic. Additionally, or alternatively, the conversion report may include a cluster effect for a particular time frame (e.g., time intervals between exposure to digital content and a corresponding conversion action).

Similarly, in some embodiments, the target-cluster options include selectable options that (upon selection) adjust the distribution of digital content to users within particular clusters. For example, the social networking system104may provide target-cluster options that increase or decrease the distribution of digital content to users within particular clusters, such as a video or digital advertisement. The social networking system104may, for instance, increase or decrease the distribution of digital content to users within clusters comprising multiple users or to users within clusters of a particular age group, gender, location, relationship status, region, religion, or other demographic. Accordingly, in some cases, the target-cluster options provide options to exploit information provided in the conversion report.

In addition to determining a cluster effect, in certain embodiments, the social networking system104compares different clustering algorithms to determine which clustering algorithm better or more accurately captures a cluster effect. In some cases, the social networking system104compares different clustering algorithms to determine which clustering algorithm generates clusters that reflect real-world clusters more accurately than the other clustering algorithm. Additionally, or alternatively, the social networking system104determines which of two or more clustering algorithms captures an increased cluster effect—that is, the clustering algorithm that captures a larger or more pronounced cluster effect. This disclosure describes some embodiments that compare clustering algorithms below with reference toFIG. 3B. But the social networking system104sometimes runs different iterations of the acts302-326to compare clustering algorithms.

In some embodiments, for instance, the social networking system104runs one iteration of the acts302-326with a first clustering algorithm and another iteration of the acts302-326with a second clustering algorithm. The social networking system104then compares the cluster effect determined in each iteration. In making that comparison, the social networking system104can make various determinations. The social networking system104may determine which of the clustering algorithms captures a larger cluster effect. Alternatively, by running several iterations with two different clustering algorithms, the social networking system104may determine which of the clustering algorithms captures a more consistent cluster effect (e.g., a cluster effect within a threshold range or statistically significant range). The social networking system104may similarly run iterations of the acts302-326with a different clustering algorithm in each iteration or different clustering algorithms in multiple iterations (e.g., three or four clustering algorithms with several iterations each).

As an illustrative example, in one or more embodiments, the social networking system104applies an alternative clustering algorithm to generate alternative clusters of users, where each alternative cluster has one or more users. This alternative clustering algorithm is a different clustering algorithm than the social networking system104used in an initial iteration or in several prior iterations. The social networking system104then separates the alternative clusters of users into two groups—that is, an alternative first group of clusters and an alternative second group of clusters.

The social networking system104subsequently determines incremental lifts for different trial groups. For example, the social networking system104determines an alternative first incremental lift between an alternative first test group and an alternative first control group, where each such group comprises alternative clusters of users from the first group of clusters. The social networking system104also determines an alternative second incremental lift between an alternative second test group and an alternative second control group, where each such group comprises individual users from the alternative second group of clusters. By comparing the alternative first incremental lift to the alternative second incremental lift, the social networking system104determines an alternative cluster effect. Based on a comparison of a cluster effect determined using an initial clustering algorithm and the alternative cluster effect determine using the alternative clustering algorithm, the social networking system104identifies either the initial clustering algorithm or the alternative clustering algorithm as generating clusters that capture a larger cluster effect.

Turning back now toFIG. 3B, this figure illustrates a representation of a sequence of acts328-354that the social networking system104performs in addition (or in the alternative) to the acts302-326shown inFIG. 3A. As additional or alternative acts,FIG. 3Billustrates the social networking system104comparing different clustering algorithms' capture of a cluster effect. By performing this comparison, the social networking system104determines how different clustering algorithms capture a cluster effect and which clustering algorithms better represent clusters.

As shown inFIG. 3B, the social networking system104performs the act328of identifying a sample of users and the act330of identifying common attributes. As suggested by their descriptions in the figures, the acts328and330inFIG. 3Brespectively correspond to the acts302and304inFIG. 3A. Accordingly, the description and embodiments set forth above for the acts302and304respectively apply to the acts328and330—except that the latter acts involve optionally involve applying different clustering algorithms as part of identifying a sample of users.

As further shown inFIG. 3B, the social networking system104performs the act332of applying a first clustering algorithm to users to generate clusters of users and the act334of applying a second clustering algorithm to users to generate alternative clusters of users. For purposes of comparison, the social networking system104applies the first and second clustering algorithms to the same group of users. In some embodiments, the social networking system104applies the first and second clustering algorithms to the sample of users identified as part of the act328. By contrast, in some embodiments, the social networking system104applies the first and second clustering algorithms as part of identifying the sample of users. To illustrate this relationship between identifying a sample of users and applying the different clustering algorithms,FIG. 3Bincludes dotted arrows from the act332to the act328and from the act332to the act328.

When applying the first and second clustering algorithms, the social networking system104applies different clustering algorithms. For example, the social networking system104optionally applies a clustering algorithm described in Leskovec and a clustering algorithm described in Mishra. Alternatively, the social networking system104optionally applies two different clustering algorithms described in Leskovec. As suggested above, the social networking system104may also apply any two different clustering algorithms suitable for a social network or social graph. Regardless of the clustering algorithm, in some embodiments, the social networking system104generates clusters by assigning each user a cluster identifier corresponding to the clusters generated by the first clustering algorithm and an alternative cluster identifier corresponding to the alternative cluster generated by the second clustering algorithm.

As the first and second clustering algorithms differ from one another, the first and second clustering algorithms presumably generate different clusters of algorithms. Accordingly, this disclosure describes the second clustering algorithm as generating alternative clusters of users—as in alternatives to the clusters of users generated by the first clustering algorithm. But in some cases, the first and second clustering algorithms may generate some clusters of users (or theoretically all clusters of users) that have the same constituent users. In other words, the first and second clustering algorithms may agree on the constituent users who are part of a cluster.

In many embodiments, however, some or all of the clusters of users generated by the first clustering algorithm differ from the alternative clusters of users. Despite those differences, the first clustering algorithm may generate one or more clusters of users comprising some users that overlap (or are also part of) an alternative cluster of users. In other words, the first and second clustering algorithms may generate overlapping clusters.

For example, the first clustering algorithm may generate the cluster202h(shown inFIG. 2) comprising the users204o,204p,204q, and204r. But the second clustering algorithm may generate an alternative cluster that comprises the users204iand204j—and the user204p. Accordingly, the cluster202hand the alternative cluster would both include the user204p, but otherwise include different users. As another example, the second clustering algorithm may generate an alternative cluster that comprises the users204o,204p,204q, and204r—and the user204i. Accordingly, in this example, the alternative cluster would subsume the users within the cluster202hand include an additional user, that is, the user204i.FIGS. 5A and 5Bbelow include additional examples of clusters and alternative clusters.

As further shown inFIG. 3B, after applying the first and second clustering algorithms, the social networking system104performs the act336of generating super clusters of users. As used in this disclosure, the term “super cluster” refers to a group of one or more clusters and one or more alternative clusters. Whereas the first clustering algorithm generates the one or more clusters, the second clustering algorithm generates the one or more alternative clusters. The social networking system104creates super clusters for later sorting clusters into trial groups without splitting a cluster and an alternative cluster that share an overlapping user. The super cluster helps ensure that a conversion action is attributed to a cluster effect based on either a cluster or an alternative cluster.

A super cluster may include, for example, both (a) the cluster202hcomprising the users204o,204p,204q, and204rand (b) an alternative cluster comprising the users204i,204j, and204p. In this example, the social networking system104groups the cluster202hand the alternative cluster into a super cluster because both clusters share an overlapping user, that is, the user204p. A super cluster may also include multiple clusters and/or multiple alternative clusters. For example, a super cluster may include (a) the cluster202hcomprising the users204o,204p,204q, and204r; (b) an alternative cluster comprising the users204i,204j, and204p; and (c) an additional alternative cluster comprising the users204hand204o. As above, the social networking system104groups the cluster202hand the two alternative clusters into a super cluster to ensure that the sorting process does not split a cluster.

Depending on the number and position overlapping users, the social networking system104may include several other possible super clusters with multiple clusters from a first clustering algorithm and multiple alternative clusters from a second clustering algorithm. By contrast, in some instance, a super cluster may include a cluster and an alternative cluster that perfectly overlap, that is, a cluster and an alternative cluster that each comprise the same users.FIGS. 5A and 5Bincludes additional examples of super clusters.

When generating super clusters, in some embodiments, the social networking system104creates super clusters that clusters and alternative clusters that share one or more overlapping users are not separated. In some cases, the social networking system104generates super clusters by assigning a super-cluster identifier to each user within a sample of users. In other words, the social networking system104assigns a super-cluster identifier to each user representing the super cluster in which the social networking system104determines the user belongs. The term “super-cluster identifier” refers to a code, combination, number, or other identifier for a particular super cluster.

As further shown inFIG. 3B, after generating super clusters, the social networking system104performs the act338of assigning super clusters to a first group and the act340of assigning super clusters to a second group. For example, in some embodiments, the social networking system104randomly assigns super clusters to the first group and the second group. To make that random assignment, the social networking system104optionally uses super-cluster identifiers to randomly sort the super clusters of users into the first group or the second group. Accordingly, the social networking system104may randomly assign super clusters based on super-cluster identifiers.

Alternatively, in some embodiments, the social networking system104divides the super clusters into the first and the second groups. For example, the social networking system104may divide the super clusters such that a certain portion of the super clusters are in the first group and a certain portion of the super clusters are in the second group (e.g., evenly dividing the super clusters into the first and second groups).

After sorting the super clusters into the first and second groups, the social networking system104further sorts the super clusters from each of the first and second groups into test-and-control groups. As shown inFIG. 3B, for instance, the social networking system104performs the act342of assigning some super clusters to a first test group and the act344of assigning some super clusters to a first control group. For example, the social networking system104may randomly assign super clusters from the first group to the first test group or to the first control group according to super-cluster identifiers. In other words, the social networking system104randomly sorts the super-cluster identifiers assigned to each user having the same super-cluster identifier into either the first test group or the first control group. By sorting super clusters by super-cluster identifier, the social networking system104ensures that it does not split up into different groups clusters and alternative clusters that share overlapping users.

Alternatively, the social networking system104divides the super clusters from the first group into a certain portion of super clusters for the first test group and a certain portion of super clusters for the first control group. For example, the social networking system104may divide the super clusters from the first group evenly into the first test group and the first control group. Similarly, the social networking system104may divide the super clusters from the first group into some other predetermined portions (e.g., two thirds in the first test group and one third in the second test group).

As further shown inFIG. 3B, the social networking system104performs the act346of assigning some super clusters to a second test group and the act348of assigning some super clusters to a second control group. As suggested by their descriptions in the figures, the acts346and348inFIG. 3Bare respectively similar to the acts328and330inFIG. 3B. Accordingly, the description and embodiments set forth above for the acts328and330respectively apply to the acts346and348—except that the latter acts involve sorting super clusters from the second group into a second test group and a second control group.

After sorting super clusters into test-and-control groups, the social networking system104performs the act350of determining a first incremental lift and the act352of determining a second incremental lift. As suggested by their descriptions in the figures, the acts350and352inFIG. 3Bare respectively similar to the acts320and322inFIG. 3A. Accordingly, the description and embodiments set forth above for the acts320and322respectively apply to the acts350and352with some exceptions. In contrast to the former acts, both the acts350and352involve determining that the latter acts involve sorting super clusters from the second group into a second test group and a second control group.

Additionally, both the first incremental lift and the second incremental lift shown inFIG. 3Breflect a cluster effect. Because both the first incremental lift and the second incremental lift capture incremental lift between test-and-control groups separated by super clusters, both the first incremental lift and the second incremental lift include incremental consumption or purchase potentially caused by a cluster effect. Accordingly, in contrast to the incremental lifts ofFIG. 3A, the incremental lifts ofFIG. 3Breflect incremental consumption or purchase accounted for by different clustering algorithms. In other words, the incremental lifts ofFIG. 3Bprovide a direct cluster-to-cluster comparison.

The incremental lifts are specific to a clustering algorithm. In some embodiments, for example, the social networking system104determines a first incremental lift that reflects conversion actions attributable to the clusters generated by the first clustering algorithm. Similarly, the social networking system104optionally determines a second incremental lift that reflects conversion actions attributable to the alternative clusters generated by the first clustering algorithm.

The social networking system104also compares the effects of the different clustering algorithms. As shown inFIG. 3B, the social networking system104performs the act354of determining a cluster-effect differential. To determine a cluster-effect differential, in some embodiments, the social networking system104compares the one clustering algorithm's incremental lift to another clustering algorithm's incremental lift. In other words, the social networking system104compares (a) the first incremental lift that reflects conversion actions attributable to the clusters generated by the first clustering algorithm to (b) the second incremental lift that reflects conversion actions attributable to the alternative clusters generated by the second clustering algorithm.

In some such embodiments, the social networking system104determines a cluster-effect differential by determining a difference between the first and second incremental lifts. In certain cases, the social networking system104identifies the clustering algorithm that accounts for the higher incremental lift as the preferable clustering algorithm. This preference is based on the assumption that the clustering algorithm that better accounts for incremental lift—or that results in a larger cluster effect—more likely represents communication between users within a cluster.

Turning now toFIG. 4, this figure illustrates a sample of users and trial groups in accordance with one or more embodiments of the acts described inFIG. 3A. As noted above, in some embodiments, the social networking system104identifies a sample of users and sorts those users into trial groups (e.g., by clusters or by users).FIG. 4depicts a sample of users402and various trial groups that the social networking system104identifies and creates when performing some of the acts302-326.

As shown inFIG. 4, the sample of users402includes a user identifier and a cluster identifier for each user within the sample. In this particular embodiment, the social networking system104applies a clustering algorithm as part of identifying the sample of users402and assigns cluster identifiers as part of generating the clusters of users. The user identifier for each user is shown in a user-identifier column404. Similarly, the cluster identifier for each user is shown in a cluster-identifier column406. For explanatory purposes, the sample of users402depicted inFIG. 4includes user identifiers and cluster identifiers corresponding to twelve users. These twelve users are merely illustrative. In additional or alternative embodiments, the sample of users402are part of a larger sample.

As further shown inFIG. 4, the social networking system104separates the clusters of users from the sample of users402into a first group408aand a second group408b. Consistent with the disclosure above, the social networking system104randomly assigns the users into the first group408aor the second group408baccording to cluster identifiers. In other words, the social networking system104sorts clusters of users by cluster identifier into either (but not both) the first group408aor the second group408b.

After separating the users from the sample of users402into the first group408aor the second group408b, the social networking system104separates clusters from the first group408ainto a first test group410aand a first control group412a. In particular, the social networking system104separates users based on cluster identifier to ensure that users having a same cluster identifier are assigned to either (but not both) the first test group410aor the first control group412a.

The social networking system104also separates users from the second group408binto a second test group410band a second control group412b. In particular, the social networking system104separates users based on user identifier. By separating the users according to user identifier—and not by cluster identifier—the social networking system104may split users who were part of a same cluster into different test or control groups. Specifically, social networking system104sorts the user corresponding to the user identifier1244and cluster identifier6003into the second test group410b. But social networking system104sorts the user corresponding to the user identifier1245and cluster identifier6003into the second control group412b. Both users share the cluster identifier6003.

As further shown inFIG. 4, each of the test groups include a corresponding advertisement column, purchase column, and conversion column. Each of the control groups include a corresponding purchase column and conversion column. In particular, an advertisement column414a, a purchase column416a, and a conversion column418acorrespond to the first test group410a; an advertisement column414b, a purchase column416c, and a conversion column418ccorrespond to the second test group410b; a purchase column416band a conversion column418bcorrespond to the first control group412a; and a purchase column416dand a conversion column418dcorrespond to the second control group412b.

The advertisement columns414aand414binclude marks indicating users to whom the social networking system104delivers a digital advertisement that features an item. As indicated by these marks in the advertisement columns414aand414b, the social networking system104delivers a digital advertisement to the users having user identifiers1235and1240. Similarly, the purchase columns416a,416b,416c, and416dinclude marks indicating users who purchase the featured item. As indicated by the marks in the purchase columns416a-416d, the users having user identifiers1234,1239,1237,1243, and1244purchased the featured item. Moreover, the conversion columns418a,418b,418c, and418dinclude marks indicating whether a purchase represents a conversion action. As indicated by the marks in the conversion columns418a-418d, the purchases of the users having user identifiers1234,1239, and1243represent conversion actions, but the purchases of the users having user identifiers1237and1244do not represent conversion actions.

Consistent with the disclosure above, the social networking system104uses the data represented inFIG. 4to determine a first incremental lift between the first test group410aand the first control group412a. Similarly, the social networking system104uses the data represented inFIG. 4to determine a second incremental lift between the second test group410band the second control group412b. Based on this data, the social networking system104determines a first incremental lift of two additional users who purchased the featured item—because two users purchased the featured item in the first test group410a(as conversion actions) compared to zero users who purchased the featured item in the first control group412a. Additionally, the social networking system104determines a second incremental lift of zero additional users who purchased the featured item—because one user purchased the featured item in the second test group410b(as a conversion action) compared to one user who purchased the featured item in the second control group412b.

In this particular embodiment, by comparing the first and second incremental lifts, the social networking system104determines the cluster effect as an expression of incremental consumption—that is, a cluster effect of two additional users who purchased the featured item. For explanatory purposes,FIG. 4depicts incremental lifts and a cluster effect on a small scale. But as noted above, the social networking system104may use a larger sample of users and determine larger incremental lifts or cluster effects using different measurements.

Turning now toFIGS. 5A and 5B, this figure illustrates a sample of users and trial groups in accordance with one or more embodiments of the acts described inFIG. 3B. As noted above, in some embodiments, the social networking system104identifies a sample of users and sorts those users into trial groups by super cluster to compare different clustering algorithms and determine a cluster-effect differential.FIGS. 5A and 5Bdepict a sample of users502and various trial groups that the social networking system104identifies and creates when performing some of the acts328-354. For explanatory purposes, the sample of users502depicted inFIG. 5Aincludes user identifiers and cluster identifiers corresponding to twenty-three users. These twenty-three users are merely illustrative. In additional or alternative embodiments, the sample of users502are part of a larger sample.

As shown inFIG. 5A, the sample of users502include a user identifier, a cluster identifier, and an alternative cluster identifier for each user within the sample. In this particular embodiment, the social networking system104applies a first clustering algorithm and a second clustering algorithm as part of identifying the sample of users502. The social networking system104further assigns a cluster identifier corresponding to first clustering algorithm—and an alternative cluster identifier corresponding to the second clustering algorithm—as part of generating clusters of users and alternative clusters of users. A user-identifier column504includes the user identifier for each user. By contrast, a cluster-identifier column506aincludes the cluster identifier generated by the first clustering algorithm for each user. Similarly, an alternative-cluster-identifier column506bincludes the alternative cluster identifier generated by the second clustering algorithm for each user.

In addition to applying the first and second clustering algorithms to generate clusters of users and alternative clusters of users, the social networking system104generates super clusters. Consistent with the disclosure above, the social networking system104groups one or more clusters of users and one or more alternative clusters of users within each super cluster and assigns each user a super-cluster identifier. A super-cluster-identifier column508includes the super-cluster identifier for each user. As indicated by the super-cluster identifiers, some of the users within the sample of users502are part of (a) a cluster comprising one cluster of users, on the one hand, and (b) an alternative cluster comprising another cluster of users, on the other hand. Such super clusters contain so-called overlapping users, who include, but are not limited to, the users having the user identifiers137,139,140,146,147,148,149,156, and157.

As further shown inFIG. 5A, the social networking system104separates the super clusters from the sample of users502into a first group510aand a second group510b. Consistent with the disclosure above, the social networking system104randomly assigns the users into either (but not both of) the first group510aor the second group510baccording to super-cluster identifiers. In other words, the social networking system104sorts super clusters of users by super cluster identifier into the first group510aor the second group510b.

Turning now toFIG. 5B, after randomly separating the super clusters from the sample of users402, the social networking system104separates super clusters from the first group510ainto a first test group512aand a first control group514a. In particular, the social networking system104separates users based on super cluster identifier to ensure that users having a same super cluster identifier are assigned to either (but not both) the first test group512aor the first control group514a.

Similarly, the social networking system104separates super clusters from the second group510binto a second test group512band a second control group514b. Here again, the social networking system104separates users based on super cluster identifier to ensure that users having a same super cluster identifier are assigned to either (but not both) the second test group512bor the second control group514b.

As further shown inFIG. 5B, each of the test groups include a corresponding advertisement column, purchase column, and conversion column. Each of the control groups include a corresponding purchase column and conversion column. In particular, an advertisement column516a, a purchase column518a, and a conversion column520acorrespond to the first test group512a; an advertisement column516b, a purchase column518c, and a conversion column520ccorrespond to the second test group512b; a purchase column518band a conversion column520bcorrespond to the first control group514a; and a purchase column518dand a conversion column520dcorrespond to the second control group514b.

The advertisement columns516aand516binclude marks indicating users to whom the social networking system104delivers a digital advertisement that features an item. As indicated by the marks in the advertisement columns516aand516b, the social networking system104delivers a digital advertisement to the users having user identifiers137,144,150,151, and158. Similarly, the purchase columns518a,518b,518c, and518dinclude marks indicating users who purchased the featured item. As indicated by the marks in the purchase columns518a-518d, the users having user identifiers136,146,149,152, and157purchased the featured item.

By contrast, the conversion columns520a,520b,520c, and520dinclude marks indicating whether a purchase represents a conversion action. To facilitate comparing clustering algorithms, a purchase represents a conversion action in the first test group512aonly when (i) the social networking system104delivers a digital advertisement to a user within a cluster of users (which are designated by cluster identifiers) and (ii) a user in the same cluster of users purchases the featured item. To be clear, such a conversion action includes another user in the same cluster of users purchasing the featured item—even if the social networking system104did not deliver the digital advertisement to the purchasing user. If the social networking system104delivers a digital advertisement to a user within a cluster of users and a user within an alternative cluster of user purchases the featured item, the purchase does not represent a conversion action for purposes of comparison—even when the cluster of users and the alternative cluster of users include an overlapping user. As indicated by the marks in the conversion column520a, the purchases of the users having user identifiers152and157represent conversion actions.

By contrast, a purchase represents a conversion action in the second test group512bonly when (i) the social networking system104delivers a digital advertisement to a user within an alternative cluster of users (which are designated by alternative cluster identifiers) and (ii) a user in the same alternative cluster of users purchases the featured item. Again, to be clear, such a conversion action includes another user in the same alternative cluster of users purchasing the featured item—even if the social networking system104did not deliver the digital advertisement to the purchasing user. If the social networking system104delivers a digital advertisement to a user within an alternative cluster of users and a user within a cluster of users purchases the featured item, the purchase does not represent a conversion action for purposes of comparison—even when the alternative cluster of users and the cluster of users include an overlapping user. As indicated by the marks in the conversion column520c, the purchases of the users having user identifiers136and149do not represent conversion actions. The purchase of the user having user identifier146also does not represent a conversion action—because the social networking system104did not deliver an advertisement to the user or any other users within his or her alternative cluster of users.

Consistent with the disclosure above, the social networking system104uses the data represented inFIG. 5Bto determine a first incremental lift between the first test group512aand the first control group514a. Similarly, the social networking system104uses the data represented inFIG. 5Bto determine a second incremental lift between the second test group512band the second control group514b. Based on this data, the social networking system104determines a first incremental lift of two additional users who purchased the featured item—because two users purchased the featured item in the first test group512a(as conversion actions) compared to zero users who purchased the featured item in the first control group514a. Additionally, the social networking system104determines a second incremental lift of zero additional users who purchased the featured item—because two users purchased the featured item in the second test group512bcompared to one user who purchased the featured item in the second control group514b. The second incremental lift is zero because none of these latter purchases represent conversion actions.

By comparing the first and second incremental lifts, the social networking system104determines the cluster-effect differential as an expression of incremental consumption that is, a cluster-effect differential of two additional users who purchased the featured item in the clusters of users generated by the first clustering algorithm. Relatedly, the social networking system104also determines that the first clustering algorithm results in a cluster effect of two additional users who purchased the featured item and that the second clustering algorithm results in a cluster effect of zero additional users who purchased the featured item. In some embodiments, the social networking system104identifies the first clustering algorithm as the clustering algorithm generating clusters that more accurately reflect real-world clusters—when compared to the second clustering algorithm—because the first clustering algorithm produces a larger cluster effect.

For explanatory purposes,FIG. 5Bdepicts incremental lifts, cluster effects, and a cluster-effect differential on a small scale. As noted above, the social networking system104may use a larger sample of users and determine larger incremental lifts, cluster effects, or cluster-effect differentials using different measurements.

As noted above, in certain embodiments, the social networking system104generates a conversion report and target-cluster options as tools for applying a cluster effect. The social networking system104in turn provides the conversion report and target-cluster options to a client device for presentation within a graphical user interface. The conversion report provides an administrator with options to adjust distribution of digital content to users within target clusters based on a determined cluster effect.

As shown inFIG. 6, the social networking system104provides a conversion report601to an administrator client device600. The administrator client device600in turn presents the conversion report601within a graphical user interface602of a screen604. The conversion report601represents the results of an advertisement analysis using multiple test groups and control groups—with a first test group and a first control group sorted by cluster and a second test group and a second control group sorted by user. The social networking system104performs this advertisement analysis to determine a specific cluster effect—the impact that one or more household members' exposure to digital content has on conversion actions by other household members. The social networking system104performs the advertisement analysis by performing the acts depicted inFIG. 3A.

To represent the results of the advertisement analysis, the conversion report601includes various incremental lifts. These incremental lifts represent first incremental lifts between the first test group and the first control group, on the one hand, and second incremental lifts between the second test group and a second control group, on the other hand. Specifically, the conversion report601includes graphics representing a first incremental-sales lift606a, a first total lift608a, a first incremental-consumption lift610a, a first conversion lift612a, a first incremental-spend-amount lift614a, and a first average-spend lift616a. Similarly, the conversion report601includes graphics representing a second incremental-sales lift606b, a second total lift608b, a second incremental-consumption lift610b, a second conversion lift612b, a second incremental-spend-amount lift614b, and a second average-spend lift616b.

As further shown inFIG. 6, the conversion report601includes cluster effects618a-618fThe social networking system104determines the cluster effects618a-618fbased on a comparison of corresponding first incremental lifts and second incremental lifts. As shown, the conversion report601includes graphics representing a first cluster effect618arepresenting a divergence between the first incremental-sales lift606aand the second incremental-sales lift606b; a second cluster effect618brepresenting a divergence between the first total lift608aand the second total lift608b; a third cluster effect618crepresenting a divergence between the first incremental-consumption lift610aand the second incremental-consumption lift610b; a fourth cluster effect618drepresenting a divergence between the first conversion lift612aand the second conversion lift612b; a fifth cluster effect618erepresenting a divergence between the first incremental-spend-amount lift614aand the second incremental-spend-amount lift614b; and a sixth cluster effect618frepresenting a divergence between the first average-spend lift616aand the second average-spend lift616b.

In addition to representations of incremental lifts and cluster effects, the conversion report601further includes target-cluster options. The target-cluster options correspond to clusters that the social networking system104determines as targets for increasing conversion actions—with increased or decreased distribution to users within the cluster. In other words, in some embodiments, the social networking system104suggests target clusters for increased or decreased digital-content distribution to users. The social networking system104identifies clusters comprising users to which increased digital-content distribution would increase conversion actions. In some such embodiments, the social networking system104identifies clusters with a highest relative cluster effect as target clusters.

As shown inFIG. 6, the social networking system104indicates target clusters within the target-cluster options620a-620c. The first target-cluster option620aindicates clusters comprising multiple users in general. The second target-cluster option620bindicates clusters in a particular location. The third target-cluster option620cindicates clusters in a particular economic class. While some of the target clusters inFIG. 6represent a particular location or economic class, in additional or alternative embodiments, the social networking system104identifies target clusters representing different demographics, including, but not limited to, age, education level, ethnicity, political party, and religion.

When an administrator interactions with a target-cluster option (e.g., by click or touch gesture), the administrator client device600sends an indication of that interaction to the social networking system104to adjust distribution of digital content to users within the target cluster. For example, when the administrator client device600detects an interaction with the first target-cluster option620a, the administrator client device600sends an indication to the social networking system104to adjust distribution of a digital advertisement. As shown inFIG. 6, the interaction indicates either increasing or decreasing distribution of the digital advertisement to users within clusters of users in general. The first target-cluster option620aincludes a radio button that (when adjusted) can increase or decrease a percentage of users within clusters to which the social networking system104distributes digital content. The second target-cluster option620band the third target-cluster option620cfunction similarly to the first target-cluster option620aand include radio buttons by which distribution may be increased or decreased.

Turning now toFIG. 7, this figure illustrates a flowchart of a series of acts700of determining a cluster effect of users' exposure to digital content on other users' conversion actions in accordance with one or more embodiments. WhileFIG. 7illustrates acts according to one embodiment, alternative embodiments may omit, add to, reorder, and/or modify any of the acts shown inFIG. 7. The acts ofFIG. 7can be performed as part of a method. Alternatively, a non-transitory computer readable storage medium can comprise instructions that, when executed by one or more processors, cause a computing device to perform the acts depicted inFIG. 7. In still further embodiments, a system can perform the acts ofFIG. 7.

As shown inFIG. 7, the acts700include an act710of generating clusters of users. In particular, in some embodiments, the act710includes generating clusters of users, each cluster having one or more users of a social networking system. Additionally, in certain embodiments, generating the clusters of users comprises generating the clusters of users based on one or more of an affinity coefficient, a designated relationship, a commonly used Internet Protocol (“IP”) address, a common physical address, a common device identifier, tagged users within images, a common employer, and a common educational institution.

Moreover, in certain embodiments, generating the clusters of users comprises assigning user identifiers and cluster identifiers to the users of the social networking system. Similarly, in some embodiments, generating the clusters of users comprises applying a clustering algorithm to assign one or more users to a particular cluster of users based on attributes of the one or more users. In some cases, generating the clusters of users comprises applying a hash function to generate the clusters of users. Relatedly, in one or more embodiments, assigning the cluster identifiers to the users of the social networking system comprises applying a clustering algorithm to assign the cluster identifiers to the users of the social networking system.

In some cases, generating the clusters of users comprises generating a first cluster comprising a single user and a second cluster comprising multiple users having one or more common attributes. Similarly, in one or more embodiments, generating the clusters of users comprises generating the clusters of users, wherein each cluster is a cluster of one or more classmates, club members, coworkers, households, neighbors, organizational members, or social-network friends.

As further shown inFIG. 7, the acts700include an act720of separating the clusters of users into a first group of clusters and a second group of clusters. For example, in certain embodiments, separating the clusters of users into a first group of clusters and a second group of clusters comprises separating the clusters of users into the first group of clusters and the second group of clusters based on the cluster identifiers. Further, in some embodiments, separating the clusters of users into the first group of clusters and the second group of clusters based on the cluster identifiers comprises randomly assigning a particular cluster of users to the first group of clusters or the second group of clusters based on randomly associating a cluster identifier corresponding to the particular cluster of users with either the first group of clusters or the second group of clusters.

As further shown inFIG. 7, the acts700include an act730of determining a first incremental lift. In particular, in some embodiments, the act730includes determining a first incremental lift between a first test group and a first control group each comprising clusters of users from the first group of clusters. For example, in some embodiments, determining the first incremental lift between the first test group and the first control group comprises randomly assigning particular clusters of users from the first group of clusters to the first test group or the first control group based on cluster identifiers. Relatedly, in certain embodiments, determining the first incremental lift between the first test group and the first control group comprises delivering one or more instances of digital content to the first test group but not to the first control group.

As noted above, a first incremental lift can come in different measurements. For example, in some embodiments, determining the first incremental lift between the first test group and the first control group comprises determining a first average-spend lift, first conversion lift, first incremental-sales lift, first incremental-consumption lift, first incremental-spend-amount lift, or first total-sales lift.

As further shown inFIG. 7, the acts700include an act740of determining a second incremental lift. In particular, in some embodiments, the act740includes determining a second incremental lift between a second test group and a second control group each comprising individual users from the second group of clusters. For example, in some embodiments, determining the second incremental lift between the second test group and the second control group comprises randomly assigning users from the second group of clusters to the second test group or the second control group based on user identifiers. Relatedly, in certain embodiments, determining the second incremental lift between the second test group and the second control group comprises delivering the one or more instances of digital content to the second test group but not to the second control group.

As noted above, a second incremental lift can come in different measurements. For example, in some embodiments, determining the second incremental lift between the second test group and the second control group comprises determining a second average-spend lift, second conversion lift, second incremental-sales lift, second incremental-consumption lift, second incremental-spend-amount lift, or second total-sales lift.

As further shown inFIG. 7, the acts700include an act750of comparing the first incremental lift to the second incremental lift to determine a cluster effect. For example, in some embodiments, the act750includes determining a divergence between the first incremental lift and the second incremental lift.

In addition to the acts710-750, in some embodiments, the acts700further include modifying distribution of digital content based on the cluster effect. Relatedly, in one or more embodiments, modifying the distribution of the digital content based on the cluster effect comprises increasing distribution of the digital content to users within clusters comprising multiple users having one or more common attributes.

Additionally, in certain embodiments, the acts700further include generating a conversion report comprising a representation of the cluster effect; and providing the conversion report to a client device. On a related note, in one or more embodiments, the conversion report comprises a selectable option to adjust distribution of digital content to users within clusters comprising multiple users having one or more common attributes. In some such embodiments, the conversion report comprises selectable options to adjust distribution of the digital content to users within clusters of a target demographic or target location.

Moreover, in some embodiments, the acts700further include applying an alternative clustering algorithm to generate alternative clusters of users, each alternative cluster having one or more users of the social networking system; separating the alternative clusters of users into an alternative first group of clusters and an alternative second group of clusters; determining an alternative first incremental lift between an alternative first test group and an alternative first control group each comprising alternative clusters of users from the alternative first group of clusters; determining an alternative second incremental lift between an alternative second test group and an alternative second control group each comprising individual users from the alternative second group of clusters; comparing the alternative first incremental lift to the alternative second incremental lift to determine an alternative cluster effect; and based on a comparison of the cluster effect and the alternative cluster effect, identifying either the clustering algorithm or the alternative clustering algorithm as generating clusters that capture a larger cluster effect.

FIG. 8illustrates a block diagram of exemplary computing device800that may be configured to perform one or more of the processes described above. One will appreciate that one or more computing devices such as the computing device800may implement the digital communications system described above. Furthermore, any of the server device(s)102, client devices108a-108n, and administrator client device600can be a computing device800. As shown byFIG. 8, the computing device800can comprise a processor802, a memory804, a storage device806, an I/O interface808, and a communication interface810, which may be communicatively coupled by way of a communication infrastructure812. While an exemplary computing device800is shown inFIG. 8, the components illustrated inFIG. 8are not intended to be limiting. Additional or alternative components may be used in other embodiments. Furthermore, in certain embodiments, the computing device800can include fewer components than those shown inFIG. 8. Components of the computing device800shown inFIG. 8will now be described in additional detail.

In one or more embodiments, the processor802includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, the processor802may retrieve (or fetch) the instructions from an internal register, an internal cache, the memory804, or the storage device806and decode and execute them. In one or more embodiments, the processor802may include one or more internal caches for data, instructions, or addresses. As an example and not by way of limitation, the processor802may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (“TLBs”). Instructions in the instruction caches may be copies of instructions in the memory804or the storage device806.

The memory804may be used for storing data, metadata, and programs for execution by the processor(s). The memory804may include one or more of volatile and non-volatile memories, such as Random Access Memory (“RAM”), Read Only Memory (“ROM”), a solid state disk (“SSD”), Flash, Phase Change Memory (“PCM”), or other types of data storage. The memory804may be internal or distributed memory.

The storage device806includes storage for storing data or instructions. As an example and not by way of limitation, storage device806can comprise a non-transitory storage medium described above. The storage device806may include a hard disk drive (“HDD”), flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (“USB”) drive or a combination of two or more of these. The storage device806may include removable or non-removable (or fixed) media, where appropriate. The storage device806may be internal or external to the computing device800. In one or more embodiments, the storage device806is non-volatile, solid-state memory. In other embodiments, the storage device806includes read-only memory (“ROM”). Where appropriate, this ROM may be mask programmed ROM, programmable ROM (“PROM”), erasable PROM (“EPROM”), electrically erasable PROM (“EEPROM”), electrically alterable ROM (“EAROM”), or flash memory or a combination of two or more of these.

The communication interface810can include hardware, software, or both. In any event, the communication interface810can provide one or more interfaces for communication (such as, for example, packet-based communication) between the computing device800and one or more other computing devices or networks. As an example and not by way of limitation, the communication interface810may include a network interface controller (“NIC”) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (“WNIC”) or wireless adapter for communicating with a wireless network, such as a WI-FI.

Additionally, or alternatively, the communication interface810may facilitate communications with an ad hoc network, a personal area network (“PAN”), a local area network (“LAN”), a wide area network (“WAN”), a metropolitan area network (“MAN”), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, the communication interface810may facilitate communications with a wireless PAN (“WPAN”) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (“GSM”) network), or other suitable wireless network or a combination thereof.

The communication infrastructure812may include hardware, software, or both that couples components of the computing device800to each other. As an example and not by way of limitation, the communication infrastructure812may include an Accelerated Graphics Port (“AGP”) or other graphics bus, an Enhanced Industry Standard Architecture (“EISA”) bus, a front-side bus (“FSB”), a HYPERTRANSPORT (“HT”) interconnect, an Industry Standard Architecture (“ISA”) bus, an INFINIBAND interconnect, a low-pin-count (“LPC”) bus, a memory bus, a Micro Channel Architecture (“MCA”) bus, a Peripheral Component Interconnect (“PCI”) bus, a PCI-Express (“PCIe”) bus, a serial advanced technology attachment (“SATA”) bus, a Video Electronics Standards Association local (“VLB”) bus, or another suitable bus or a combination thereof.

As mentioned above, in one or more embodiments, the digital communications system comprises a social-networking system. A social-networking system may enable its users (such as persons or organizations) to interact with the system and with each other. The social-networking system may, with input from a user, create and store in the social-networking system a user profile associated with the user. The user profile may include demographic information, communication-channel information, and information on personal interests of the user. The social-networking system may also, with input from a user, create and store a record of relationships of the user with other users of the social-networking system, as well as provide services (e.g. wall posts, photo-sharing, on-line calendars and event organization, messaging, games, or advertisements) to facilitate social interaction between or among users. Also, the social-networking system may allow users to post photographs and other visual media items to a user's profile page (typically known as “wall posts” or “timeline posts”) or in a photo album, both of which may be accessible to other users of the social-networking system depending upon the user's configured privacy settings.

FIG. 9illustrates an example network environment900of a social-networking system. Network environment900includes a client device906, a social networking system902, and a third-party system908connected to each other by a network904. AlthoughFIG. 9illustrates a particular arrangement of client device906, social networking system902, third-party system908, and network904, this disclosure contemplates any suitable arrangement of client device906, social networking system902, third-party system908, and network904. As an example and not by way of limitation, two or more of client device906, social networking system902, and third-party system908may be connected to each other directly, bypassing network904. As another example, two or more of client device906, social networking system902, and third-party system908may be physically or logically co-located with each other in whole or in part. Moreover, althoughFIG. 9illustrates a particular number of client devices906, social networking systems902, third-party systems908, and networks904, this disclosure contemplates any suitable number of client devices906, social networking systems902, third-party systems908, and networks904. As an example and not by way of limitation, network environment900may include multiple client devices906, social networking systems902, third-party systems908, and networks904.

This disclosure contemplates any suitable network904. As an example and not by way of limitation, one or more portions of network904may include an ad hoc network, an intranet, an extranet, a virtual private network (“VPN”), a local area network (“LAN”), a wireless LAN (“WLAN”), a wide area network (“WAN”), a wireless WAN (“WWAN”), a metropolitan area network (“MAN”), a portion of the Internet, a portion of the Public Switched Telephone Network (“PSTN”), a cellular telephone network, or a combination of two or more of these. Network904may include one or more networks904.

Links may connect client device906, social networking system902, and third-party system908to communication network904or to each other. This disclosure contemplates any suitable links. In particular embodiments, one or more links include one or more wireline (such as for example Digital Subscriber Line (“DSL”) or Data Over Cable Service Interface Specification (“DOCSIS”)), wireless (such as for example Wi-Fi or Worldwide Interoperability for Microwave Access (“WiMAX”)), or optical (such as for example Synchronous Optical Network (“SONET”) or Synchronous Digital Hierarchy (“SDH”)) links. In particular embodiments, one or more links each include an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, a portion of the Internet, a portion of the PSTN, a cellular technology-based network, a satellite communications technology-based network, another link, or a combination of two or more such links. Links need not necessarily be the same throughout network environment900. One or more first links may differ in one or more respects from one or more second links.

In particular embodiments, client device906may be an electronic device including hardware, software, or embedded logic components or a combination of two or more such components and capable of carrying out the appropriate functionalities implemented or supported by client device906. As an example and not by way of limitation, a client device906may include any of the computing devices discussed above in relation toFIG. 9. A client device906may enable a network user at client device906to access network904. A client device906may enable its user to communicate with other users at other client devices906.

In particular embodiments, client device906may include a web browser, such as MICROSOFT INTERNET EXPLORER, GOOGLE CHROME or MOZILLA FIREFOX, and may have one or more add-ons, plug-ins, or other extensions, such as TOOLBAR or YAHOO TOOLBAR. A user at client device906may enter a Uniform Resource Locator (“URL”) or other address directing the web browser to a particular server (such as server, or a server associated with a third-party system908), and the web browser may generate a Hyper Text Transfer Protocol (“HTTP”) request and communicate the HTTP request to server. The server may accept the HTTP request and communicate to client device906one or more Hyper Text Markup Language (“HTML”) files responsive to the HTTP request. Client device906may render a webpage based on the HTML files from the server for presentation to the user. This disclosure contemplates any suitable webpage files. As an example and not by way of limitation, webpages may render from HTML files, Extensible Hyper Text Markup Language (“XHTML”) files, or Extensible Markup Language (“XML”) files, according to particular needs. Such pages may also execute scripts such as, for example and without limitation, those written in JAVASCRIPT, JAVA, MICROSOFT SILVERLIGHT, combinations of markup language and scripts such as AJAX (Asynchronous JAVASCRIPT and XML), and the like. Herein, reference to a webpage encompasses one or more corresponding webpage files (which a browser may use to render the webpage) and vice versa, where appropriate.

In particular embodiments, social networking system902may be a network-addressable computing system that can host an online social network. Social networking system902may generate, store, receive, and send social-networking data, such as, for example, user-profile data, concept-profile data, social-graph information, or other suitable data related to the online social network. Social networking system902may be accessed by the other components of network environment900either directly or via network904. In particular embodiments, social networking system902may include one or more servers. Each server may be a unitary server or a distributed server spanning multiple computers or multiple datacenters. Servers may be of various types, such as, for example and without limitation, web server, news server, mail server, message server, advertising server, file server, application server, exchange server, database server, proxy server, another server suitable for performing functions or processes described herein, or any combination thereof. In particular embodiments, each server may include hardware, software, or embedded logic components or a combination of two or more such components for carrying out the appropriate functionalities implemented or supported by server. In particular embodiments, social networking system902may include one or more data stores. Data stores may be used to store various types of information. In particular embodiments, the information stored in data stores may be organized according to specific data structures. In particular embodiments, each data store may be a relational, columnar, correlation, or other suitable database. Although this disclosure describes or illustrates particular types of databases, this disclosure contemplates any suitable types of databases. Particular embodiments may provide interfaces that enable a client device906, a social networking system902, or a third-party system908to manage, retrieve, modify, add, or delete, the information stored in data store.

In particular embodiments, social networking system902may store one or more social graphs in one or more data stores. In particular embodiments, a social graph may include multiple nodes—which may include multiple user nodes (each corresponding to a particular user) or multiple concept nodes (each corresponding to a particular concept)—and multiple edges connecting the nodes. Social networking system902may provide users of the online social network the ability to communicate and interact with other users. In particular embodiments, users may join the online social network via social networking system902and then add connections (e.g., relationships) to a number of other users of social networking system902whom they want to be connected to. Herein, the term “friend” may refer to any other user of social networking system902with whom a user has formed a connection, association, or relationship via social networking system902.

In particular embodiments, social networking system902may provide users with the ability to take actions on various types of items or objects, supported by social networking system902. As an example and not by way of limitation, the items and objects may include groups or social networks to which users of social networking system902may belong, events or calendar entries in which a user might be interested, computer-based applications that a user may use, transactions that allow users to buy or sell items via the service, interactions with advertisements that a user may perform, or other suitable items or objects. A user may interact with anything that is capable of being represented in social networking system902or by an external system of third-party system908, which is separate from social networking system902and coupled to social networking system902via a network904.

In particular embodiments, social networking system902may be capable of linking a variety of entities. As an example and not by way of limitation, social networking system902may enable users to interact with each other as well as receive content from third-party systems908or other entities, or to allow users to interact with these entities through an application programming interfaces (“API”) or other communication channels.

In particular embodiments, a third-party system908may include one or more types of servers, one or more data stores, one or more interfaces, including but not limited to APIs, one or more web services, one or more content sources, one or more networks, or any other suitable components, e.g., that servers may communicate with. A third-party system908may be operated by a different entity from an entity operating social networking system902. In particular embodiments, however, social networking system902and third-party systems908may operate in conjunction with each other to provide social-networking services to users of social networking system902or third-party systems908. In this sense, social networking system902may provide a platform, or backbone, which other systems, such as third-party systems908, may use to provide social-networking services and functionality to users across the Internet.

In particular embodiments, social networking system902also includes user-generated content objects, which may enhance a user's interactions with social networking system902. User-generated content may include anything a user can add, upload, send, or “post” to social networking system902. As an example and not by way of limitation, a user communicates posts to social networking system902from a client device906. Posts may include data such as status updates or other textual data, location information, photos, videos, links, music or other similar data or media. Content may also be added to social networking system902by a third-party through a “communication channel,” such as a newsfeed or stream.

FIG. 10illustrates example social graph1000. In particular embodiments, social networking system902may store one or more social graphs1000in one or more data stores. In particular embodiments, social graph1000may include multiple nodes—which may include multiple user nodes1002or multiple concept nodes1004—and multiple edges1006connecting the nodes. Example social graph1000illustrated inFIG. 10is shown, for didactic purposes, in a two-dimensional visual map representation. In particular embodiments, a social networking system902, client device906, or third-party system908may access social graph1000and related social-graph information for suitable applications. The nodes and edges of social graph1000may be stored as data objects, for example, in a data store (such as a social-graph database). Such a data store may include one or more searchable or query able indexes of nodes or edges of social graph1000.

In particular embodiments, a user node1002may correspond to a user of social networking system902. As an example and not by way of limitation, a user may be an individual (human user), an entity (e.g., an enterprise, business, or third-party application), or a group (e.g., of individuals or entities) that interacts or communicates with or over social networking system902. In particular embodiments, when a user registers for an account with social networking system902, social networking system902may create a user node1002corresponding to the user, and store the user node1002in one or more data stores. Users and user nodes1002described herein may, where appropriate, refer to registered users and user nodes1002associated with registered users. In addition, or as an alternative, users and user nodes1002described herein may, where appropriate, refer to users that have not registered with social networking system902. In particular embodiments, a user node1002may be associated with information provided by a user or information gathered by various systems, including social networking system902. As an example and not by way of limitation, a user may provide his or her name, profile picture, contact information, birth date, sex, marital status, family status, employment, education background, preferences, interests, or other demographic information. Each user node of the social graph may have a corresponding web page (typically known as a profile page). In response to a request including a user name, the social-networking system can access a user node corresponding to the user name, and construct a profile page including the name, a profile picture, and other information associated with the user. A profile page of a first user may display to a second user all or a portion of the first user's information based on one or more privacy settings by the first user and the relationship between the first user and the second user.

In particular embodiments, a concept node1004may represent a third-party webpage or resource hosted by a third-party system908. The third-party webpage or resource may include, among other elements, content, a selectable or other icon, or other inter-actable object (which may be implemented, for example, in JavaScript, AJAX, or PHP codes) representing an action or activity. As an example and not by way of limitation, a third-party webpage may include a selectable icon such as “like,” “check in,” “eat,” “recommend,” or another suitable action or activity. A user viewing the third-party webpage may perform an action by selecting one of the icons (e.g., “eat”), causing a client device906to send to social networking system902a message indicating the user's action. In response to the message, social networking system902may create an edge (e.g., an “eat” edge) between a user node1002corresponding to the user and a concept node1004corresponding to the third-party webpage or resource and store edge1006in one or more data stores.

In particular embodiments, a pair of nodes in social graph1000may be connected to each other by one or more edges1006. An edge1006connecting a pair of nodes may represent a relationship between the pair of nodes. In particular embodiments, an edge1006may include or represent one or more data objects or attributes corresponding to the relationship between a pair of nodes. As an example and not by way of limitation, a first user may indicate that a second user is a “friend” of the first user. In response to this indication, social networking system902may send a “friend request” to the second user. If the second user confirms the “friend request,” social networking system902may create an edge1006connecting the first user's user node1002to the second user's user node1002in social graph1000and store edge1006as social-graph information in one or more of data stores. In the example ofFIG. 10, social graph1000includes an edge1006indicating a friend relation between user nodes1002of user “A” and user “B” and an edge indicating a friend relation between user nodes1002of user “C” and user “B.” Although this disclosure describes or illustrates particular edges1006with particular attributes connecting particular user nodes1002, this disclosure contemplates any suitable edges1006with any suitable attributes connecting user nodes1002. As an example and not by way of limitation, an edge1006may represent a friendship, family relationship, business or employment relationship, fan relationship, follower relationship, visitor relationship, sub scriber relationship, superior/subordinate relationship, reciprocal relationship, non-reciprocal relationship, another suitable type of relationship, or two or more such relationships. Moreover, although this disclosure generally describes nodes as being connected, this disclosure also describes users or concepts as being connected. Herein, references to users or concepts being connected may, where appropriate, refer to the nodes corresponding to those users or concepts being connected in social graph1000by one or more edges1006.

In particular embodiments, an edge1006between a user node1002and a concept node1004may represent a particular action or activity performed by a user associated with user node1002toward a concept associated with a concept node1004. As an example and not by way of limitation, as illustrated inFIG. 10, a user may “like,” “attended,” “played,” “listened,” “cooked,” “worked at,” or “watched” a concept, each of which may correspond to an edge type or subtype. A concept-profile page corresponding to a concept node1004may include, for example, a selectable “check in” icon (such as, for example, a clickable “check in” icon) or a selectable “add to favorites” icon. Similarly, after a user clicks these icons, social networking system902may create a “favorite” edge or a “check in” edge in response to a user's action corresponding to a respective action. As another example and not by way of limitation, a user (user “C”) may listen to a particular song (“Ramble On”) using a particular application (SPOTIFY, which is an online music application). In this case, social networking system902may create a “listened” edge1006and a “used” edge (as illustrated inFIG. 10) between user nodes1002corresponding to the user and concept nodes1004corresponding to the song and application to indicate that the user listened to the song and used the application. Moreover, social networking system902may create a “played” edge1006(as illustrated inFIG. 10) between concept nodes1004corresponding to the song and the application to indicate that the particular song was played by the particular application. In this case, “played” edge1006corresponds to an action performed by an external application (SPOTIFY) on an external audio file (the song “Imagine”). Although this disclosure describes particular edges1006with particular attributes connecting user nodes1002and concept nodes1004, this disclosure contemplates any suitable edges1006with any suitable attributes connecting user nodes1002and concept nodes1004. Moreover, although this disclosure describes edges between a user node1002and a concept node1004representing a single relationship, this disclosure contemplates edges between a user node1002and a concept node1004representing one or more relationships. As an example and not by way of limitation, an edge1006may represent both that a user likes and has used at a particular concept. Alternatively, another edge1006may represent each type of relationship (or multiples of a single relationship) between a user node1002and a concept node1004(as illustrated inFIG. 10between user node1002for user “E” and concept nodes1004for “SPOTIFY”).

In particular embodiments, social networking system902may create an edge1006between a user node1002and a concept node1004in social graph1000. As an example and not by way of limitation, a user viewing a concept-profile page (such as, for example, by using a web browser or a special-purpose application hosted by the user's client device906) may indicate that he or she likes the concept represented by the concept nodes1004by clicking or selecting a “Like” icon, which may cause the user's client device906to send to social networking system902a message indicating the user's liking of the concept associated with the concept-profile page. In response to the message, social networking system902may create an edge1006between user node1002associated with the user and concept nodes1004, as illustrated by “like” edge1006between the user and concept nodes1004. In particular embodiments, social networking system902may store an edge1006in one or more data stores. In particular embodiments, an edge1006may be automatically formed by social networking system902in response to a particular user action. As an example and not by way of limitation, if a first user uploads a picture, watches a movie, or listens to a song, an edge1006may be formed between user node1002corresponding to the first user and concept nodes1004corresponding to those concepts. Although this disclosure describes forming particular edges1006in particular manners, this disclosure contemplates forming any suitable edges1006in any suitable manner. In one or more embodiments, the social networking system902includes a node for each digital graphic and associated animation(s).

In particular embodiments, an advertisement may be text (which may be HTML-linked), one or more images (which may be HTML-linked), one or more videos, audio, one or more ADOBE FLASH files, a suitable combination of these, or any other suitable advertisement in any suitable digital format presented on one or more webpages, in one or more e-mails, or in connection with search results requested by a user. In addition, or as an alternative, an advertisement may be one or more sponsored stories (e.g., a news-feed or ticker item on social networking system902). A sponsored story may be a social action by a user (such as “liking” a page, “liking” or commenting on a post on a page, RSVPing to an event associated with a page, voting on a question posted on a page, checking in to a place, using an application or playing a game, or “liking” or sharing a website) that an advertiser promotes, for example, by having the social action presented within a pre-determined area of a profile page of a user or other page, presented with additional information associated with the advertiser, bumped up or otherwise highlighted within news feeds or tickers of other users, or otherwise promoted. The advertiser may pay to have the social action promoted. As an example and not by way of limitation, advertisements may be included among the search results of a search-results page, where sponsored content is promoted over non-sponsored content.

In particular embodiments, an advertisement may be requested for display within social-networking-system webpages, third-party webpages, or other pages. An advertisement may be displayed in a dedicated portion of a page, such as in a banner area at the top of the page, in a column at the side of the page, in a GUI of the page, in a pop-up window, in a drop-down menu, in an input field of the page, over the top of content of the page, or elsewhere with respect to the page. In addition or as an alternative, an advertisement may be displayed within an application. An advertisement may be displayed within dedicated pages, requiring the user to interact with or watch the advertisement before the user may access a page or utilize an application. The user may, for example view the advertisement through a web browser.

A user may interact with an advertisement in any suitable manner. The user may click or otherwise select the advertisement. By selecting the advertisement, the user may be directed to (or a browser or other application being used by the user) a page associated with the advertisement. At the page associated with the advertisement, the user may take additional actions, such as purchasing a product or service associated with the advertisement, receiving information associated with the advertisement, or subscribing to a newsletter associated with the advertisement. An advertisement with audio or video may be played by selecting a component of the advertisement (like a “play button”). Alternatively, by selecting the advertisement, social networking system902may execute or modify a particular action of the user.

An advertisement may also include social-networking-system functionality that a user may interact with. As an example and not by way of limitation, an advertisement may enable a user to “like” or otherwise endorse the advertisement by selecting an icon or link associated with endorsement. As another example and not by way of limitation, an advertisement may enable a user to search (e.g., by executing a query) for content related to the advertiser. Similarly, a user may share the advertisement with another user (e.g., through social networking system902) or RSVP (e.g., through social networking system902) to an event associated with the advertisement. In addition or as an alternative, an advertisement may include social-networking-system context directed to the user. As an example and not by way of limitation, an advertisement may display information about a friend of the user within social networking system902who has taken an action associated with the subject matter of the advertisement.

In particular embodiments, one or more servers may be authorization/privacy servers for enforcing privacy settings. In response to a request from a user (or other entity) for a particular object stored in a data store, social networking system902may send a request to the data store for the object. The request may identify the user associated with the request and may only be sent to the user (or a client device906of the user) if the authorization server determines that the user is authorized to access the object based on the privacy settings associated with the object. If the requesting user is not authorized to access the object, the authorization server may prevent the requested object from being retrieved from the data store, or may prevent the requested object from be sent to the user. In the search query context, an object may only be generated as a search result if the querying user is authorized to access the object. In other words, the object must have a visibility that is visible to the querying user. If the object has a visibility that is not visible to the user, the object may be excluded from the search results. Although this disclosure describes enforcing privacy settings in a particular manner, this disclosure contemplates enforcing privacy settings in any suitable manner.

The foregoing specification is described with reference to specific exemplary embodiments thereof. Various embodiments and aspects of the disclosure are described with reference to details discussed herein, and the accompanying drawings illustrate the various embodiments. The description above and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of various embodiments.