Generating content feed based on policy associated with adjacent content

A method for generating a content feed includes receiving, from a content provider, a target content item and a policy for the target content item, specifying a prohibited classification of the content that cannot be published adjacent to the target content item. The method also includes identifying a slot in a content feed including multiple content items. A machine learning model is then accessed and applied to two adjacent content items that are adjacent to the slot to determine whether each of the adjacent content items is the prohibited classification. Responsive to determining that the adjacent content items are not the prohibited classification, the target content item is placed in the slot, and the content feed including the target content item is sent for display to a viewing user.

BACKGROUND

This disclosure relates generally to generating a content feed for display to a user. More specifically, this disclosure relates to classifying a content item into a classification and determining whether to place a target content item next to the content item based on policy for the target content item and the classification of the content item.

A content feed is a sequence of content items that are to be provided to users. It may be used to provide content recommendations, targeting certain audiences based on user profiles and/or historical user behaviors. For example, a content feed manager may add recent news and updates to a content feed and share this content feed with users. For example, a first user who often searches for sports-related topics may be presented with a content feed including different sports news or products related to different sports, and a second user who often searches for fashion-related topics may be presented with a content feed including different fashion news or products related to the current fashion trend. Using content feeds, users can see different content items provided by different content providers without having to navigate to those content providers' sites. Users also can click a content item to be directed to the content provider's site to obtain additional information related to the content item.

In some cases, a content feed may include user-generated content items, brand-generated content items, and/or a combination thereof. Existing content aggregators or content feed managers generally manage content feed based on audience-based topic exclusion control, which filters at a user level based on what the users tend to look at or click through, such that their feeds will be dominated by those topics.

SUMMARY

A content feed is a sequence of content items that are to be provided to users. In some cases, a content feed may include user-generated content items, brand-generated content items, and/or a combination thereof. However, certain users or brands may or may not want their content items to be placed adjacent to another content item or a particular type of content item in a feed, such as (but not limited to) nudity content, violence content, and/or political content.

One or more embodiments address the above-described problem by using a machine learning model to classify whether each adjacent content item is in a prohibited class based on a policy for a target content item, and only allow the target content item to be placed in a slot between the adjacent content items when the adjacent content items are not in a prohibited class.

In accordance with one or more aspects of the disclosure, a content feed manager is configured to receive, from a content provider, a target content item and a policy for the target content item. The content feed manager is also configured to generate, for a viewing user of an online system, a content feed including multiple content items, and identifies a slot in the content feed. The slot is between two adjacent content items of the multiple content items in the content feed. The content feed manager then accesses a machine learning model trained to classify whether a content item is the prohibited classification. In some embodiments, the machine learning model is trained based on a set of training examples that includes features about multiple content items and labels for each of the other content items of whether each of the multiple content items is the prohibited classification.

The content feed manager then applies the trained machine learning model to each of the adjacent content items to determine whether each of the adjacent content items is the prohibited classification, and determines whether to place the target content item in the slot based on whether either of the adjacent content items is the prohibited classification. Responsive to determining to place the target content item in the slot, the content feed manager inserts the target content item in the slot, and sends the content feed containing the target content item for display to the viewing user.

DETAILED DESCRIPTION

System Architecture

FIG.1is a block diagram of a system environment100for an online system140.

The system environment100shown byFIG.1comprises one or more client devices110, a network120, one or more third-party systems130, and the online system140. In alternative configurations, different and/or additional components may be included in the system environment100. For example, the online system140is a social networking system, a content sharing network, or another system providing content to users.

One or more third party systems130may be coupled to the network120for communicating with the online system140, which is further described below in conjunction withFIG.2. In one embodiment, a third party system130is an application provider communicating information describing applications for execution by a client device110or communicating data to client devices110for use by an application executing on the client device. In other embodiments, a third party system130provides content or other information for presentation via a client device110. A third party system130may also communicate information to the online system140, such as advertisements, content, or information about an application provided by the third party system130.

FIG.2is a block diagram of an architecture of the online system140. The online system140shown inFIG.2includes a user profile store205, a content store210, an action logger215, an action log220, an edge store225, a web server230, a content feed manager235, a training dataset240, a modeling engine245, and a machine learning content classification model250. In other embodiments, the online system140may include additional, fewer, or different components for various applications. Conventional components such as network interfaces, security functions, load balancers, failover servers, management and network operations consoles, and the like are not shown so as to not obscure the details of the system architecture.

One or more content items included in the content store210include content for presentation to a user and a bid amount. The content is text, image, audio, video, or any other suitable data presented to a user. In various embodiments, the content also specifies a page of content. For example, a content item includes a landing page specifying a network address of a page of content to which a user is directed when the content item is accessed. The bid amount is included in a content item by a user and is used to determine an expected value, such as monetary compensation, provided by an advertiser to the online system140if content in the content item is presented to a user, if the content in the content item receives a user interaction when presented, or if any suitable condition is satisfied when content in the content item is presented to a user. For example, the bid amount included in a content item specifies a monetary amount that the online system140receives from a user who provided the content item to the online system140if content in the content item is displayed. In some embodiments, the expected value to the online system140of presenting the content from the content item may be determined by multiplying the bid amount by a probability of the content of the content item being accessed by a user.

In various embodiments, a content item includes various components capable of being identified and retrieved by the online system140. Example components of a content item include: a title, text data, image data, audio data, video data, a landing page, a user associated with the content item, or any other suitable information. The online system140may retrieve one or more specific components of a content item for presentation in some embodiments. For example, the online system140may identify a title and an image from a content item and provide the title and the image for presentation rather than the content item in its entirety.

Various content items may include an objective identifying an interaction that a user associated with a content item desires other users to perform when presented with content included in the content item. Example objectives include: installing an application associated with a content item, indicating a preference for a content item, sharing a content item with other users, interacting with an object associated with a content item, or performing any other suitable interaction. As content from a content item is presented to online system users, the online system140logs interactions between users presented with the content item or with objects associated with the content item. Additionally, the online system140may or may not receive compensation from a user associated with content item as online system users perform interactions with a content item that satisfy the objective included in the content item.

Additionally, a content item may include one or more targeting criteria specified by the user who provided the content item to the online system140. Targeting criteria included in a content item request specify one or more characteristics of users eligible to be presented with the content item. For example, targeting criteria are used to identify users having user profile information, edges, or actions satisfying at least one of the targeting criteria. Hence, targeting criteria allow a user to identify users having specific characteristics, simplifying subsequent distribution of content to different users.

In various embodiments, the content store210includes multiple campaigns, which each include one or more content items. In various embodiments, a campaign in associated with one or more characteristics that are attributed to each content item of the campaign. For example, a bid amount associated with a campaign is associated with each content item of the campaign. Similarly, an objective associated with a campaign is associated with each content item of the campaign. In various embodiments, a user providing content items to the online system140provides the online system140with various campaigns each including content items having different characteristics (e.g., associated with different content, including different types of content for presentation), and the campaigns are stored in the content store.

In one embodiment, targeting criteria may specify actions or types of connections between a user and another user or object of the online system140. Targeting criteria may also specify interactions between a user and objects performed external to the online system140, such as on a third party system130. For example, targeting criteria identifies users that have taken a particular action, such as sent a message to another user, used an application, joined a group, left a group, joined an event, generated an event description, purchased or reviewed a product or service using an online marketplace, requested information from a third party system130, installed an application, or performed any other suitable action. Including actions in targeting criteria allows users to further refine users eligible to be presented with content items. As another example, targeting criteria identifies users having a connection to another user or object or having a particular type of connection to another user or object.

The action log220may be used by the online system140to track user actions on the online system140, as well as actions on third party systems130that communicate information to the online system140. Users may interact with various objects on the online system140, and information describing these interactions is stored in the action log220. Examples of interactions with objects include: commenting on posts, sharing links, checking-in to physical locations via a client device110, accessing content items, and any other suitable interactions. Additional examples of interactions with objects on the online system140that are included in the action log220include: commenting on a photo album, communicating with a user, establishing a connection with an object, joining an event, joining a group, creating an event, authorizing an application, using an application, expressing a preference for an object (“liking” the object), and engaging in a transaction. Additionally, the action log220may record a user's interactions with advertisements on the online system140as well as with other applications operating on the online system140. In some embodiments, data from the action log220is used to infer interests or preferences of a user, augmenting the interests included in the user's user profile and allowing a more complete understanding of user preferences.

In one embodiment, the edge store225stores information describing connections between users and other objects on the online system140as edges. Some edges may be defined by users, allowing users to specify their relationships with other users. For example, users may generate edges with other users that parallel the users' real-life relationships, such as friends, co-workers, partners, and so forth. Other edges are generated when users interact with objects in the online system140, such as expressing interest in a page on the online system140, sharing a link with other users of the online system140, and commenting on posts made by other users of the online system140. Edges may connect two users who are connections in a social network, or may connect a user with an object in the system. In one embodiment, the nodes and edges form a complex social network of connections indicating how users are related or connected to each other (e.g., one user accepted a friend request from another user to become connections in the social network) and how a user is connected to an object due to the user interacting with the object in some manner (e.g., “liking” a page object, joining an event object or a group object, etc.). Objects can also be connected to each other based on the objects being related or having some interaction between them.

The content feed manager235assembles content items in the content store210and/or content items from third party content providers into content feeds, and provides the content feeds to the client devices110for display. The content feeds may be generated based on user profiles and/or classifications of content items.

Machine Learning Content Classification Model

The machine learning content classification model250is configured to classify content items into one or more classifications. For example, in some embodiments, the machine learning content classification model250is trained to classify a given content item such as nudity, violence, or politics. Alternatively, in some embodiments, the content classification model250is trained to classify a given content item as good, medium, or bad. The content classification model250is trained by the modeling engine245using a training dataset240. Additional details about the content feed manager235, the training dataset240, the modeling engine245, and the machine learning content classification model250are further described below with respect toFIGS.3-5.

FIG.3illustrates an example process300of training the machine learning content classification model250, using training dataset240. The training dataset240includes multiple training examples. The training examples include features304,307about multiple content items302,305and labels303,306for each of the content items. The labels303,306indicate a classification of the content items or whether the content item is a prohibited classification. The modeling engine245is configured to process the features304,307and the labels303,306corresponding to the content items302,305to train the content classification model250.

The features304,307may include content-specific features and/or user engagement features. The content-specific features may include (but are not limited to) features associated with the images or pixels of media content items, such as whether each pixel represents human skin, or whether a magnitude change of a vector over time is greater than a threshold. The content-specific features may also include (but are not limited to) features associated with text contained in the content items, such as whether political terms are shown in the content items. The content specific features may also include features associated with a creator, an original poster, a re-poster, a commenter, and/or a responder of the content item. The user engagement features may include whether most of the users reacted positively (e.g., clicks thumb up, responds with happy emoji) or negatively (e.g., clicks thumb down, responds with angry emoji, or flags the content item as inappropriate) to the content item.

In some embodiments, the labels303,306are generated by human raters. For example, the human raters manually review the examples of the content items, and manually rate whether each content item contains nudity, violence, and/or politics. In some embodiments, the labels303,306are organically generated by users of the online system140. For example, users may be able to flag a content item posted on the online system as containing nudity, violence, politics, or otherwise inappropriate. In some embodiments, these flagged content items can be used as training data. Alternatively, or in addition, the flagged content items are further processed to be attached with labels. For example, if a content item is flagged as nudity by more than 30% of the viewers, the content is labeled as nudity. If a content item is flagged as containing nudity by less than 30% of the viewers, the content is labeled as some nudity. If a content item is not flagged as containing nudity by any viewer, the content is labeled as non-nudity.

Using the training dataset240, the content classification model250is trained such that, responsive to receiving a given content item310, associated with one or more features312, the content classification model250is configured to output a classification314, indicating the classification of the given content item310.

In some embodiments, the classification may be good or bad. In some embodiments, the classification may be good, medium, or bad. In some embodiments, the classification may be violent or non-violent, nudity or non-nudity, political or non-political, etc. In some embodiments, multiple classification models are trained, and each of the classification models is trained to determine whether a content item is a particular classification or not. For example, a first model may be trained to determine whether a content item contains violence, some violence, or non-violence, a second model may be trained to determine whether a content item contains nudity, some nudity, or non-nudity, and/or a third model may be trained to determine whether a content item is political, medium political, or non-political.

In some embodiments, the content classification model250is trained to generate a score indicating a relative level of the content item. For example, the classification may be a numeric number between 0-10 for nudity, violence, and/or political, where 0 as non-nudity, non-violence, or non-political, and 10 as the highest level of nudity, violence, or political.

Each classification model may be trained via a different machine learning method. For example, the nudity classification model may be trained by detecting skin-colored pixels in an image, locating or forming skin regions based on the detected skin pixels, and analyzing the skin regions for clues of nudity or non-nudity.

As another example, the violence classification model may use features such as (but not limited to) optical flow, space-time interest points, and motion binary patterns. In some embodiments, a type of optical flow descriptor that may be used to detect violence is the crowd scenes. The crowd scenes may be identified and used to calculate the statistics of how the flow vectors magnitude changes over time to detect the change in violence and non-violence behavior. Alternatively, or in addition, the orientation of flow vectors in addition to magnitude values may be used as features to detect violence. Alternatively, or in addition, spatial-temporal features may be extracted from a candidate location to detect violence.

In yet another example, the political classification model may use natural language processing techniques to identify political inclination of a message or a news article, or identify a writer's opinion concerning specific political issues or actions related to those.

FIG.4is a block diagram of the content feed manager235. In some embodiments, the content feed manager235has access to a content repository420containing multiple content items422,424,426. Such content may be organically generated by users using the online system140. In some embodiments, the content feed manager235also receives content from one or more content provider(s)410. The content provider(s)410may include news outlets, brand sites, e-commerce sites, and/or other third-party systems.

The content feed manager235uses the content in the repository420and content received from the content provider(s)410to assemble a content feed for display to a user at client device110. Each content feed includes multiple content items. In some embodiments, the content feed manager235assembles a content feed based on a user profile, a search query entered by a user, or an action log of a user containing a user's historical interactions with different types of content. Alternatively or in addition, the content feed manager235assembles the content feed further based on policies associated with some of the content items.

For example, the content feed manager235may first identify a plurality of content items from the repository420based on the user profile or the user's action log. For example, the user may have interacted with fashion-related content frequently. The content feed manager235selects a plurality of fashion-related content items from the repository420and generates a content feed for the user. Such content items may be organically generated by users of the online system140or obtained from other third-party content providers.

On the other hand, the content feed manager235also receives, from a content provider410, a target content item412to be displayed for the user. The target content412is associated with a policy414. The policy414specifies a prohibited classification (and/or allowed classification) of content that cannot be (and/or can be) published adjacent to the target content item412.

In some embodiments, the policy414is content provider410specific. For example, when the content provider410is a brand site, the brand site may have a single policy that requires all of their content items not to be displayed adjacent to political content. Such policy may be sent to the content feed manager235when the content provider410was registered or set up to provide content items to the content feed manager235. Once the policy is set up, the content provider410may also make changes to its policy later at any time. In some embodiments, the content manager235provides a web portal or an application for content providers to enter or update their policies. The content feed manager235may then store the policies from different content providers in a local storage. In response to receiving a target content412from the content provider410, the content feed manager235retrieves the policy associated with the content provider410from the storage.

In some embodiments, the policy414is content item specific. For example, each content item may be associated with a separate policy. Such a policy may be stored with each content item as metadata. The content provider410is configured to send the content item412and its corresponding policy414together as a single message or two consecutive messages. Responsive to receiving the message(s) containing content item412and/or its corresponding policy414, the content feed manager235parses the message to identify the content item412and its corresponding policy414.

In some embodiments, only certain content items received from content provider410have policies associated therewith. In some embodiments, the content items stored in the content repository420also have policies associated therewith. The content feed manager235is configured to identify all the applicable policies associated with all the content items in a content feed, and make sure that the content feed complies with all the applicable policies.

For example, assuming the content items422,424,426stored in the content repository420are not associated with any policies. The content feed manager235first assembles a content feed selecting a subset of content items422,424from the content repository420. The content feed manager235then identifies a slot432in the content feed. As illustrated, the slot432is between two adjacent content items422,424in the content feed430. The content feed manager235also receives a target content item412and its corresponding policy414from the content providers410. The policy414specifies a prohibited classification of content that cannot be published adjacent to the target content item412.

Because of the existence of the policy414, the content feed manager235needs to make sure that the adjacent content items422,424are not the prohibited classification. To determine whether the adjacent content items422,424are the prohibited classification, the content feed manager235accesses the trained content classification model250, and applies the trained content classification model250to each of the two adjacent content items422,424to determine a classification442,444for each of the two adjacent content items422,424. The classifications442,444for the two adjacent content items422,424are then compared with the prohibited classification specified in the policy414to determine whether the classifications442,444are prohibited or allowed classifications for the target content item412. The content feed manager235then determines whether to place the target content item412in the slot432based on whether either of the adjacent content items422,424is the prohibited classification. Responsive to determining to place the target content item412in the slot432, the content feed manager235inserts the target content item in the slot432and sends the content feed to the client device110for display to the viewing user.

For example, the content provider410may be a brand site, and the policy414may specify that political content cannot be published adjacent to the target content item. The content classification model250determines that none of the adjacent content422,424are political content. Responsive to such a determination, the content feed manager235inserts the content412into the slot432to generate the content feed, and sends the content feed to the client device110for display to the viewing user.

On the other hand, if the content classification model250determines that at least one of the adjacent content422,424is the prohibited classification, the content feed manager235will prevent the content412to be inserted into the slot432. In some embodiments, responsive to determining that at least one of the adjacent content items422,424is the prohibited classification, the content feed manager235requests for a new content item from the same content provider410or a different content provider. The new content item may be associated with a same or a different policy. When the new content item is received, the content feed manager235determines whether the new content item can be inserted in the slot432based on the policy for the new content item.

In some embodiments, responsive to determining that at least one of the adjacent content422,424is the prohibited classification, the content feed manager235requests for a new content item from the same or a different content provider with the classifications of the adjacent content items422,424, causing the content provider to select a new content item that does not have a policy that prohibits the classification of the adjacent content items422,424. Responsive to receiving the new content item, the content feed manager235can then insert the new content item into the slot432, and sends the content feed, including the new content item for display to the viewing user.

In some embodiments, responsive to determining that at least one of the adjacent content422,424is the prohibited classification, the content feed manager235selects one or more new adjacent content items from the repository to replace the adjacent content items422,424. When the new adjacent content items are selected, the content feed manager235applies the content classification model250to the new adjacent content items again to determine whether they are the prohibited classification.

In some embodiments, after the content feed manager235applies the content classification model250to the content items422,424, the classifications of the adjacent content items are stored as metadata with the content items422,424in the repository420. As such, when a content feed is generated to include the same content item422or424again, the content feed manager235can retrieve its classification from the metadata without having to apply the content classification model250again.

In some embodiments, when a content item is generated (e.g., by a user of the online system140), the online system140applies the content classification model250to the content item to determine its classification on the fly, and stores the classification with the content item as metadata. In some embodiments, all the content items422,424,426in the content repository420are batch processed by the content classification model250, and the classification for each content items is stored with the content item as metadata. As such, when any one of the content items422,424,426in the content repository420is selected to form a content feed, its classifications can be obtained directly from the metadata.

In some embodiments, the adjacent content items422,424in a content feed are displayed sequentially in time. The slot432is a time slot between the adjacent content items422,424. In some embodiments, a single content item may be broken down into multiple pieces, and the slot432is a slot between different pieces of the same content item.

In some embodiments, the policy414for the target content item412may have different definitions of being “adjacent.” For example, a policy may define that adjacent means directly adjacent (including before and after) to another content item, and only the directly adjacent content item is required not to be the prohibited classification. As another example, the policy414may define a time frame, within which means adjacent. For example, if a directly adjacent content item is shorter than the time frame, another content item before or after the directly adjacent content item may also be deemed as adjacent based on the policy. As such, the indirectly adjacent content also needs to meet the policy requirement. On the other hand, if a directly adjacent content item is longer than the time frame, only a portion of the directly adjacent content item needs to meet the policy requirement.

In some embodiments, the adjacent content items422,424in a content feed are displayed adjacent to each other physically in a user interface. The slot432is a space slot between the adjacent content items422,424. In some embodiments, the policy414may define adjacent as directly adjacent to another content item physically. In some embodiments, the policy414may define adjacent as within a distance (e.g., within a number of lines of text, within an inch, within a number of content items) from another content item. In such cases, more than two content items may be deemed as adjacent, and each of these content items needs to comply with the policy for the target content item.

In some embodiments, the policy further specifies different classification requirements for different distances. For example, a policy may specify that a direct adjacent content items cannot be classified as bad or medium, and a content item that is further away from the direct adjacent content cannot be classified as bad.

In some embodiments, the policy may specify a score threshold for adjacent content items, including the directly adjacent content items or indirectly adjacent content items. Different weights may be assigned to the different directly or indirectly adjacent content items. The further away the adjacent content item is, the less weight is assigned to the adjacent content item. An overall score may be computed for the adjacent content items based on their classifications and weights. In some embodiments, each content item in the content feed may be considered, and an overall score is generated based on the classification of each content item in the content feed.

Responsive to an overall score of the adjacent content items or the content feed, the content feed manager235may reorganize the content items to move the content items that are a prohibited classification to be further away from the target content item, and the content items that are not the prohibited classification to be closer to the target content item.

In some embodiments, the content feed manager235selects one content item at a time. For example, the content feed manager235selects a first content item422from the content repository420, then receives a target content item412and its corresponding policy414from the content provider410. After the content feed manager235determines that the first content item422complies with the policy414, the content feed manager235appends the target content item412to the first content item422. The content feed manager235then selects a second content item424, and determines whether the second content item424complies with the policy414. If the second content item424also complies with the policy414, the second content item424is appended to the target content item412.

In some embodiments, when an adjacent content item does not comply with the policy414of the target content item412, the adjacent content item is replaced with a new adjacent content item that complies with the policy414, or the target content item412is replaced with a new target content item412associated with a policy that does not prohibit the classification of the adjacent content items422and424. This process can repeat as many times as necessary to build a content feed including any number of content items.

In some embodiments, the content feed manager235selects an adjacent content item, and sends the classification of the content item to one or more content providers. The one or more content providers determine whether they would like to provide a target content item for display next to the adjacent content item. For example, the content feed manager235selects a first adjacent content item422and obtains a classification of the first adjacent content item422(which may be performed by applying the content classification model250on the fly, or retrieving the classification generated by batch processing all the content items422,424,426in the content repository420using the content classification model250ahead of the time). The content feed manager235then sends a request for a target content item to the content provider410, the request including the classification of the adjacent content item422. Responsive to receiving the request, the content provider410determines whether a target content item and/or which target content item is to be provided.

FIG.5is a flowchart of an example method500for generating a content feed for display to a viewing user, according to an embodiment. In other embodiments, the method may include different and/or additional steps than those shown inFIG.5. Additionally, steps of the method may be performed in different orders than the order described in conjunction withFIG.5in various embodiments.

The method500may be performed by the online system140. The online system140receives505, from a content provider (e.g., third party system130, content provider410), a target content item and a policy for the target content item. The policy specifies a prohibited classification of content that cannot be published adjacent to the target content item. For example, the policy may specify that the target content item cannot be displayed adjacent to violent content or political content. In some embodiments, being adjacent means physically adjacent to a content item on a display. In some embodiments, being adjacent means contiguous in time or sequence.

The online system140also generates510, for a viewing user of an online system, a content feed comprising a plurality of content items. In some embodiments, the plurality of content items may be generated organically by users of the online system. Alternatively, or in addition, the plurality of content items may be provided by third party systems, such as a news site, a brand site, an e-commerce site, etc. In some embodiments, the plurality of content items are selected based on the user profile of the viewing user.

In some embodiments, generating the content feed includes accessing a content repository containing a plurality of content items and selecting one or more content items from the plurality of content items to form the content feed. In some embodiments, one or more content items are selected based on a user profile of the viewing user or an action log of the viewing user. The action log of the viewing user records may record information about actions the viewing user performed on the online system, content provider systems, and/or other third party system, including webpage viewing histories, advertisements that were engaged, purchases made, and other patterns from shopping and buying.

The online system140also identifies515a slot in the content feed. The slot is between two adjacent content items of the plurality of content items in the content feed. In some embodiments, each content item is a short video or a portion of a long video. The content feed is configured to sequentially display the plurality of content items. The adjacent content items are the two content items that are to be displayed contiguously in time or sequence. The slot is a time slot between the two adjacent content items. Alternatively, in some embodiments, the adjacent content items are the two content items that are to be simultaneously displayed to the viewing user in a user interface, and the adjacent content items are physically adjacent to the target content item in the user interface. The slot is a space slot between the two adjacent content items on the user interface.

The online system140also520accesses a machine learning model (e.g., machine learning content classification model250) trained to classify whether a content item is the prohibited classification. The machine learning model is trained based on a set of training examples. The training examples include features (e.g., features304,307) about a plurality of other content items (e.g., content items302,305) and labels (e.g., labels303,306) for each of the other content items of whether the other content item is the prohibited classification.

In some embodiments, the features may include content-specific features and/or user engagement features. The content-specific features may include (but are not limited to) features associated with the images or pixels of media content items, such as whether each pixel represents human skin, or whether a magnitude change of a vector over time is greater than a threshold. The content-specific features may also include (but are not limited to) features associated with text contained in the content items, such as whether political terms are shown in the content items. The content specific features may also include features associated with a creator, an original poster, a re-poster, a commenter, and/or a responder of the content item. The user engagement features may include whether most of the users reacted positively (e.g., clicks thumb up, responds with happy emoji) or negatively (e.g., clicks thumb down, responds with angry emoji, or flags the content item as inappropriate) to the content item.

The online system140applies525the trained machine learning model to each of the adjacent content items to determine whether each of the adjacent content items is the prohibited classification. Each of the adjacent content items corresponds to a set of features. The trained machine learning model takes the set of features as input to determine whether the corresponding adjacent content item is a prohibited classification. The prohibited classification may include (but is not limited to) nudity, violence, and/or political. In some embodiments, the policy may further specify a discrete level or a numeric score as a threshold associated with the prohibited classification. For example, the policy may specify that a nudity level of the adjacent content needs to be lower than 5 (out of 10), and a political level of the adjacent content needs to be lower than 3 (out of 10).

In some embodiments, the plurality of content items in the content repository are batch processed by the trained machine learning model. For example, for each of the plurality of content items in the content repository, the machine learning model is applied to the content item to determine a classification of the content item, and the classification is stored as metadata of the content item in the repository.

In some embodiments, the two adjacent content items are indirectly adjacent to the target content, for example, the adjacent items may be N items away from the target content item in the content feed, where N is fewer than a predetermined threshold. In some embodiments, the policy for the target content item is associated with N, where the greater N is, the less strict the policy is. For example, each content item may be classified as very political, medium political, or non-political. When a content item in the content feed is directly adjacent to the target content item, the policy may require that the adjacent content item cannot be very political or medium political. When a content item in the content feed is indirectly adjacent (e.g., less than 2 items away from) to the target content item, the policy may require that the adjacent item cannot be very political, and medium political is allowed. When a content item in the content feed is 2 or more items away from the target content item, the policy may specify no restrictions.

After the online system140applies the trained machine learning model to each of the adjacent content items, the online system140then determines530whether to place the target content item in the slot based on whether either of the adjacent content items is the prohibited classification. In some cases, the policy for the target content item may specify that no more than one of the adjacent content items can be the prohibited classification. In such a case, one of the adjacent content items being the prohibited classification does not prevent the target content item to be placed in the slot. In some cases, the policy for the target content item may specify that none of the adjacent content items can be the prohibited classification. In such a case, one of the adjacent content items being the prohibited classification would prevent the target content item to be placed in the slot.

Responsive to determining to place the target content item in the slot, the online system140inserts535the target content in the slot, and sends540the content feed including the target content item for display to the viewing user. On the other hand, in some embodiments, when the online system140determines not to place the target content item in the slot, the online system140may request from the same content provider or a different content provider a new target content item. Alternatively, the online system140may remove the adjacent content item that is the prohibited classification.

On the other hand, in some embodiments, responsive to determining that at least one of the adjacent content, is the prohibited classification, the online system140requests for a new target content item from the same or a different content provider with the classifications of the adjacent content items, causing the content provider to select a new target content item that does not have a policy that prohibits the classification of the adjacent content items. Responsive to receiving the new target content item, the online system140can simply insert the new content item into the slot, and sends the content feed including the new content item for display to the viewing user.

Alternatively, in some embodiments, responsive to determining that at least one of the adjacent content is the prohibited classification, the online system140selects one or more new adjacent content items from the repository to replace the adjacent content items that are in the prohibited class. When the new adjacent content items are selected, the online system140applies the trained machine learning model to the new adjacent content items again to determine whether they are the prohibited classification. Alternatively, in some embodiments, the classification is stored with the content items as metadata, and the online system140retrieves the metadata associated with the new adjacent content items to determine whether they are the prohibited classification. Alternatively, in some embodiments, the online system140filters the content items based on their classifications to directly select new adjacent content items that are not the prohibited classifications.

In some embodiments, the online system140selects one content item at a time. For example, the online system140selects a first content item from the content repository, then receives a target content item and its corresponding policy from the content provider. After the online system determines that the first content item complies with the policy, the online system then selects a second content item, and determines whether the second content item complies with the policy.

Each time when an adjacent content item does not comply with the policy of the target content item, the adjacent content item is replaced with a new adjacent content item that complies with the policy, or the target content item is replaced with a new target content item associated with a policy that does not prohibit the classification of the adjacent content item. This process may repeat as many times as necessary to assemble a content feed, including any number of content items.

In some embodiments, the content feed manager selects an adjacent content item, and sends the classification of the content item to one or more content providers. The one or more content providers determine whether they would like to provide a target content item for display next to the adjacent content item. For example, the online system140selects a first adjacent content item and obtains a classification of the first adjacent content item (which may be performed by applying the trained machine learning model on the fly, or retrieving the classification generated by batch processing all the content items in the repository using the trained machine learning model ahead of the time). The content feed manager then sends a request for a target content item to the content provider, the request including the classification of the adjacent content item. Responsive to receiving the request, the content provider determines whether a target content item and/or which target content item is to be provided.

Alternative Embodiments