SYSTEMS, METHODS AND DEVICES FOR THE IDENTIFICATION OF CONTENT CREATORS USING MACHINE LEARNING

Described herein are systems, methods, and devices for scoring digital content creators and their creations. The systems, methods, devices describe herein enable the content buyers to describe a desired work product and find the ideal content creator for the project.

BACKGROUND OF THE INVENTION

Digital media creators are continually working to create advertisements, information sources and social media to aid companies in disseminating information and marketing products.

SUMMARY OF THE INVENTION

Digital media content providers are faced with many potential problems when finding personnel to create media. Often, the creation of digital media requires that the creator not only be a skilled artist, but also a skilled computer software user. Given the type of media being created, the creator may also benefit from other traits, such as gender, age, social class, geographic location, ideology, personality, etc. For example, if the digital media content provider was contracted to produce an advertisement to place on the packaging of a children's toy that propels soft darts, the content provider seeks to find the ideal candidate to produce the work or digital media. Currently, a content provider might need to rely on past experience or intuition, rather than data analytics, for the selection process. In this case, the content provider may select a young male creator rather than a more qualified and experienced creator, simply because content provider has limited data and must act on intuition.

Prior technologies suffer from the ability to accurately identify and match content creators with a user's request for media generation. The subject matter, among other items, offers an improved technological tool in the form of a machine learning algorithm trained via supervised learning to more accurately identify and match the most appropriate content creator with a given user's request for media generation. One of the exemplary features of the subject matter is the use of an expanded training set of categories to train the machine learning comprising an internal content rating category; an internal creator rating category, an external creator rating category; an external content rating category; a content performance category; a crowd-sourced content rating category; a reliability score category; a crowd-sourced interest mapping category; an internal interest mapping category; a creator social metric category; a content score; a creator score; an interest graph; a content type score; or any combination thereof. In some embodiments, the trained machine learning algorithm provides a predicted list of ideal content creators for a given user request for media generation. In some embodiments, the user selects one or more content creators from the predicted list. In some embodiments, the user selection is fed back into the machine learning to improve the identification and matching of future content creators with similar users for similar media generation.

The systems, methods, and devices described herein are used to gather data regarding the work product and performance of individual content creators over time, evaluate said data, provide to the user an interface for inputting proposed new project data, and provide to the user a list of possible creator matches for said proposed project, sorted by matchmaking scores. By utilizing the systems, methods, and devices described herein, the digital media content provider can rely on data analytics to aid in the determination and matchmaking of ideal candidates with specific content creation.

Described herein, in certain embodiments, is a computer-implemented system comprising: a digital processing device comprising: at least one processor, an operating system configured to perform executable instructions, a memory, and a computer program including instructions executable by the digital processing device to optimize a computer's effectiveness in identifying content creators comprising: a) a software module configured to receive a request for media generation; b) a software module configured to receive a plurality of content creator data; c) an identification module that applies a machine learning algorithm to process the request for media generation and the plurality of content creator data to identify one or more content creators to perform the request for media generation; d) a validation module that receives identification data following generation of the requested media by the identified content creator; and e) an optimization module that feeds back the identification data into the identification module to improve the machine learning algorithm. Described herein, in certain embodiments, is a computer-implemented system the request for media generation is a request for the generation of any form of media. Described herein, in certain embodiments, is a computer-implemented system wherein the request for media generation comprises one or more of the following: a description of the media, the interests which the media intends to target, the demographic which the media intends to target, and the required content type. Described herein, in certain embodiments, is a computer-implemented system wherein the content creator data comprises one or more of the following: a creator score, an interest graph, and a content type score. Described herein, in certain embodiments, is a computer-implemented system wherein the creator score comprises one or more of the following: a content score, an internal creator score, an external creator score, creator social metrics and a reliability score. Described herein, in certain embodiments, is a computer-implemented system wherein the content score comprises one or more of the following: an internal content rating, an external content rating, content performance data, and a crowd sourced content rating. Described herein, in certain embodiments, is a computer-implemented system wherein the interest graph comprises one or more of the following: crowd sourced interest mapping and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented system wherein the content type score comprises one or more of the following: an internal content rating, an external content rating, and a crowd sourced content rating. Described herein, in certain embodiments, is a computer-implemented system wherein the machine learning algorithm uses an aggregate of two or more the following parameters to identify a content creator: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping, wherein each term is variably weighted. Described herein, in certain embodiments, is a computer-implemented system wherein the machine learning algorithm uses an aggregate of the following parameters to identify a content creator: creator score, interest graph and content type score, wherein each term is variably weighted.

Described herein, in certain embodiments, is a computer-implemented system wherein a plurality of content creators are identified. Described herein, in certain embodiments, is a computer-implemented system wherein the plurality of content creators are ranked according to one or more of the following parameters: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented system wherein the plurality of content creators are ranked according to one of the following parameters: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented system wherein the plurality of content creators are ranked according an aggregate of the following parameters creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping, wherein each parameters is variably weighted. Described herein, in certain embodiments, is a computer-implemented system wherein the plurality of content creators are ranked according an aggregate of the following parameters: creator score, interest graph and content type score, wherein each parameters is variably weighted. Described herein, in certain embodiments, is a computer-implemented system wherein the identification data comprises one or more of the following: the selection of an identified content creator for media generation, the non-selection of an identified content creator for media generation, and any content creator data generated as a result of a selected content creator generating media in response to a request for media generation. Described herein, in certain embodiments, is a computer-implemented system wherein the content creator data generated as a result of new media generation comprises one or more of the following: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented system wherein the content creator data generated as a result of new media generation comprises one or more of the following: content score, internal creator score, external creator score, internal content rating, external content rating, content performance data, and crowd sourced content rating.

Described herein, in certain embodiments, is a computer-implemented method of identifying content creators comprising: a) receiving, by a computer a request for media generation; b) receiving, by a computer a plurality of content creator data; c) applying a machine learning identification algorithm to process the request for media generation and the plurality of content creator data to identify one or more content creators to perform the request for media generation; d) receiving, by a computer identification data following generation of the requested media by the identified content creator; and e) optimizing the machine learning identification algorithm by feeding back the identification data into the machine learning identification algorithm. Described herein, in certain embodiments, is a computer-implemented method wherein the request for media generation is a request for the generation of any form of media. Described herein, in certain embodiments, is a computer-implemented method wherein the request for media generation comprises one or more of the following: a description of the media, the interests which the media intends to target, the demographic which the media intends to target, and the required content type. Described herein, in certain embodiments, is a computer-implemented method wherein the content creator data comprises one or more of the following: a creator score, an interest graph, and a content type score. Described herein, in certain embodiments, is a computer-implemented method wherein the creator score comprises one or more of the following: a content score, an internal creator score, an external creator score, creator social metrics, and a reliability score. Described herein, in certain embodiments, is a computer-implemented method wherein the content score comprises one or more of the following: an internal content rating, an external content rating, content performance data, and a crowd sourced content rating. Described herein, in certain embodiments, is a computer-implemented method wherein the interest graph comprises one or more of the following: crowd sourced interest mapping and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented method wherein the content type score comprises one or more of the following: an internal content rating, an external content rating, and a crowd sourced content rating. Described herein, in certain embodiments, is a computer-implemented method wherein the machine learning algorithm uses an aggregate of two or more the following parameters to identify a content creator: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping, wherein each term is variably weighted. Described herein, in certain embodiments, is a computer-implemented method wherein the machine learning algorithm uses an aggregate of the following parameters to identify a content creator: creator score, interest graph and content type score, wherein each term is variably weighted. Described herein, in certain embodiments, is a computer-implemented method wherein a plurality of content creators are identified. Described herein, in certain embodiments, is a computer-implemented method wherein the plurality of content creators are ranked according to one or more of the following parameters: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented method wherein the plurality of content creators are ranked according to one of the following parameters: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented method wherein the plurality of content creators are ranked according an aggregate of the following parameters creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping, wherein each parameters is variably weighted. Described herein, in certain embodiments, is a computer-implemented method wherein the plurality of content creators are ranked according an aggregate of the following parameters: creator score, interest graph and content type score, wherein each parameters is variably weighted. Described herein, in certain embodiments, is a computer-implemented method wherein the identification data comprises one or more of the following: the selection of an identified content creator for media generation, the non-selection of an identified content creator for media generation, and any content creator data generated as a result of a selected content creator generating media in response to a request for media generation. Described herein, in certain embodiments, is a computer-implemented method wherein the content creator data generated as a result of new media generation comprises one or more of the following: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a computer-implemented method wherein the content creator data generated as a result of new media generation comprises one or more of the following: content score, internal creator score, external creator score, internal content rating, external content rating, content performance data, and crowd sourced content rating.

Described herein, in certain embodiments, is a non-transitory computer-readable storage media encoded with a computer program including instructions executable by a processor to create to optimize a computer's effectiveness in identifying content creators comprising: a) a software module configured to receive a request for media generation; b) a software module configured to receive a plurality of content creator data; c) an identification module that applies a machine learning algorithm to process the request for media generation and the plurality of content creator data to identify one or more content creators to perform the request for media generation; d) a validation module that receives identification data following generation of the requested media by the identified content creator; and e) an optimization module that feeds back the identification data into the identification module to improve the machine learning algorithm. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the request for media generation is a request for the generation of any form of media. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the request for media generation comprises one or more of the following: a description of the media, the interests which the media intends to target, the demographic which the media intends to target, and the required content type. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the content creator data comprises one or more of the following: a creator score, an interest graph, and a content type score. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the creator score comprises one or more of the following: a content score, an internal creator score, an external creator score, creator social metrics, and a reliability score. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the content score comprises one or more of the following: an internal content rating, an external content rating, content performance data, and a crowd sourced content rating. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the interest graph comprises one or more of the following: crowd sourced interest mapping and internal interest mapping. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the content type score comprises one or more of the following: an internal content rating, an external content rating, and a crowd sourced content rating. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the machine learning algorithm uses an aggregate of two or more the following parameters to identify a content creator: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping, wherein each term is variably weighted. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the machine learning algorithm uses an aggregate of the following parameters to identify a content creator: creator score, interest graph and content type score, wherein each term is variably weighted. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein a plurality of content creators are identified. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the plurality of content creators are ranked according to one or more of the following parameters: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the plurality of content creators are ranked according to one of the following parameters: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the plurality of content creators are ranked according an aggregate of the following parameters creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping, wherein each parameters is variably weighted. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the plurality of content creators are ranked according an aggregate of the following parameters: creator score, interest graph and content type score, wherein each parameters is variably weighted.

Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the identification data comprises one or more of the following: the selection of an identified content creator for media generation, the non-selection of an identified content creator for media generation, and any content creator data generated as a result of a selected content creator generating media in response to a request for media generation. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the content creator data generated as a result of new media generation comprises one or more of the following: creator score, interest graph, content type score, content score, internal creator score, external creator score, creator social metrics, reliability score, internal content rating, external content rating, content performance data, crowd sourced content rating, crowd sourced interest mapping, and internal interest mapping. Described herein, in certain embodiments, is a non-transitory computer-readable storage media wherein the content creator data generated as a result of new media generation comprises one or more of the following: content score, internal creator score, external creator score, internal content rating, external content rating, content performance data, and crowd sourced content rating.

Another aspect provided herein is a method for adaptive identifying content creators data selection through a network appliance connected between computing devices in a network, the method comprising: collecting, from a content creator, by the network appliance, a crowd-sourced content rating, collecting from an internal editor, by the network appliance, an internal content rating, and an internal creator rating; collecting from an brand user, by the network appliance, an external content rating and an external creator rating; determining, by the computer device, a content performance, a reliability score, and a creator social metric based on a social media data; calculating, by the computer device, a content score based on the internal content rating, the external content rating, the content performance, and the crowd-sourced content rating; calculating, by the computer device, a creator score based on the content score, the internal creator rating, the external creator rating, the reliability score, and the creator social metric; collecting at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if at least one of the content score and the creator score are below a first predefined threshold.

In some embodiments, at least one of the internal creator rating and the external creator rating are associated with a creator profile. In some embodiments, at least one of the crowd-sourced content rating, internal content rating, external content rating, are associated with a creator profile are associated with a content of a campaign. In some embodiments, the method further comprises collecting from a content creator, by the network appliance, at least one of a crowd sourced interest map and an internal interest map. In some embodiments, the method further comprises calculating an interest graph from at least one of the crowd sourced interest map and the internal interest map. In some embodiments, the method further comprises collecting at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if the interest graph is below a second predefined threshold. In some embodiments, the method further comprises calculating a content type score from at least one of the internal content rating, the external content rating, and the crowd-sourced content rating. In some embodiments, the method further comprises collecting at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if the content type score is below a third predefined threshold.

Another aspect provided herein is a method of rearranging content and creator icons on a graphical user interface (GUI) of a computer system, the method comprising: collecting from a content creator, via the GUI, a crowd-sourced content rating associated with a content of a campaign; collecting from an internal editor, via the GUI, an internal content rating associated with the content of the campaign, and an internal creator rating associated with the creator; collecting from an brand user, via the GUI, an external content rating associated with the content of the campaign and an external creator rating associated with the creator; determining, by the computer system, a content performance, a reliability score, and a creator social metric based on a social media data; calculating, by the computer system, a content score based on the internal content rating, the external content rating, the content performance, and the crowd-sourced content rating; and calculating, by the computer system, a creator score based on the content score, the internal creator rating, the external creator rating, the reliability score, and the creator social metric; automatically arranging an icon associated with the content creator and an icon associated with the content of the campaign based on the content score, the creator score, or both.

In some embodiments, the method further comprises collecting from a content creator, via the GUI, at least one of a crowd sourced interest map and an internal interest map. In some embodiments, the method further comprises calculating an interest graph from at least one of the crowd sourced interest map and the internal interest map. In some embodiments, the method further comprises collecting at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if the interest graph is below a second predefined threshold. In some embodiments, the method further comprises calculating a content type score from at least one of the internal content rating, the external content rating, and the crowd-sourced content rating. In some embodiments, the method further comprises collecting at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if the content type score is below a third predefined threshold.

Another aspect provided herein is a method for adaptive identifying content creators data selection through a network appliance connected between computing devices in a network, the method comprising: receiving, by the network appliance, an internal content rating (A), an internal creator rating (B), an external creator rating (C), an external content rating (D), and a crowd-sourced content rating (E); determining, by a first computer device, a content performance (F), a reliability score (G), and a creator social metric (H) based on a social media data; calculating, a content score (I), wherein I=X1*A+X2*D+X3*E+X+*F, wherein X1, X2, X3, and X4are content score weights; calculating, a creator score (J), wherein J=Y1*I+Y2*B+Y3*C+Y4*G+Y5*H, wherein Y1, Y2, Y3, Y4, and Y5are creator score weights; and transmitting the content score and the creator score to a second computer device.

In some embodiments, at least one of the internal creator rating and the external creator rating are associated with a creator profile. In some embodiments, at least one of the crowd-sourced content rating, internal content rating, external content rating, are associated with a creator profile are associated with a content of a campaign. In some embodiments, the method further comprises receiving by the network appliance a crowd sourced interest map (K) and an internal interest map (L). In some embodiments, the method further comprises calculating an interest graph (M) from at least one of the crowd sourced interest map (N) and the internal interest map (O), wherein M=Z1*N+Z2*O and wherein Z1and Z5are interest graph weights. In some embodiments, the method further comprises calculating a content type score (P) from at least one of the internal content rating (A), the external content rating (D), and the crowd-sourced content rating (E), wherein P=Q1*A+Q2*D+Q3*E and wherein Q1, Qs, and Q3are content type score weights.

Another aspect provided herein is a computer-implemented system comprising: a computer-readable storage device coupled to the at least one processor and having instructions stored thereon which, when executed by the at least one processor, cause the at least one processor to perform operations comprising: receiving, from a user interface, a request for media generation; determining a plurality of content creators based on the request for media generation; training a machine learning algorithm using the plurality of content creators; receiving, from a user interface, a selection of one or more selected content creators of the content creators of the plurality of content creators; assigning unrefined weights to a predictor variable of the machine learning algorithm; processing the request for media generation through the machine learning algorithm to determine a plurality of recommended options, the recommended options comprising one or more content creators; providing the recommended options to the user interface; receiving, from the user interface, a data set comprising selections of the one or more content creators; adjusting the predictor variables of the machine learning algorithm based on the selections of the one or more content creators; feeding back the data set, and the request for media generation through the machine learning algorithm to improve the ability of the machine learning algorithm to determine a plurality of recommended options.

In some embodiments, the request for media generation is a request for the generation of any form of media. In some embodiments, the request for media generation comprises one or more of the following: a description of the media, the interests which the media intends to target, the demographic which the media intends to target, and a required content type. In some embodiments, each of the plurality of content creators are associated with a content creator data comprising one or more of a creator score, and an interest graph. In some embodiments, the creator score comprises one or more of the following: a content score, an internal creator score, an external creator score, creator social metrics and a reliability score. In some embodiments, the interest graph comprises one or more of the following: crowd sourced interest mapping and internal interest mapping. In some embodiments, the predictor variables comprise weights associated with the content creator data. In some embodiments, the request for media generation is associated with a content score. In some embodiments, the content score comprises one or more of the following: an internal content rating, an external content rating, content performance data, and a crowd sourced content rating. In some embodiments, the predictor variables comprise weights associated with the content score.

Another aspect provided herein is a computer-implemented method of training a machine learning algorithm for identifying content creators comprising: collecting a plurality of content creator data and a plurality of requests for media generation, wherein each request for media generation is associated with a selected creator data; create a first training set comprising the plurality of selected content creator data, the plurality of requests for media generation, and a plurality of non-selected content creator data; training the machine learning algorithm in a first stage using the first training set; create a second training set comprising the first training set and the non-selected content creator data that are incorrectly identified as a selected content creator training the neural network in a second stage using the second training set;

The system of claim11, wherein the request for media generation is a request for the generation of any form of media. In some embodiments, the request for media generation comprises one or more of the following: a description of the media, the interests which the media intends to target, the demographic which the media intends to target, and a required content type. In some embodiments, each of the plurality of content creators are associated with a content creator data comprising one or more of a creator score, and an interest graph. In some embodiments, the creator score comprises one or more of the following: a content score, an internal creator score, an external creator score, creator social metrics and a reliability score. In some embodiments, the interest graph comprises one or more of the following: crowd sourced interest mapping and internal interest mapping. In some embodiments, the predictor variables comprise weights associated with the content creator data. In some embodiments, the request for media generation is associated with a content score. In some embodiments, the content score comprises one or more of the following: an internal content rating, an external content rating, content performance data, and a crowd sourced content rating. In some embodiments, the predictor variables comprise weights associated with the content score.

DETAILED DESCRIPTION OF THE INVENTION

Described herein, in certain embodiments, are systems, methods, and devices used to identify skilled content creators, and match them with buyers of custom content, (“Buyers”). In some embodiments, the systems, methods, and devices used herein utilize various sources of information in the matchmaking process.FIG.1identifies exemplary key metrics used to identify content creators.

Identification of Content Creators

In order to identify the best content creators, the methods and systems herein utilize, in some embodiments, multiple sources of information, scores and aggregate scores.

In some embodiments, the systems, methods, and devices used herein utilize an internal content rating score (101) comprising a score received by a system author regarding an individual piece of content on a numerical scale. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the internal content rating score (101) is based on a variety of factors including, but not limited to, aesthetics, creativity, composition, and adherence to guidelines.

In some embodiments, the systems, methods, and devices used herein utilize an internal creator rating score (102) comprising a score received by a system author regarding a creator on a numerical scale. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the internal creator rating score (102) is based on a variety of factors including, but not limited to, social reach and engagement, and social profile content.

In some embodiments, the systems, methods, and devices used herein utilize an external creator rating score (103) comprising a score received by a brand user regarding a creator on a numerical scale. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the external creator rating score (103) is based on a variety of factors including, but not limited to, aesthetics, creativity, composition, social profile content, and adherence to guidelines.

In some embodiments, the systems, methods, and devices used herein utilize an external content rating score (104), comprising a score received by a brand user regarding an individual content on a numerical scale. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the external content rating score (104) is based on a variety of factors including, but not limited to, social reach and engagement.

In some embodiments, the systems, methods, and devices used herein utilize a content performance score (105), comprising numerical performance metrics on the engagement of custom content from social media platform. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the content performance score (105) is received via an application program interface (API). Content performance scores (105) include reach, engagement (made of comment ratio, like ratio, share ratio), click-through rate, ad performance, and conversion rate.

In some embodiments, the systems, methods, and devices used herein utilize a crowd-sourced content rating score (106), comprising a score received by another creator regarding an individual content on a numerical scale. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the crowd-sourced content rating score (106) is based on a variety of factors including, but not limited to, aesthetics, creativity, composition, and adherence to guidelines.

In some embodiments, the systems, methods, and devices used herein utilize a reliability score (107), comprising a score determined by campaign performance a numerical scale. In some embodiments, the numerical scale is a scale from 1-3, 1-5, or 1-10. In some embodiments, the reliability score (107) is based on, but not limited to, adherence to guidelines, timeliness of content creation, content timeliness, and the number of required iterations to create approved content.

In some embodiments, the systems, methods, and devices used herein utilize a crowd-sourced interest mapping score (108), wherein the crowd-sourced interest mapping score comprises scores received by other creators regarding a creator based on data gathered from social content, bio, descriptions and commentary generated by the creators work.

In some embodiments, the systems, methods, and devices used herein utilize an internal interest mapping score (109), comprising an interest selected by or assigned to a creator based on perceived creator interests, social content, bio, descriptions, and commentary generated by the creators work.

In some embodiments, the systems, methods, and devices used herein utilize an internal creator social metric (110), wherein the creator social metric comprises third parties interaction with a creator's media. In some embodiments, the creator social metric comprises social media follower count, a creators engagement in social media, social media generated as a result of the creator or the creators work, comments generated as a result of the creator or the creators work, or sentiment related to the creator or the creators work.

In some embodiments, a Content Score (111) is used as a scoring aggregate to aid in matchmaking. In some embodiments, the Content Score is calculated as follows: Content Score=A (Internal Content Rating)*/+B (External Content Rating)*/+C (Content Performance)*/+D (Crowd-Sourced Content Rating). Each individual piece of content on the system gets its own Content Score and is updated as new content ratings are received.

In some embodiments, a creator score (112) is used as a scoring aggregate to aid in matchmaking. In some embodiments, the creator score is calculated as follows: Creator Score=A (Content Score)*/+B (Internal Creator Rating)*/+C (External Creator Rating)*/+D (Reliability Score). Each individual creator on the system has its own creator score that gets updated as new content is created and new ratings are received.

In some embodiments, an interest graph (113) is created. In some embodiments, the interest graph is a set of individual interest scores. For example, a creator may have a score of 8.7 for pets, and 6.5 for DIY, and 4.3 beauty, and 4.2 for fitness. In some embodiments, an Interest Score is calculated as follows: Interest Score=A (Crowd-Source Interest Mapping)*/+B (Internal Interest mapping).

In some embodiments, a content type mapping is created. In some embodiments, the content type mapping is a set of content type scores (114). In some embodiments, each content type (e.g. image, short-form video, long-form video, reviews, animated GIF) is given a score based on A (Internal Content Rating)*/+B (External Content Rating)*/+C (Content Performance)*/+D (Crowd-sourced Content Rating)

In some embodiments, the process of matchmaking is described inFIG.2. In the process, a buyer will create a brief which contains a description (text form explanation for the content they wish to purchase), target interests (the type of audience), required content type (e.g. image, short-form video, long-form video, reviews, animated GIF), and target demographics (age, gender, location) (201). In some embodiments, the system author has collected data for use in the determination of creators' ratings (202). For example, in one embodiment, each content creator has an associated creator score, interest graph and content type score (203). In some embodiments, the system computes a brief creator score using the following formula: Brief Creator Score=A (Creator Score)*/+B (Interest Graph)*/+C (Content Type Score) (204). The Brief Creator Score is 0 if the specified demographics do not match the creator's demographics. In some embodiments, the system ranks each of the creators by the Brief Creator Score in descending order. The system applies a filter for the best matched creators by selecting either a top percentage or top # of creators or a predetermined threshold for the brief based on how much content is required in the Brief (205).

In some embodiments, the systems and methods herein collect at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if at least one of the content score and the creator score are below a first predefined threshold. In some embodiments, the systems and methods herein collect at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if the interest graph is below a second predefined threshold. In some embodiments, the systems and methods herein collect at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings if the content type score is below a third predefined threshold. In some embodiments, at least one of the first predefined threshold, the second predefined threshold, and the third predefined threshold comprise a score of about 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, or more including increments therein. In some embodiments, at least one of the first predefined threshold, the second predefined threshold, and the third predefined threshold is determined by a machine learning algorithm based on at least one of additional crowd-sourced content ratings, internal content ratings, internal creator ratings, external content ratings, and the external creator ratings.

In some embodiments, machine learning algorithms are utilized to aid in creator performance evaluation. In some embodiments, machine learning algorithms are utilized to aid in creator content evaluation. In some embodiments, machine learning algorithms are utilized to aid in new project evaluation. In some embodiments, machine learning algorithms are used utilized to aid in matching a content creator with a new project.

Examples of machine learning algorithms may include a support vector machine (SVM), a naïve Bayes classification, a random forest, a neural network, deep learning, or other supervised learning algorithm or unsupervised learning algorithm for classification and regression. The machine learning algorithms may be trained using one or more training datasets.

FIG.3shows a non-limiting example of an interface300for selecting a reason for an external creator rating. As shown, the exemplary reasons for an external creator rating comprise rating the content of a creator311, rating the demographic of the creator312, and rating the engagement313of the creator. Alternatively, in some embodiments, the reasons for an external creator rating comprise any additional feedback. As shown, an interface for selecting the exemplary reasons for an external creator rating further comprises a username of the creator301, a description of the creator302, creator information (e.g. a location of the creator, a number of creator followers, a number of created posts, a country, an age, a relationship, a number of children)303, an interest of the creator304, an engagement rate306, an average engagement rate307, and a brand304worked on by the creator.

FIG.4shows a non-limiting example of an interface400for selecting the external content rating. As shown, the exemplary interface for selecting the external content rating comprises a client name401, a project name402, a creator username403, a number of approved creator content404, a number of creator content waiting for approval405, a goal number of creator content406, a product image407, a product image caption408, a reviewed-by date409, and an icon to rate the content410.

FIG.5shows a non-limiting example of an interface500for selecting the external creator rating. As seen, the exemplary interface for selecting the external creator rating comprises a username of the creator501, a description of the creator502, creator information (e.g. a location of the creator, a number of creator followers, a number of created posts, a country, an age, a relationship, a number of children)503, an interest of the creator504, an engagement rate505, an average engagement rate506, and a brand worked on by the creator507for one or more creators.

FIG.6shows a non-limiting example of an interface600for selecting the rating of an internal content rating. As shown, the exemplary interface for selecting the rating of an internal content rating comprises a content image601, a rating selection panel602, a do's list603and a don'ts list604. As shown, the rating selection panel comprises selections from 1-5. In some embodiments, the rating selection panel comprises selections from 1-3, 1-4, 1-5, 1-6, 1-7, 1-8, 1-9, 1-10 or more selections. In some embodiments, a selection of 1 is a positive selection indicating satisfaction with the internal content. In some embodiments, a selection of 1 is a negative selection indicating a lack of satisfaction with the internal content. As shown, the do's list comprises a list of requirements613for the internal content. In some embodiments, the don'ts list comprises a list of negative requirements of the internal content.

FIG.7shows a non-limiting example of an interface700for selecting the internal content rating. As shown, the interface for selecting the internal content rating comprises a creator username701, a creator email address702, a customer703, a project name704, a number of creator followers705, a number of creator posts706, creator information (e.g. a number of engagements, an creator engagement rate, a creator gender, a creator age, a creator location)707, a creator interest708, a creator tag709, and a creator content rating selection pane710. As shown the creator content rating selection pane allows internal users to select a quality associated with the content. As shown, the internal user can further approve or reject the content.

FIG.8shows a non-limiting example of an interface800for viewing content performance data. As shown the exemplary interface for viewing content performance data comprises a client name801, a project name802, an engagements indication803, an engagement rate indication804, a number of creators805, a number of content created806, a number of people reached with the content807, a number of days left in the campaign808, a number of likes809, a number of comments810, a bid price811, an eCPPC price812, a percent delivered813, and a total budget814. In some embodiments, individual content and global campaign performance metrics are leveraged to drive matching.

FIG.9shows a non-limiting example of an interface900for selecting a crowd sourced content rating. As shown, the exemplary interface for selecting a crowd sourced content rating comprises an image901and a rating bar901. In some embodiments, the rating bar allows a creator to rate content from other creators to build a crowd sourced model of quality. As shown, the rating bar allows a creator to enter a rating of 1-5, wherein 1 is a poor rating and 5 is an excellent rating. Alternatively, in some embodiments, the rating bar allows a creator to enter a rating of 1-3, 1-6, 1-7, 1-8 or 1-10.

FIG.10shows a non-limiting example of an interface1000for selecting a reason for a crowd sourced content rating. As shown the exemplary interface for selecting a reason for a crowd sourced content rating comprises the rating bar1001, a feedback form1002, and a submit button1003. Further, as shown the feedback form allows a creator to select one or more feedback reasons of what could have been better with the content. In some embodiments, the one or more feedback reasons comprise a behavior, a communication, an accuracy, a timing, or another reason.

FIG.11shows a non-limiting example of an interface1100for selecting the crowd-sourced interest mapping. As seen the exemplary interface for selecting the crowd-sourced interest mapping allows a creator to select and/or validate other creators' interest to ensure their interests match their respective portfolios. As shown, the interests1101comprise art and design, auto, beauty, books, technology, DIY, travel, family, fashion, fitness, food, gaming, lifestyle, entertainment, sports, photography, and pets.

FIG.12shows a non-limiting example of an interface1200for displaying a reliability score. As seen the exemplary interface for displaying a reliability score shows creator information1201(e.g. a creator email, a creator image, a lead completion percentage, a number of successful posts, an IG average PQ, an IG publisher rank, a creator location, a creator tag, a creator age, a creator gender), a creator reliability ratio1202(including data such as a creator invite to accept ratio, a creator accept to approved ratio, an approved to creator content ratio, a creator content to post ratio, a creator rejection rate, a creator fix needed rate, a creator predicted vs actual value), user information1203(including data such as a profile tag, an interest tag, a creator registration date, a creator name, a creator mailing address, and a creator phone number). In some embodiments, the creator invite to accept ratio reflects a ratio between a number invitations sent to a creator to provide a content for a campaign and a number of campaigns accepted by the creator. In some embodiments, the creator accept to approved ratio, reflects a ratio between a number of campaigns accepted and a number of campaigns approved for initiation. In some embodiments, approved to creator content ratio reflects a ratio between a number of campaigns approved for initiation and a number of campaigns created. in some embodiments, the creator content to post ratio reflects a ratio a number of campaigns created and a number of posts created. In some embodiments, the creator rejection rate reflects a number of contents provided by the creator that were rejected by the buyer. In some embodiments, the creator fix needed rate reflects a number of amendments required by the buyer. In some embodiments, the reliability ratio is calculated based on one or more of the creator invite to accept ratio, creator accept to approved ratio, approved to creator content ratio, creator content to post ratio, creator rejection rate, or the creator fix needed rate. In some embodiments, the creator predicted vs actual value displays a positive or negative difference between a predicted reliability ratio and the calculated reliability ratio. In some embodiments, the predicted reliability ratio is determined based on the previous content, the creator location, the creator tag, the creator age, the creator gender, the profile tag, an interest tag, the creator registration date, the creator name, the creator mailing address, the creator phone number, or any combination thereof.

FIG.13shows a non-limiting example of an interface1300for selecting an internal interest. As shown, the exemplary interface for selecting an internal interest comprises a client name1301, a project name1302, a creator name1303, a creator blurb1304, a content image1305, a content caption1306, a post accepted notification1307, and a content tagging input region1308. As shown, the content tagging input region enables interest mapping based on validation/correction of ML selected interest based on created content. Further, as shown the content tags comprise a pet, a male, a female, a family, an art, a beauty, a vehicle, an entertainment, a landscape/scenery, a fitness, a sports/outdoors, a food and drinks, a hobby and activity, a fashion, a business/tech, and a miscellaneous tag.

In some embodiments, the machine-learning model herein predictively identifies and matches the best creators for a specific brief. In some embodiments, this is a supervised machine-learning model that is trained using extensive data inputs (listed) to predict the best creators based on the attributes of the brief. In some embodiments, this model is measured against a verified test set that consists of creator matches for a variety of briefs. In some embodiments, the machine has been trained to achieve a high-degree of accuracy against the test set. Additionally, in some embodiments, the model also adjusts its predictions based on real-time selections that are made by the buyer during the creator selection process.

Terms and Definitions

As used herein, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. Any reference to “or” herein is intended to encompass “and/or” unless otherwise stated.

As used herein, the term “about” refers to an amount that is near the stated amount by 10%, 5%, or 1%, including increments therein.

Machine Learning

In some embodiments, machine learning algorithms are utilized to optimize a computer's effectiveness in identifying content creators. In some embodiments, machine learning algorithms are utilized by the identification module to process the request for media generation and the plurality of content creator data to identify one or more content creators to perform the request for media generation. In some embodiments, the machine learning algorithms utilized by the identification module employ one or more forms of labels including but not limited to human annotated labels and semi-supervised labels. The human annotated labels can be provided by a hand-crafted heuristic. For example, the hand-crafted heuristic can comprise examining differences between public and county records. The semi-supervised labels can be determined using a clustering technique to find properties similar to those flagged by previous human annotated labels and previous semi-supervised labels. The semi-supervised labels can employ a XGBoost, a neural network, or both.

In some embodiments, machine learning algorithms are utilized by the identification module to process the request for media generation and the plurality of content creator data to identify one or more content creators to perform the request for media generation, using a distant supervision method. The distant supervision method can create a large training set seeded by a small hand-annotated training set. The distant supervision method can comprise positive-unlabeled learning with the training set as the ‘positive’ class. The distant supervision method can employ a logistic regression model, a recurrent neural network, or both. The recurrent neural network can be advantageous for Natural Language Processing (NLP) machine learning.

Examples of machine learning algorithms can include a support vector machine (SVM), a naïve Bayes classification, a random forest, a neural network, deep learning, or other supervised learning algorithm or unsupervised learning algorithm for classification and regression. The machine learning algorithms can be trained using one or more training datasets.

In some embodiments, the machine learning algorithm utilizes regression modeling, wherein relationships between predictor variables and dependent variables are determined and weighted. In some embodiments, a machine learning algorithm is used to select catalogue images and recommend project scope. A non-limiting example of a multi-variate linear regression model algorithm is seen below: probability=A0+A1(X1)+A2(X2)+A3(X3)+A4(X4)+A5(X5)+A6(X6)+A7(X7) . . . wherein A1(A1, A2, A3, A4, A5, A6, A7, . . . ) are “weights” or coefficients found during the regression modeling; and Xi(X1, X2, X3, X4, X5, X6, X7, . . . ) are data collected from the User. Any number of Aiand Xivariable can be included in the model. For example, in a non-limiting example wherein there are 7 Xiterms, X1is the number of creators and X2is the number of content campaigns. In some embodiments, the programming language “R” is used to run the model.

In some embodiments, training comprises multiple steps. In a first step, an initial model is constructed by assigning probability wrights to predictor variables. In a second step, the initial model is used to “recommend” content creators. In a third step, the validation module accepts identification data and feeds back the verified data to identification module calculation. At least one of the first step, the second step, and the third step can repeat one or more times continuously or at set intervals.

In some embodiments, a brief creator score based the information in the brief, and weights applied to the internal content rating, the internal creator rating, the external creator rating, the external content rating, the content performance, the crowd-sourced content rating, the reliability score, and creator social metrics. In some embodiments, the weights comprise predictor variables.

Computing System

Referring toFIG.14, a block diagram is shown depicting an exemplary machine that includes a computer system1400(e.g., a processing or computing system) within which a set of instructions can execute for causing a device to perform or execute any one or more of the aspects and/or methodologies for static code scheduling of the present disclosure. The components inFIG.14are examples only and do not limit the scope of use or functionality of any hardware, software, embedded logic component, or a combination of two or more such components implementing particular embodiments.

Computer system1400may include one or more processors1401, a memory1403, and a storage1408that communicate with each other, and with other components, via a bus1440. The bus1440may also link a display1432, one or more input devices1433(which may, for example, include a keypad, a keyboard, a mouse, a stylus, etc.), one or more output devices1434, one or more storage devices1435, and various tangible storage media1436. All of these elements may interface directly or via one or more interfaces or adaptors to the bus1440. For instance, the various tangible storage media1436can interface with the bus1440via storage medium interface1426. Computer system1400may have any suitable physical form, including but not limited to one or more integrated circuits (ICs), printed circuit boards (PCBs), mobile handheld devices (such as mobile telephones or PDAs), laptop or notebook computers, distributed computer systems, computing grids, or servers.

Computer system1400includes one or more processor(s)1401(e.g., central processing units (CPUs) or general purpose graphics processing units (GPGPUs)) that carry out functions. Processor(s)1401optionally contains a cache memory unit1402for temporary local storage of instructions, data, or computer addresses. Processor(s)1401are configured to assist in execution of computer readable instructions. Computer system1400may provide functionality for the components depicted inFIG.14as a result of the processor(s)1401executing non-transitory, processor-executable instructions embodied in one or more tangible computer-readable storage media, such as memory1403, storage1408, storage devices1435, and/or storage medium1436. The computer-readable media may store software that implements particular embodiments, and processor(s)1401may execute the software. Memory1403may read the software from one or more other computer-readable media (such as mass storage device(s)1435,1436) or from one or more other sources through a suitable interface, such as network interface1420. The software may cause processor(s)1401to carry out one or more processes or one or more steps of one or more processes described or illustrated herein. Carrying out such processes or steps may include defining data structures stored in memory1403and modifying the data structures as directed by the software.

The memory1403may include various components (e.g., machine readable media) including, but not limited to, a random access memory component (e.g., RAM1404) (e.g., static RAM (SRAM), dynamic RAM (DRAM), ferroelectric random access memory (FRAM), phase-change random access memory (PRAM), etc.), a read-only memory component (e.g., ROM1405), and any combinations thereof. ROM1405may act to communicate data and instructions unidirectionally to processor(s)1401, and RAM1404may act to communicate data and instructions bidirectionally with processor(s)1401. ROM1405and RAM1404may include any suitable tangible computer-readable media described below. In one example, a basic input/output system1406(BIOS), including basic routines that help to transfer information between elements within computer system1400, such as during start-up, may be stored in the memory1403.

Fixed storage1408is connected bidirectionally to processor(s)1401, optionally through storage control unit1407. Fixed storage1408provides additional data storage capacity and may also include any suitable tangible computer-readable media described herein. Storage1408may be used to store operating system1409, executable(s)1410, data1411, applications1412(application programs), and the like. Storage1408can also include an optical disk drive, a solid-state memory device (e.g., flash-based systems), or a combination of any of the above. Information in storage1408may, in appropriate cases, be incorporated as virtual memory in memory1403.

In one example, storage device(s)1435may be removably interfaced with computer system1400(e.g., via an external port connector (not shown)) via a storage device interface1425. Particularly, storage device(s)1435and an associated machine-readable medium may provide non-volatile and/or volatile storage of machine-readable instructions, data structures, program modules, and/or other data for the computer system1400. In one example, software may reside, completely or partially, within a machine-readable medium on storage device(s)1435. In another example, software may reside, completely or partially, within processor(s)1401.

Computer system1400may also include an input device1433. In one example, a user of computer system1400may enter commands and/or other information into computer system1400via input device(s)1433. Examples of an input device(s)1433include, but are not limited to, an alpha-numeric input device (e.g., a keyboard), a pointing device (e.g., a mouse or touchpad), a touchpad, a touch screen, a multi-touch screen, a joystick, a stylus, a gamepad, an audio input device (e.g., a microphone, a voice response system, etc.), an optical scanner, a video or still image capture device (e.g., a camera), and any combinations thereof. In some embodiments, the input device is a Kinect, Leap Motion, or the like. Input device(s)1433may be interfaced to bus1440via any of a variety of input interfaces1423(e.g., input interface1423) including, but not limited to, serial, parallel, game port, USB, FIREWIRE, THUNDERBOLT, or any combination of the above.

In particular embodiments, when computer system1400is connected to network1430, computer system1400may communicate with other devices, specifically mobile devices and enterprise systems, distributed computing systems, cloud storage systems, cloud computing systems, and the like, connected to network1430. Communications to and from computer system1400may be sent through network interface1420. For example, network interface1420may receive incoming communications (such as requests or responses from other devices) in the form of one or more packets (such as Internet Protocol (IP) packets) from network1430, and computer system1400may store the incoming communications in memory1403for processing. Computer system1400may similarly store outgoing communications (such as requests or responses to other devices) in the form of one or more packets in memory1403and communicated to network1430from network interface1420. Processor(s)1401may access these communication packets stored in memory1403for processing.

Examples of the network interface1420include, but are not limited to, a network interface card, a modem, and any combination thereof. Examples of a network1430or network segment1430include, but are not limited to, a distributed computing system, a cloud computing system, a wide area network (WAN) (e.g., the Internet, an enterprise network), a local area network (LAN) (e.g., a network associated with an office, a building, a campus or other relatively small geographic space), a telephone network, a direct connection between two computing devices, a peer-to-peer network, and any combinations thereof. A network, such as network1430, may employ a wired and/or a wireless mode of communication. In general, any network topology may be used.

Information and data can be displayed through a display1432. Examples of a display1432include, but are not limited to, a cathode ray tube (CRT), a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic liquid crystal display (OLED) such as a passive-matrix OLED (PMOLED) or active-matrix OLED (AMOLED) display, a plasma display, and any combinations thereof. The display1432can interface to the processor(s)1401, memory1403, and fixed storage1408, as well as other devices, such as input device(s)1433, via the bus1440. The display1432is linked to the bus1440via a video interface1422, and transport of data between the display1432and the bus1440can be controlled via the graphics control1421. In some embodiments, the display is a video projector. In some embodiments, the display is a head-mounted display (HMD) such as a VR headset. In further embodiments, suitable VR headsets include, by way of non-limiting examples, HTC Vive, Oculus Rift, Samsung Gear VR, Microsoft HoloLens, Razer OSVR, FOVE VR, Zeiss VR One, Avegant Glyph, Freefly VR headset, and the like. In still further embodiments, the display is a combination of devices such as those disclosed herein.

In addition to a display1432, computer system1400may include one or more other peripheral output devices1434including, but not limited to, an audio speaker, a printer, a storage device, and any combinations thereof. Such peripheral output devices may be connected to the bus1440via an output interface1424. Examples of an output interface1424include, but are not limited to, a serial port, a parallel connection, a USB port, a FIREWIRE port, a THUNDERBOLT port, and any combinations thereof.

In addition or as an alternative, computer system1400may provide functionality as a result of logic hardwired or otherwise embodied in a circuit, which may operate in place of or together with software to execute one or more processes or one or more steps of one or more processes described or illustrated herein. Reference to software in this disclosure may encompass logic, and reference to logic may encompass software. Moreover, reference to a computer-readable medium may encompass a circuit (such as an IC) storing software for execution, a circuit embodying logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware, software, or both.

In accordance with the description herein, suitable computing devices include, by way of non-limiting examples, server computers, desktop computers, laptop computers, notebook computers, sub-notebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, Internet appliances, mobile smartphones, tablet computers, personal digital assistants, video game consoles, and vehicles. Those of skill in the art will also recognize that select televisions, video players, and digital music players with optional computer network connectivity are suitable for use in the system described herein. Suitable tablet computers, in various embodiments, include those with booklet, slate, and convertible configurations, known to those of skill in the art.

Non-Transitory Computer Readable Storage Medium

Computer Program

Digital Processing Device

In some embodiments, the digital processing device includes a display to send visual information to a user. In some embodiments, the display is a liquid crystal display (LCD). In further embodiments, the display is a thin film transistor liquid crystal display (TFT-LCD). In some embodiments, the display is an organic light emitting diode (OLED) display. In various further embodiments, on OLED display is a passive-matrix OLED (PMOLED) or active-matrix OLED (AMOLED) display. In some embodiments, the display is a plasma display. In other embodiments, the display is a video projector. In yet other embodiments, the display is a head-mounted display in communication with the digital processing device, such as a VR headset. In further embodiments, suitable VR headsets include, by way of non-limiting examples, HTC Vive, Oculus Rift, Samsung Gear VR, Microsoft HoloLens, Razer OSVR, FOVE VR, Zeiss VR One, Avegant Glyph, Freefly VR headset, and the like. In still further embodiments, the display is a combination of devices such as those disclosed herein.

Non-Transitory Computer Readable Storage Medium

Computer Program

Web Application

Web Application

Referring toFIG.15, in a particular embodiment, an application provision system comprises one or more databases1500accessed by a relational database management system (RDBMS)1510. Suitable RDBMSs include Firebird, MySQL, PostgreSQL, SQLite, Oracle Database, Microsoft SQL Server, IBM DB2, IBM Informix, SAP Sybase, SAP Sybase, Teradata, and the like. In this embodiment, the application provision system further comprises one or more application severs1520(such as Java servers, .NET servers, PHP servers, and the like) and one or more web servers1530(such as Apache, IIS, GWS and the like). The web server(s) optionally expose one or more web services via app application programming interfaces (APIs)1540. Via a network, such as the Internet, the system provides browser-based and/or mobile native user interfaces.

Referring toFIG.16, in a particular embodiment, an application provision system alternatively has a distributed, cloud-based architecture1600and comprises elastically load balanced, auto-scaling web server resources1610and application server resources1620as well synchronously replicated databases1630.

Mobile Application

In some embodiments, a computer program includes a mobile application provided to a mobile computing device. In some embodiments, the mobile application is provided to a mobile computing device at the time it is manufactured. In other embodiments, the mobile application is provided to a mobile computing device via the computer network described herein.

Those of skill in the art will recognize that several commercial forums are available for distribution of mobile applications including, by way of non-limiting examples, Apple® App Store, Google® Play, Chrome WebStore, BlackBerry® App World, App Store for Palm devices, App Catalog for webOS, Windows® Marketplace for Mobile, Ovi Store for Nokia® devices, Samsung® Apps, and Nintendo® DSi Shop.

Standalone Application

Web Browser Plug-In

Software Modules

Databases

In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more databases, or use of the same. In view of the disclosure provided herein, those of skill in the art will recognize that many databases are suitable for storage and retrieval of XXX information. In various embodiments, suitable databases include, by way of non-limiting examples, relational databases, non-relational databases, object oriented databases, object databases, entity-relationship model databases, associative databases, and XML databases. Further non-limiting examples include SQL, PostgreSQL, MySQL, Oracle, DB2, and Sybase. In some embodiments, a database is internet-based. In further embodiments, a database is web-based. In still further embodiments, a database is cloud computing-based. In a particular embodiment, a database is a distributed database. In other embodiments, a database is based on one or more local computer storage devices.

EXAMPLES

The following illustrative example is representative of embodiments of the software applications, systems, and methods described herein and are not meant to be limiting in any way.

In one example, a buyer creates a brief comprising a description (e.g. a text-form explanation of advertising content they wish to purchase), a target interest (e.g. the type of audience), a required content type (e.g. image, short-form video, long-form video, reviews, animated gif), a target demographic (e.g. age, gender, location), and any additional information about the campaign, such as required product.

Based on the information in the brief, the system automatically identifies the best creators for the campaign by applying a ML process to generate a brief creator score that is unique for each creator brief/creator combination. The system then leverages creator/content metrics and scores collected via performance data, internal/external ratings and crowd-sourced creator/content annotation.

The buyer then accepts or denies a content from a creator. This acception or denial provides additional data to the system to recalculate the brief creator score based on such selections to suggest additional creators. The brief creator score is continually recalculated based on the selections, to provide constant improvement in the list of top creators for the brief.

Best Pet Toys creates a brief comprising a description of a video they are requesting to advertise a super-chewy dog toy to male young adults in California.

Based on the information in the brief, the system automatically identifies Bob and Joe as the best creators for the campaign based on a brief creator score that is unique for each creator brief/creator combination. The system then leverages creator/content metrics and scores collected via performance data, internal/external ratings and crowd-sourced creator/content annotation.

The buyer then accepts Joe as a creator. This acception or denial provides additional data to the system to recalculate the brief creator score for Joe and Bob to suggest additional creators in the future. The brief creator score is continually recalculated based on the selections, to provide constant improvement in the list of top creators for the brief.

In one example, a buyer creates a brief comprising a description (e.g. a text-form explanation of advertising content they wish to purchase), a target interest (e.g. the type of audience), a required content type (e.g. image, short-form video, long-form video, reviews, animated gif), a target demographic (e.g. age, gender, location), and any additional information about the campaign, such as required product.

The system then selects one or more creators based on a brief creator score that is unique for each creator brief/creator combination and is based the information in the brief, and weights applied to the internal content rating, the internal creator rating, the external creator rating, the external content rating, the content performance, the crowd-sourced content rating, the reliability score, and creator social metrics. The weights are constantly updated by a machine learning process as clients select, rate, and score each creator and/or their content.

Best Pet Toys creates a brief comprising a description of a video they are requesting to advertise a super-chewy dog toy to male young adults in California.

Based on the information in the brief, the system automatically identifies Bob and Joe as the best creators for the campaign based on a brief creator score that is unique for each creator brief/creator combination. The system then leverages creator/content metrics and scores collected via performance data, internal/external ratings and crowd-sourced creator/content annotation.

Although Bob has a higher brief creator score based the information in the brief, and weights applied to the internal content rating, the internal creator rating, the external creator rating, the external content rating, the content performance, the crowd-sourced content rating, the reliability score, and creator social metrics, the buyer selects Joe as a creator. As such, the weights applied to the internal content rating, the internal creator rating, the external creator rating, the external content rating, the content performance, the crowd-sourced content rating, the reliability score, and creator social metrics are updated using a machine learning algorithm to select Joe more often than Bob for future campaigns similar to the super-chewy dog toy campaign.