Patent Publication Number: US-2021174392-A1

Title: System and method of adapting a programmatic advertising transaction in response to content analysis

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application is a continuation of nonprovisional patent application Ser. No. 16/037,824 filed on Jul. 17, 2018, entitled SYSTEM AND METHOD OF ADAPTING A PROGRAMMATIC ADVERTISING TRANSACTION IN RESPONSE TO CONTENT, which application claims the benefit of provisional patent application Ser. No. 62/533,650 filed on Jul. 17, 2017, which applications are incorporated herein by reference in their entireties. 
    
    
     FIELD OF THE DISCLOSURE 
     The present disclosure generally relates to content management systems and, more particularly but not exclusively, to content management systems configured to analyze content prior to placement of advertising with such content such as within a programmatic advertising environment. 
     BACKGROUND 
     Online publishers typically provide or allocate portions of their web pages for advertising purposes. These areas are sometimes referred to as impression opportunities. Some of the impression opportunities are available for advertising products, services, or businesses not directly provided by or related to the online publishers. In some embodiments, these impression opportunities are available to advertisers for a price. These impression opportunities are sometimes made available directly to advertisers or through one or more advertising networks (sometimes referred to as “ad networks”) operating as middlemen. In some embodiments, advertising exchanges provide a platform for buying and selling advertising impressions by consolidating and managing impression opportunities across a plurality of publishers. Advertisers interested in impression opportunities may comprise an individual, company or other client entity, which may be further represented by one or more advertising agencies. An advertising agency may design and/or implement an advertising campaign on behalf of a client and procure or buy impression opportunities based on the advertising campaign. 
     Many advertisers would prefer that their ads are not placed on a page to be displayed next to or “against” objectionable content such as might be presented on a webpage being served to a consumer via the consumers client device (e.g., computer, mobile device, tablet and the like). Objectionable content may comprise objectively objectionable content such as jihadi beheading videos and the like or subjectively objectionable content such as negative news stories about an advertiser, content deemed to be immoral or scandalous by the advertiser and so on. 
     Unfortunately, present programmatic advertising environment technologies do not adequately match advertisements to content against which an advertisement might be presented such as on a webpage provided to a consumer. 
     SUMMARY 
     Various deficiencies in the prior art are addressed by systems, methods, architectures, mechanisms, apparatus or protocols configured to determine whether to present an advertisement contemporaneously with a presentation of user requested content at a user device by processing a retrieved content asset in accordance with a content identification mechanism to generate thereby a content identifier associated with the requested user content, comparing the generated content identifier to one or more lists of content identifiers associated with content categorized as objectionable or non-objectionable in accordance with criteria associated with the proposed advertisement, determining whether to present the advertising in response to the generated content identifier being associated with known objectionable or non-objectionable content, otherwise initiating a text-based correlation process to determine whether to present the advertising in response to a medium confidence correlation of the content to criteria indicative of objectionable or non-objectionable content while continuing the correlation process to achieve a high correlation of the content to criteria indicative of objectionable or non-objectionable such that the content may be included in the appropriate list(s). 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The teachings herein can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which: 
         FIG. 1  depicts a simplified block diagram suitable for use in understanding the present invention. 
         FIG. 2  depicts an exemplary trust engine  200 . 
         FIG. 4  depicts a flow diagram of a layered brand safety defense method according to one embodiment. 
         FIG. 3  depicts a high-level block diagram of a computing device suitable for use in performing functions described herein. 
     
    
    
     To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. 
     DETAILED DESCRIPTION 
     The following description and drawings merely illustrate the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the invention and are included within its scope. Furthermore, all examples recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the invention and the concepts contributed by the inventor(s) to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions. Additionally, the term, “or,” as used herein, refers to a non-exclusive or, unless otherwise indicated (e.g., “or else” or “or in the alternative”). Also, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. 
     The numerous innovative teachings of the present application will be described with reference to the presently preferred exemplary embodiments. However, it should be understood that this class of embodiments provides only a few examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed inventions. Moreover, some statements may apply to some inventive features but not to others. Those skilled in the art and informed by the teachings herein will realize that the invention is also applicable to various other technical areas or embodiments. 
     Various deficiencies in the prior art are addressed by systems, methods, architectures, mechanisms or apparatus for rapidly analyzing content to gain some indication as to whether or not the content would be objectionable to an advertiser such that the advertiser may determine whether or not to advertise against/adjacent the content (e.g., on the same displayed/presented page viewed by a user). 
     For example, an advertiser may automatically win an advertisement placement or impression with a demographically appropriate viewer of some webpage the context of a programmatic advertising environment. However, prior to authorizing the placement of the advertisement (e.g., forwarding the URL associated with the advertising asset toward the publisher) the advertiser or some other entity within the programmatic advertising environment may forward toward a trust engine platform, according to one of the embodiments, one or more URLs associated with the content against which the advertisement is to be placed. The trust engine analyzes the content and/or components thereof to extract information categorizing the content in a useful manner such that the advertiser or other entity within the programmatic advertising environment may make a final placement decision as to the advertisement. 
     The final placement decision may comprise placing the advertisement as originally intended, in which case the URL of the originally intended advertisement is forwarded to the publisher such that the advertising asset may be inserted into the webpage or other content being provided to the user. 
     The final placement decision may comprise not placing the original advertisement, such as by forwarding to the publisher the URL associated with a blank advertisement asset, an alternate advertisement asset and the like. 
     The final placement decision may comprise generating a secondary bidding process where the winning advertiser stands in the shoes of the original publisher and attempts to sell to another advertiser the advertisement placement that won in the initial bidding process but no longer wishes to use for itself. 
       FIG. 1  depicts a simplified block diagram suitable for use in understanding the present invention. In particular,  FIG. 1  depicts a programmatic advertising environment  100  in communication with a trust engine  200  such as described below with respect to  FIG. 2 . It will be appreciated by those skilled in the art that the programmatic advertising environment  100  depicted herein with respect to  FIG. 1  merely illustrate some of the most basic functional elements within the programmatic advertisement placement environments in existence today. As such, it will be appreciated by those skilled in the art that numerous other functional elements, entities and the like normally associated with various programmatic advertising environments in existence today are contemplated by the inventor as relevant to the various embodiments though not specifically depicted herein for the sake of simplicity. 
     The programmatic advertising environment  100  of  FIG. 1  is depicted as comprising advertisers  110  in communication with one or more demand side platforms (DSP)  120 .  FIG. 1  also depicts publishers  150  in communication with one or more supply side platforms (SSP)  140 . The DSPs  120  interact with SSPs  140  via one or more exchanges  130  to conduct automated auctions as part of a programmatic advertisement bidding process wherein available placement spots or impression opportunities associated with users/viewers of publisher webpages, apps or other content sources are auctioned to advertisers seeking to place advertisements in front of such viewers/viewers depending upon various criteria, such as the demographic of the user/viewer and so on. The process is fully automated and quickly executed, preferably during the time a webpage is loading or app is retrieving content. 
     As an example, upon a user landing at a website the publisher identifies the user via previously placed cookies (i.e., small persistent data stored in the user&#39;s browser), via a username or via some other tracking mechanism. The publisher&#39;s website interacts with the SSP (or perhaps directly) to identify or characterize this user in some way (e.g., age, sex, income or other information accumulated and attachable to this user ID) so that advertisers bidding for advertising placement before such users may bid appropriately. Advertisers are bidding via a demand-side platform interacting with an exchange that cooperates with both the demand-side platform and the supply-side platform. The winning bid is quickly resolved and a URL associated with the advertising asset is normally provided by the DSP or advertiser to the publisher so the publisher can include the advertising asset in the webpage being built or otherwise provided to the user. Various embodiments contemplate the interruption of the present interaction cycle of one or more entities associated with the programmatic advertisement environment. 
     That is, various embodiments contemplate a need to interrupt the programmatic advertisement placement process to avoid sending an advertising asset URL to the publisher until the advertiser or the advertisers agency is satisfied that placement of the advertisement against the content to be presented to the user/viewer is appropriate in terms of brand safety, corporate message, brand equity, goodwill or some other criteria. 
     Therefore, in one embodiment the URL of the proposed content assets is propagated to the trust engine  200  prior to transmitting the URL of advertisement assets to the publisher. The URL of the proposed content assets may be transmitted by the publishers  150 , SSPs  140 , exchanges  130 , DSPs  120 , advertisers  110  or any other entity shown or not shown within the programmatic advertising environment  100  such that assembly/delivery of a webpage or other content to a user does not include advertising deleterious to brand safety or other criteria of the advertiser. 
     After processing content associated with one or more URLs on behalf of a particular advertiser, the trust engine  200  may return information indicating whether or not the content is appropriate to the advertiser, descriptive information about the content, categorization of the content, URLs associated with alternate advertising and/or other information as benefits the advertiser. 
       FIG. 2  depicts an exemplary trust engine  200 . As depicted in  FIG. 2 , trust engine  200  includes one or more processor(s)  210 , a memory  220  and a network interface  230 NI. Other functional elements are omitted for simplification of the discussion. 
     The processor(s)  210  is coupled to each of the memory  220  and the network interface  230 NI. The processor(s)  210  is adapted to cooperate with the memory  220 , the network interface  230 N, and various support circuits (not shown) to perform the trust engine functions as described herein. 
     The network interface  230 NI is adapted to facilitate communications with various network elements, nodes and other entities within the system  100 . 
     The memory  220 , generally speaking, stores programs, data, tools and the like that are adapted for use in providing various trust engine/trust management functions which may be invoked by advertisers or other entities within the programmatic advertising environment  100  described above with respect to  FIG. 1 . 
     The memory  220  includes various management system programming modules and databases adapted to implement the various trust engine functions described herein. 
     For example, the memory  220  is depicted as including a content intake and identification engine  221 , a content identification and categorization database  222 , a content component extractor  223 , and audio content processor  224 , an image content processor  225 , a video content processor  226 , a metadata and context processor  227 , a content evaluation engine  228  and a DSP/SSP interface  229 . Although primarily depicted and described herein with respect to specific functions being performed by invoking the specific functions  221 - 229 , it will be appreciated that the various functions described herein may be combined into larger engines, databases and the like. Various embodiments, all of the specific functions  221 - 229  are combined into an overall content processing function. The following description will primarily focus on such an overall content processing function, with some identification of specific functions/tasks indicated therein. 
     The content intake and identification engine  221  is configured to receive content related URLs from entities within the programmatic advertising environment  100  and identify the associated content. Generally speaking, the content intake and identification engine  221  receives the URL of the content assets to be published prior to assembly/delivery of the webpage (or app imagery) to the user. The content assets may include still or moving imagery, audio, text and so on including metadata and other information such as packet header information, address information, data structure identification information and the like. 
     With the content asset URL, the trust engine  200  retrieves the relevant content and subjects the retrieved content to various content analysis techniques to determine whether advertising placement against the content is appropriate. In various embodiments, content analysis comprises a hierarchy of content analysis techniques constructed to rapidly yield a decision as to place and were not placing an advertisement against that content. Trust engine processing may comprise one of a number of techniques, such as blacklisting and white listing publishers, content sources and the like. Processing may also include using text, metadata and/or other information to identify context associated with still or moving image content, such as examining header information within the image packets, related image synopsis or identification information associated with the image and so on. 
     Content identification may be performed in accordance with third-party tools such the photo DNA tools or processes such as those currently used to identify highly objectionable content such as still and video images depicting child pornography and the like. Briefly, such tools comprise a database listing “fingerprints” of pictures or videos of such highly objectionable content which were previously extracted in accordance with various processes which generate a unique identifier or “fingerprint” of a picture or video. Thus, in some embodiments, the content intake and identification engine  221  utilizes a predefined content identification mechanism or process each still or moving image associated with a received content related URL to extract therefrom such a unique identifier or “fingerprint” which is then compared to identifiers associated with known objectionable content. A database of identifiers associated with such known objectionable content may be included within the content ID/categorization database  222  and or included within a local or remote third-party database (not shown) accessed via the trust engine  200 . 
     Whitelist/Blacklist 
     For example, a quick positive decision regarding placement may be reached if the publisher is part of a white list of known good publishers such that all of the content these publishers will present to users is deemed to be acceptable to the brand. Similarly a quick negative decision may be reach of the publisher as part of a blacklist of known bad publishers such that none of the content these publishers were present to users is deemed to be acceptable to the brand. This whitelist/blacklist processing may be performed as part of the initial transaction such that no further processing is necessary by the trust engine to get to a placement decision. Other brand safety decisions may be included within existing exchanges such that there is no need for trust engine processing of certain programmatic advertising placement transactions. 
     Extracting and Comparing Content Fingerprints 
     Photo DNA and eGLYPH technologies (such as provided by Hany Farid) provide a computationally efficient mechanism to identify whether a particular image or video has previously been flagged as obscene or inappropriate in some way. The photo DNA technologies have been applied to child pornography wherein each new or unknown image is automatically processed to extract a unique identifier as well as manually screened and associated with the ranking such as acceptable, obscene but not involving children, obscene involving children or whatever. A hash table is built using the identifiers of all images determined to be obscene involving children. The hash table is then used to process images received by any domain or publisher. 
     Briefly stated, any new image received by a domain or publisher is automatically processed to extra the unique identifier of the new image, which identifier is then pushed into the hash table to see if it matches that of an image previously identified as obscene involving children (in this example). If the identifier of the new image matches anything in the hash table than the new image is simply dropped from the domain or publisher. As a secondary matter the source of the image may also be forwarded to law enforcement etc. This type of processing is legally acceptable since obscenity involving children is not protected speech and can therefore be discarded. This type of content also violates the terms of service of virtually any domain or publisher or website. 
     The eGLYPH technology operates in a manner similar to that of the photo DNA technology, except the eGLYPH technology is directed towards video imagery. I expect that the specific content identification or fingerprinting techniques are a little bit different but built along the same lines as photo DNA. I also expect that there is manual processing of video imagery contemplated with respect to the eGLYPH technology. As such, the eGLYPH technology may be used to process video imagery in the same way as he photo DNA technology processes still imagery. 
     Within the context of child pornography and the like, photo DNA and eGLYPH technologies operate to prevent such imagery from being circulated within a domain or publisher or website. 
     Content identification/categorization database  222  is used to store content identification information, content categorization information and so on as described herein. Several types of content categories are contemplated as being relevant to some or all advertisers. 
     Universally Bad Categories 
     Various embodiments utilize photo DNA, eGLYPH and/or other technologies to uniquely identify still and moving imagery in accordance with various categories consistent with the categories of concern to the various brands. One category might comprise extremist videos or pictures, extremists messages or postings and so on. Another category might comprise kittens. Another category might comprise right-wing political movements such as Nazis are alt-right trite/white supremacy movements. Another category might simply comprise violence above some threshold level (kind of an MPAA rating of PG-13, X and the like). There are a number of different categories that are contemplated. 
     Brand Specific Categories 
     In addition to general categories as mentioned above, there are also brand specific categories to be identified. These types or categories can be built or identified on the fly in all likelihood. For example, United airlines probably does not wish to advertise next to content describing flight attendants beating the crap out of somebody on the plane. The news story describing such an incident may be readily discerned by analyzing the text associated with that news story. No need to analyze still or moving imagery make this decision. 
     The 200 ms Loop and the Limits of Photo DNA 
     Generally speaking, the time between a user clicking on a link associated with a webpage and the execution of a relevant programmatic advertisement placement is about  200  ms. That is, as the webpage is being loaded in the user&#39;s browser and presented to the user via whatever display device (phone display, computer display etc.) the automated bidding and placement process is executed such that the relevant advertisements are loaded contemporaneously with or shortly after the content is loaded such that the content and advertising assets are displayed together (i.e., adjacent with each other). There may also be further processing in terms of adapting the placed advertisement such as to the formatting of the content such that it blends seamlessly from an aesthetic perspective. 
     Trust engine processing inevitably adds some processing time to the loop, such as time to extract still or moving imagery identification information (e.g., digital “fingerprints”) from the proposed content to be displayed adjacent to the advertising. 
     If the proposed content has already been processed via photo DNA or eGLYPH manual screening and deemed to be unsafe then various embodiments contemplate hashing the extracted imagery fingerprint with the “unsafe content hash table” in immediately identifying the content as unsafe and proceeding accordingly (e.g., place a backup or secondary advertisements, place new advertisement or whatever). 
     If the proposed content has already been processed via photo DNA or eGLYPH manual screening and deemed to be safe, then when the extracted imagery fingerprint is hashed with the “safe content hash table” the trust engine will immediately identify the content as safe and proceed accordingly. To conserve computational/memory resources, various embodiments avoid the use of a single or global “safe content hash table” and instead utilize/generate numerous safe content hash tables based upon information such as content source and the like. In various embodiments, a single safe content hash table is generated/utilized for each publisher or advertiser, which hash table is augmented over time as additional content from the publisher or advertiser is processed. 
     It may be the case that a number of ads are placed against/adjacent unsafe content due to a time lag in determining what content is safe or not. This may be acceptable and in fact may be inevitable given the limits of processing power and speed and so on. The point is to minimize the number of subjectively unsafe brand placements and minimize a number as rapidly as possible. Therefore, various embodiments optionally provide for manual processing of content from at least some sources, such as sources that are associated with non-preferred or blacklisted sources for example. Various embodiments utilize a combination of manual processing of content and automatic processing of content. Various embodiments utilize automatic processing of content alone. 
     Multiple Hash Tables 
     Various embodiments utilize multiple hash tables to augment existing highly objectionable content hash tables such as use with the photo DNA technologies, databases and the like. That is, various embodiments extract image fingerprints from all content being manually or automatically processed, whether such content is deemed to be objectionable or not ultimately. Upon categorization of content the image fingerprint associated with the content is hashed into one or more category specific hash tables or databases such as “objectionable” content, “safe” content, “rated X” content, “rated PG-13” content, “political” content, “violent” content, “religious” content and so on. In this manner, subsequent retrieved content such as still or moving images may be processed against multiple hash tables or databases to more rapidly categorize the subsequently retrieved content and thereby speed up the advertising placement/no placement decision process. 
     In various embodiments, trust engine processing is contemporaneous with, and supplemental to, the existing processing performed on behalf of anti-child pornography and anti-extremist efforts. 
     In various embodiments, trust engine processing resulting in data relevant to anti-child pornography and anti-extremist efforts may be provided to organizations associated with such efforts for the benefit of all advertisers. 
     Various embodiments contemplate categorizing content in a manner useful to individual brands in real time or substantially real time, depending upon the type of content, the amount of content to be processed and so on. As previous noted, such content categorizing builds existing databases such as category related databases which in turn improves subsequent processing of received content. As an example, a user downloading a webpage will be one of thousands or millions of users ultimately downloading the same or similar webpage. Once the content associated with that webpage his process, the relevant databases are updated to reflect this processing. As such, the trust engine will preferably perform first impression processing of content only once. The trust engine may perform subsequent processing of first impression processed content to improve categorization of that content where the likelihood of additional access of the content by demographically relevant users is high such that the effort is justified. 
     In various embodiments, if proposed content has never been processed via photo DNA or eGLYPH manual screening then trust engine  100  may perform first impression processing of the content. This may be performed in a cursory or sampling manner or a detailed manner, depending upon time constraints and the like. 
     Thus, in various embodiments, content component extractor  223  operates to extract and/or sample content indicated via a received URL. For example, content component extractor  223  may extract an audio track from video content, one or more images from video content, one or more portions of an image from still or moving imagery and so on. In addition to content component extraction, metadata and other data may also be extracted by content component extractor  223  in various embodiments. 
     In various embodiments, content processing AI tools are utilized to perform automated content analysis and categorization of the proposed content. 
     Various embodiments contemplate an ability to broadly categorize content as acceptable or not using a sequent of applied techniques. 
     Various embodiments, audio content processor  224  performs analysis of audio information associated with an audio track of a video stream or audio content. For example, such audio information may be processed using speech to text processing followed by context, keyword identification and other processing techniques to identify whether or not the content is brand safe. Extracted audio information may also be subjected to language analysis to enable automated text extraction in the relevant language. For example, objectionable videos may be determined by identifying the native language of the relevant audio track, performing speech to text processing to provide thereby native language text, converting the native language text into English text and then performing keyword analysis, context analysis and so on to get some sense as to whether or not the video contains unwanted content. In one embodiment, identification of the language may be enough. For example, an advertisement to be served against content of a particular language to someone who does not speak that language may, according to some advertiser criteria, be sufficient to determine that placement of the advertisement with this content is not appropriate. 
     In various embodiments the trust engine extracts metadata from the content such as thumbnail imagery or other imagery/data carried in the header or other locations within the content packet stream for subsequent processing. Various embodiments the trust engine samples video imagery to provide a small but representative number of still images to process. 
     In various embodiments the trust engine uses existing image matching tool such as those provided by Google to identify specific images or at least images similar to e proposed image. For example, if a still image or a number of images extracted from a video stream are deemed by the Google matching tool or any other tool as similar to unwanted content then the trust engine may in some instances deem the proposed content to be objectionable content such that no advertising placement is appropriate. 
     Various embodiments the trust engine uses an image comparison tool to identify a number of potentially identical or relevant images, extracts from these images and the available metadata text or contextual information, and analyzes the extracted metadata text or contextual information to determine whether imposed content is objectionable content such that no advertising placement is appropriate. 
     In various embodiments, a cache is used to store content or components thereof during processing. These embodiments content may be fully processed according to multiple techniques to derive content identification fingerprints, content categorization, context information associated with related information and so on. When sufficient information to determine whether or not the content or content component is appropriate for brand safety purposes, a URL may be provided to the programmatic advertising environment  100  indicative of the primary advertisement to be placed, a secondary advertisement he placed, a blank advertisement and the like. It is noted that the cache content may be further processed to extract therefrom additional categorization information and the like useful in performing future content processing operations. Thus the content/content component cache operates to store content or links thereto as part of a comprehensive content processing methodology in which a hierarchical arrangement of content processing techniques are deployed in execution speed rank order such that a relatively fast placement/non-placement decision may be made by the advertiser as sufficient information is determined. 
       FIG. 3  depicts a high-level block diagram of a computing device, such as a processor in a communications network element, suitable for use in performing functions described herein such as those associated with the various elements described herein with respect to the figures. 
     In particular, any of the various functional entities described herein, such as network routing entities, network management entities, server devices, client devices and so on within the communication network may be implemented in accordance with a general computing device structure such as described herein with respect to  FIG. 3 . 
     As depicted in  FIG. 3 , computing device  300  includes a processor element  303  (e.g., a central processing unit (CPU) or other suitable processor(s)), a memory  304  (e.g., random access memory (RAM), read only memory (ROM), and the like), a cooperating module/process  305 , and various input/output devices  306  (e.g., a user input device (such as a keyboard, a keypad, a mouse, and the like), a user output device (such as a display, a speaker, and the like), an input port, an output port, a receiver, a transmitter, and storage devices (e.g., a persistent solid state drive, a hard disk drive, a compact disk drive, and the like)). 
     It will be appreciated that the functions depicted and described herein may be implemented in hardware or in a combination of software and hardware, e.g., using a general purpose computer, one or more application specific integrated circuits (ASIC), or any other hardware equivalents. In one embodiment, the cooperating process  305  can be loaded into memory  304  and executed by processor  303  to implement the functions as discussed herein. Thus, cooperating process  305  (including associated data structures) can be stored on a computer readable storage medium, e.g., RAM memory, magnetic or optical drive or diskette, and the like. 
     It will be appreciated that computing device  300  depicted in  FIG. 3  provides a general architecture and functionality suitable for implementing functional elements described herein or portions of the functional elements described herein. 
     It is contemplated that some of the steps discussed herein may be implemented within hardware, for example, as circuitry that cooperates with the processor to perform various method steps. Portions of the functions/elements described herein may be implemented as a computer program product wherein computer instructions, when processed by a computing device, adapt the operation of the computing device such that the methods or techniques described herein are invoked or otherwise provided. Instructions for invoking the inventive methods may be stored in tangible and non-transitory computer readable medium such as fixed or removable media or memory, or stored within a memory within a computing device operating according to the instructions. 
     Various modifications may be made to the systems, methods, apparatus, mechanisms, techniques and portions thereof described herein with respect to the various figures, such modifications being contemplated as being within the scope of the invention. For example, while a specific order of steps or arrangement of functional elements is presented in the various embodiments described herein, various other orders/arrangements of steps or functional elements may be utilized within the context of the various embodiments. Further, while modifications to embodiments may be discussed individually, various embodiments may use multiple modifications contemporaneously or in sequence, compound modifications and the like. 
     Layered Brand Safety Defense 
     The various embodiments discussed above contemplate a layered defense approach to brand safety within the context of programmatic advertising. Different defensive layers supporting brand safety may be implemented via a local brand safety platform, via remote platforms or by some combination of local and remote platforms. 
       FIG. 4  depicts a flow diagram of a layered brand safety defense method according to one embodiment. Various modifications to this method are contemplated by the inventors including different orders of the elements, a mix of remote and local processing of content or content related information and so on. Various steps and/or portions thereof of the method  400  of  FIG. 4  may be performed by the trust engine  200  described above with respect to  FIG. 2  either alone or in conjunction with other entities interacting with the trust engine  200 . 
     At step  410 , a uniform resource locator (URL) pointing to content is received by the brand safety platform. Typically this URL will be received after the successful conclusion of a programmatic bidding process in which the advertiser utilizing the brand safety platform has won the right to present an advertisement to a user in conjunction with the presentation of the content to the user. In various embodiments, the URL is received before or even during the programmatic bidding process but that the processing steps described herein and elsewhere in this specification may begin as quickly as possible. 
     Optionally, at step  410  a timer is started. Within the context of the method  400  of  FIG. 4  the timer represents an amount of time elapsed after conclusion (or start) of a programmatic bidding process. In some embodiments, if a certain amount of time has elapsed been a default decision or decision based upon the then best estimate is provided. 
     For example, if anything more than a several hundred millisecond, one second, five seconds or some other threshold time has elapsed, then the method  400  may exit and provide to the SSP a URL pointing to a default one of the primary, secondary, tertiary or blank advertisement. If sufficient processing has occurred to indicate that the content is likely not universally objectionable, then a URL pointing to primary advertisement may be provided. If sufficient processing is occurred to indicate that the content may be objectionable with respect to brand image, then it URL pointing to a secondary or tertiary advertisement may be provided. Other decisions may also be made based upon the circumstances and related circumstances as will be appreciated by those skilled in the art. 
     Content URL Processing 
     At step  420 , an evaluation of the URL itself is performed. That is, at step  420  and evaluation of the website pointed to by the URL, any metadata or text associated with the URL and so on is evaluated such as by comparing the URL or related metadata or text to one or more blacklists and or one or more white lists. 
     If the blacklist comparison indicates that the URL is blacklisted then a URL_fail flag may be set and/or a URL_fail routine may be invoked. Either of these mechanisms may be used to trigger a decision to not place primary advertising next to the content to be displayed to the user (or to place secondary or tertiary advertising next to the content to be displayed to the user). 
     If the white list comparison indicates that the URL is whitelisted then a URL_pass flag may be set and/or a URL_pass routine may be invoked. Either of these mechanisms may be used to trigger a decision to place primary advertising next to the content to be displayed to the user. 
     In some embodiments, an affirmative comparison of either blacklisting or white listing of content associated with the URL is dispositive in terms of the decision to place or not place primary advertising next to the content to be displayed to the user. In some embodiments, an affirmative comparison of one or both of blacklisting or white listing of content associated with the URL is not by itself dispositive; rather, this information is used in conjunction with subsequently determined information to make a decision about whether or not to place primary advertising next to the content to be displayed to the user. In this case the setting or resetting the various flags provide a mechanism by which blacklist/white list affirmative comparison results may be propagated to subsequent processing elements/routines within the brand safety platform. 
     Optionally at step  420 , if the result of the blacklisting and white listing comparisons is inconclusive, the URL_fail and/or URL_pass flags may be reset and the method  400  may proceed. 
     Content Identification Processing 
     At step  430 , an evaluation of the photo DNA, eGLYPH or other content identifier associated with the content indicated by the URL is performed. Such evaluation is optionally performed while the optional timer remains less than a threshold amount of time. 
     If remote evaluation of the content identifier is used, then the URL is forwarded to a remote processing entity such as provided by Counter Extremism Project (CEP) for further processing such as described at the CEP website at www.counterextremism.com. Other third-party entities may also be used to perform this function, which function will ultimately result in receiving an evaluation of the content identified by the URL. 
     If local evaluation of the content identifier is used, then the content or a sample of the content is retrieved via the URL. Content identifiers are then extracted from the retrieved content or portion thereof such as in accordance with the photo DNA, eGLYPH or other processes to produce a unique content identifier or “fingerprint” associated with the content proposed to be presented to a user in conjunction with the primary advertisement. Generally speaking, extracting content identification information rapidly provides a unique mechanism by which the content may be identified. 
     In various embodiments, the extracted content identification information is hashed into one or more blacklists hash tables and or one or more white list hash tables to determine whether or not the content matches content previously determined to be universally objectionable, brand safety objectionable or appropriate. The various blacklist and/or white list hash tables may be stored locally or remotely. The various blacklist and/or white list hash tables may be shared with other entities as content related information is determined by the brand safety platform. 
     If the blacklist comparison indicates that the extracted content identification information matches that of previously blacklisted content then a CONTENT_fail flag may be set and/or a CONTENT_fail routine may be invoked. Either of these mechanisms may be used to trigger a decision to not place primary advertising next to the content to be displayed to the user (or to place secondary or tertiary advertising next to the content to be displayed to the user). 
     If the white list comparison indicates that the extracted content identification information matches that of a previously whitelisted content then a CONTENT_pass flag may be set and/or a CONTENT_pass routine may be invoked. Either of these mechanisms may be used to trigger a decision to place primary advertising next to the content to be displayed to the user. 
     In some embodiments, an affirmative comparison of either blacklisting or white listing of content associated with the content identification information is dispositive in terms of the decision to place or not place primary advertising next to the content to be displayed to the user. In some embodiments, an affirmative comparison of one or both of blacklisting or white listing of content matching the content of notification information is not by itself dispositive; rather, this information is used in conjunction with subsequently determined information to make a decision about whether or not to place primary advertising next to the content to be displayed to the user. In this case the setting or resetting the various flags provide a mechanism by which blacklist/white list affirmative comparison results may be propagated to subsequent processing elements/routines within the brand safety platform. 
     Optionally at step  430 , if the result of the blacklisting and white listing comparisons is inconclusive, the CONTENT_fail and/or CONTENT_pass flags may be reset and the method  400  may proceed. 
     Content Text Processing 
     At step  440 , an evaluation of the text associated with the content to be displayed to the user is evaluated, such as metadata, content text, descriptive text, surrounding article text and other contextual information associated with the content. Such evaluation is optionally performed while the optional timer remains less than a threshold amount of time. 
     Evaluation of text associated with the content may definitively produce a result sufficient to determine that the content is universally objectionable, incompatible with the brand or appropriate for the brand. This evaluation may be performed using keyword searching, associative strength analysis, link strength analysis and/or any number of correlation techniques. Ultimately content text processing is intended to identify with some level of confidence whether or not the content itself (i.e., the still or moving imagery) is appropriate for display in conjunction with the advertising. Differencing engines, neural networks, brute force computing techniques and the like are all suitable for use in performing this task though processing power, memory and time constraints of selected processing mechanisms may lead to inexact though useful/actionable results. 
     If the text evaluation mechanisms are sufficient to determine that the content is objectionable, then a TEXT_fail flag may be set and/or a TEXT_fail routine may be invoked. Either of these mechanisms may be used to trigger a decision to not place primary advertising next to the content to be displayed to the user (or to place secondary or tertiary advertising next to the content to be displayed to the user). 
     If the text evaluation mechanisms are sufficient to determine that the content is acceptable, then a TEXT_pass flag may be set and/or a TEXT_pass routine may be invoked. Either of these mechanisms may be used to trigger a decision to place primary advertising next to the content to be displayed to the user. 
     If the text evaluation mechanisms are sufficient to determine that the content is compatible with brand safety, then a COMPATIBLE_pass flag may be set and/or a COMPATIBLE_pass routine may be invoked. Either of these mechanisms may be used to trigger a decision to place secondary or tertiary advertising next to the content to be displayed to the user. 
     If the text evaluation mechanisms are unable to determine that the content is universally objectionable, compatible or incompatible with brand safety, then additional processing may be invoked to generate more text-based information to be processed. Specifically, various embodiments contemplate the use of one or more mechanisms to derived additional text-based information pertaining to the content to be displayed to the user, wherein such mechanisms may be invoked locally or remotely such as via a third party provider services. 
     In various embodiments, additional text-based information associated with the content is derived by processing the audio information associated with the content. That is, in various embodiments audio to text processing may be performed to extracted text from audio information of a primary or separate audio program (SAP) associated with the content. This additional text-based information associated with the content may then be processed using one or more text evaluation mechanisms such as described above. 
     In various embodiments, additional text-based information associated with the content is derived by on screen text scraping of the still or moving imagery of the content. That is, in various embodiments text identification techniques may be used to identify those portions of still or moving imagery associated with text such as screen scrolls, closed captions and so on such that these portions may be extracted or “scraped” from the imagery and subjected to optical character recognition (OCR) techniques to derive therefrom additional text-based information associated with the content. This additional text-based information associated with the content may then be processed using one or more text evaluation mechanisms such as described above. 
     Content Processing 
     At step  450 , an evaluation is performed of the still or moving imagery, or samples, subsamples, image regions and the like thereof associated with the content. Such evaluation is optionally performed while the optional timer remains less than a threshold amount of time. 
     In various embodiments, a local or remotely accessed reverse image look up service is used to process images information associated with the content to be displayed to the user. In various embodiments the reverse image look up service provides descriptive information associated with the content sufficient to enable a determination as to whether or not the content is universally objectionable, brand incompatible or brand acceptable. The descriptive information may include a confidence indicator suitable for assigning a weight to the descriptive information in determining whether to display primary, secondary or no advertising alongside the content to be displayed to the user. 
     In various embodiments, the reverse image look up service provides potential identical or similar images (or URL pointers thereto) to that of the content to be displayed to the user. 
     In various embodiments, the potential identical or similar images may be further processed according to any of steps  420 - 450  to generate additional information suitable for use in determining whether or not to place advertisements with the content to be presented to the user. For example, if video content to be displayed to the user is identical or similar to that of video content provided by blacklisted sites then an inference may be made that the content to be this way to the user is also objectionable. Similarly, metadata, text or contextual information associated with the “identical or similar” video content to be processed as described with respect to step  440  to render a decision. 
     In various embodiments, reverse image look up services will provide multiple medium confidence results rather than a single high confidence results. In these embodiments, the medium confidence results may be evaluated to render an advertisement placement decision with respect to the content to be presented to the user. In various embodiments, multiple medium confidence images/results may be further processed according to any of steps  420 - 450  to generate additional information suitable for use in determining whether or not to place advertisements with the content to be presented to the user. 
     In various embodiments, the medium confidence results are ranked in order of confidence level and/or other criteria and processed as described above. In various embodiments, a correlation process is initiated to provide text-based correlations between text extracted from user requested content assets and text associated with proposed advertisements (e.g., brand-related, advertiser related and the like). Place or don&#39;t place decisions may be made in response to medium confidence correlations (e.g., between 40% and 60% for example) while continued correlation processing may ultimately yield a high confidence correlation (e.g., above 60% or 70% for example). 
     The various processing steps described herein with respect to  FIG. 4  such as URL evaluation, text evaluation and content evaluation may be performed in conjunction with a programmatic advertisement transacting platform or portion thereof. The various processing steps utilized hash tables and databases and the like which over time build a valuable archive of information suitable for streamlining the processing of subsequently received content. Various embodiments contemplate sharing such databases for public service purposes such as eradicating universally objectionable content, and commercial purposes such as generating databases, hash tables, white list, blacklist and the like specifically constructed to reflect brand safety concerns of one or more brands or brand types. Further, access to these databases, hash tables, white list, blacklist and the like may also be included within the programmatic advertising transaction process as an add-on brand safety feature. 
     The various embodiments described herein contemplate the processing of one or more types of information associated with a content asset to determine whether that content asset is associated with objectively or subjectively objectionable content such that an advertisement should not the placed for adjacent presentation with the content. The amount of processing necessary to reach such a determination depends in part upon the type of information extracted from the content assets for processing, whether that information type requires intermediate processing steps and the like. As noted with respect to, illustratively,  FIG. 4 , text information, audio information, still/moving image information and the like may be extracted from a content asset and examined to identify data indicative of objectionable content, data indicative of not objectionable content and neutral data. Such data may comprise the source of the content, metadata associate of the content, language extracted from audio tracks, images extracted from video tracks and the like. Each of these different types of information requires different processing techniques and some of these processing techniques take longer than others to reach a result that is sufficiently accurate or of high enough confidence. 
     As noted above, medium confidence indication of factors driving a decision to place or not place an advertisement are perfectly adequate within the context of a timing loop which is ultimately linked to quality of user experience. In essence, various embodiments are configured to extract and process information from a user request the content asset to achieve a sufficient accuracy or confidence level or correlation to relevant brand criteria to make a decision about advertising placement. The decision may be wrong for a first user requesting that content, or even a first group of users requesting the content. However, the content is continually processed so that subsequent user request for such content may be more rapidly evaluated in view of proposed advertisements to be placed adjacent the content viewed by those subsequent users. 
     As an example, if the content asset has extracted from it text information having a medium confidence correlation (e.g., 30%, 40%, 50%, 60% or whatever correlation percentage, whether low, medium or high, is deemed appropriate by an advertiser or brand) to text-based criteria associated with objectionable content for an advertiser then a medium confidence decision may be made that the advertisement should not be placed adjacent that content. Further, the extracted text information may also be weighted such as increasing the correlative weight of content sources known to provide objectionable content (e.g., pornographic websites, jihadi or militant websites, foreign propaganda websites and the like), descriptive terms known to relate to objectionable content (e.g., profanity, sexual language, militant language, selected foreign languages and the like) and so on. In addition to text-based correlation, audio information, still image information, moving image information and the like, or portions thereof, may be correlated to like media to rapidly identify suspected objectionable content. Various artificial intelligence techniques/mechanisms may be employed to perform the various steps and processes. 
     Advantageously, various embodiments operate to quickly provide a brand safety decisions pertaining to placing an advertisement next to content to be presented to a user. The decision may comprise placing an initial advertisement next to the content, placing a secondary or tertiary advertisement next to the content or placing now advertisement next to the content. Generally speaking, some content is deemed to be universally objectionable such that no advertisement should be associated with it. In this case, an advertisement URL associated with a blank advertisement may be transmitted to the SSP. Some content is deemed to be objectionable to the values of a particular brand or its corporate owners such that at least certain advertisement associated with the brand or its corporate owners should not be displayed next to such content. Decision may be made quickly where the content is deemed to be objectionable 
     In various embodiments, a stand-alone placement decision tool is provided wherein, in response to an input comprising a content URL (i.e., a URL pointing to content to be presented to a user) an advertising URL (i.e., a URL pointing to an ad to be presented/placed adjacent to the content, such as on the same displayed web page) and criteria indicative information (e.g., advertiser, brand, campaign identifier, hash table identifier and the like), various processes such as describe above are performed to produce as an output a place/no-place decision indication. 
     In various embodiments, a stand-alone content processing tool (optionally used with a stand-alone or integrated placement decision tool) is provided for processing content to extract characterizing data therefrom and assign a unique identification number to the content based on Photo DNA or other content processing methodologies. 
     In various embodiments, an integrated placement decision tool is provided such that the place/no-place decision may be integrated into or hooked into (e.g., such as via an applications programming interface (API), framework or other mechanism) a platform to enable TE control/automation of the place/no-place decision. 
     In various embodiments, a Placement Decision Console (PDC) is provided to enable real time viewing/tracking of placement decisions and historical reconstruction of non-decision impact such as identifying placements avoided, potential or actual bad placements and an indication of user response or other results, other monitoring activities and the like. The PDC enables advertiser or their agencies to determine what ads have been “not placed” or “placed” and where/when did this happened, the ratio/cost of place / no-place decisions and so on. It is contemplated that the PDC utilizes marketing services descriptions to include showing of place/no-place decisions according to one or more of the following criteria: Timestamp; User/Viewer; Ad; Adjacent content; Mode (web, mobile web, access network, MSO); Prior page/channel viewed by user; Next page/channel selected by user; Other information; Current/live activity; Prior/Historical Activity (i.e., pre-decision data for bad adjacencies that occurred so the trust engine can gauge reaction) and/or other criteria/parameters of interest. 
     In various embodiments, an Integrated Content Processing (ICP) tool is provided for interacting with publishers, content management platforms and the like such that content may be received/processed prior to, or contemporaneously with, the distribution/publishing of content (i.e., not related to a particular placement decision). 
     In various embodiments, a Stand-Alone or Integrated ad processing tool for processing advertisements and ancillary advertiser provided information (e.g., target audiences, relevant community standards, corporate mission, PR goals and the like) to extract characterizing data therefrom and assigning a unique identification number to the content based on Photo DNA or other processing of the content. 
     In various embodiments, advertisements are processed or otherwise caused (e.g., via agreements with advertisers or ad server entities) to include consumer feedback hooks for insertion into ads served to elicit thereby a consumer/user response to a question/query, such as: 
     “Does this ad belong with this content” (y/n) 
     “This ad should not follow me across the Internet because I . . . ” &lt;“am out of the market”&gt;; &lt;“already bought product”&gt;; &lt;“bought other product”&gt;; &lt;“Other Reason”&gt; 
     Various embodiments contemplate multiple phases of AI training, using multiple training and live data sets, to sufficiently accurately characterize advertising, brand equity, target market, market sensibilities/drivers and/or various criteria/parameters such as Company/brand level criteria, Product level criteria and the like. Further training of the advertising related AI is adapted to extracting data within the advertising creative/branding process suitable for use in automating the ad characterizing data and therefore improving the precision of ad creation, targeting and place/no-place decisions. Further training of the AI provides a mechanism to efficiently and accurately characterize content in manner consistent with ad characterizing (e.g., to allow fast hashing and placement decisions). Other embodiments are also contemplated by the inventors. 
     The various embodiments discussed above with respect to the figures contemplate a number of different mechanisms by which to achieve the goals of placing or not placing advertising for display next to content deemed subjectively (e.g., from a brand perspective, corporate perspective and the like) of objectively objectionable. 
     One embodiment comprises a method, apparatus, computer readable medium, computer program product and/or other mechanism for determining whether to present an advertisement contemporaneously with a presentation of user requested content at a user device, comprising: receiving, at an advertisement placement apparatus, a Uniform Resource Locator (URL) associated with a user requested content asset; retrieving the user requested content asset via the respective URL; processing the retrieved content asset in accordance with a content identification mechanism to generate thereby a content identifier associated with the requested user content; comparing the generated content identifier to one or more lists of content identifiers associated with content categorized in accordance with criteria associated with the proposed advertisement; in response to the generated content identifier being included in a list of content identifiers associated with non-objectionable content, transmitting a message indicating that the proposed advertisement may be presented with the user requested content; in response the generated content identifier being included in a list of content identifiers associated with objectionable content, transmitting a message indicating that the proposed advertisement should not be presented with the user requested content; in response to the generated content identifier not being included in the non-objectionable content list or the objectionable content list, initiating a text-based correlation process to correlate text-based information extracted from the retrieved content asset with text-based content criteria associated with the proposed advertisement; in response to a medium confidence correlation indicative of the user requested content comprising objectionable content, adding the generated content identifier to the objectionable content list and transmitting a message indicating that the proposed advertisement should not be presented with the user requested content; in response to a medium confidence correlation indicative of the user requested content comprising non-objectionable content, adding the generated content identifier to the non-objectionable content list and transmitting a message indicating that the proposed advertisement may be presented with the user requested content. 
     The above embodiments may be modified by, in response to the text-based correlation process failing to achieve a medium confidence correlation within a predefined period of time, transmitting a message indicating that the proposed advertisement should not be presented with the user requested content. 
     The above embodiments may be modified by, in response to the text-based correlation process failing to achieve a medium confidence correlation within a predefined period of time, transmitting a message indicating that the proposed advertisement may be presented with the user requested content. 
     The above embodiments may be modified by, in response to a high confidence correlation indicative of the user requested content comprising objectionable content, adding the generated content identifier to the objectionable content list and terminating the text-based correlation process. 
     The above embodiments may be modified by, in response to a high confidence correlation indicative of the user requested content comprising non-objectionable content, adding the generated content identifier to the non-objectionable content list and terminating the text-based correlation process. 
     The above embodiments may be modified by, in response to the text-based correlation failing to achieve a high confidence correlation within a second predefined period of time, initiating a non-text correlating process to correlate non-text-based information extracted from the retrieved content asset with non-text-based content criteria associated with the proposed advertisement. 
     The above embodiments may be modified in that the predefined identification mechanism comprises a sampling mechanism configured so extract from the content a plurality of predefined data elements and arrange the extracted data elements in a predefined manner to provide thereby the content identifier. 
     The above embodiments may be modified by, comparing the generated content identifier to one or more lists of categorized content identifiers comprises hashing the content identifier to respective one or more hash tables formed using categorized content identifiers. 
     The above embodiments may be modified in that the hash tables are generated in accordance with brand specific placement criteria. 
     Although various embodiments which incorporate the teachings of the present invention have been shown and described in detail herein, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings. Thus, while the foregoing is directed to various embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof.