Patent Description:
Due to the widespread popularity of digital content for the distribution of entertainment and news, ensuring data security of a processing pipeline for the digital content is vitally important to its creators, owners and distributors alike. However, as machine learning models continue to improve, detection of malware capable of producing deepfakes and introduced into one or more processing nodes of a content processing pipeline will continue to be challenging. As a result, in the absence of a robust and reliable solution for assessing content processing pipeline security, subtly manipulated or even substantially fake digital content may inadvertently be broadcast or otherwise distributed in violation of contractual agreement or regulatory restrictions, thereby subjecting the content owners and/or distributors to potential legal jeopardy.

"<NPL>, discusses a method for detecting deepfake images based on artifacts of the output.

Patent application <CIT> discloses processing a test sample image with a trained transformation function to obtain a transformed matrix. A measure of similarity of the test image based on the transformed matrix is compared to a threshold to determine whether the test sample is novel to a batch of material samples that are provided to train the transformation function.

There are provided systems and methods for securing a content processing pipeline, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.

The following description contains specific information pertaining to implementations in the present disclosure. One skilled in the art will recognize that the present disclosure may be implemented in a manner different from that specifically discussed herein. The drawings in the present application and their accompanying detailed description are directed to merely exemplary implementations. Unless noted otherwise, like or corresponding elements among the figures may be indicated by like or corresponding reference numerals. Moreover, the drawings and illustrations in the present application are generally not to scale, and are not intended to correspond to actual relative dimensions.

The present application discloses systems and methods for securing a content processing pipeline that overcome the drawbacks and deficiencies in the conventional art. By inserting a synthesized test image configured to activate one or more neurons of a malicious neural network into a content stream provided as an input stream to a content processing pipeline, the present security solution advantageously enables detection of the malicious neural network when it is present in the pipeline. In addition, by comparing the synthesized test image included in an output stream received from one or more content processing nodes of the pipeline to an expected image corresponding to the synthesized test image, the present solution enables identification of the node or nodes of the pipeline infected by the malicious neural network.

It is noted that, in some implementations, the present security solution may be performed as a substantially automated process by a substantially automated system. It is further noted that, as used in the present application, the terms "automation," "automated", and "automating" refer to systems and processes that do not require the participation of a human user, such as a system administrator. Although, in some implementations, a human system operator or administrator may review the security assessments produced by the automated systems and according to the automated methods described herein, that human involvement is optional. Thus, the methods described in the present application may be performed under the control of hardware processing components of the disclosed automated systems.

It is also noted that, as defined in the present application, a neural network (NN), also known as an artificial neural network (ANN), is a type of machine learning framework in which patterns or learned representations of observed data are processed using highly connected computational layers that map the relationship between inputs and outputs. A "deep neural network," in the context of deep learning, may refer to a neural network that utilizes multiple hidden layers between input and output layers, which may allow for learning based on features not explicitly defined in raw data. As such, various forms of NNs may be used to make predictions about new data based on past examples or "training data. " In various implementations, NNs may be utilized to perform image processing or natural-language processing.

<FIG> shows a diagram of an exemplary system for securing a content processing pipeline, according to one implementation. As shown in <FIG>, system <NUM> includes computing platform <NUM> having hardware processor <NUM>, system memory <NUM> implemented as a non-transitory storage device, and optional display <NUM>. According to the present exemplary implementation, system memory <NUM> stores software code <NUM> including optional NN <NUM>, as well as content library <NUM> providing content stream <NUM>, and image database <NUM> including one or more synthesized test images <NUM> (hereinafter "test image(s) <NUM>").

It is noted that, as shown by <FIG> and described below, in some implementations in which software code <NUM> includes optional NN <NUM> (NN <NUM> in <FIG>), NN <NUM> may be implemented as a generative adversarial network (GAN) including a discriminator configured to model a malicious neural network (hereinafter "malicious NN"), such as a deepfake NN. In those implementations, NN <NUM> may be used to produce synthesized test image <NUM>, which may be subsequently stored on and obtained from image database <NUM>.

As further shown in <FIG>, system <NUM> is implemented within a use environment including training platform <NUM>, content processing pipeline <NUM> including first processing node 152a, intermediate processing node 152b, and last processing node 152c, as well as communication network <NUM>, and system administrator or other user <NUM> (hereinafter "user <NUM>") utilizing user system <NUM> including display <NUM>. In addition, <FIG> shows network communication links <NUM> communicatively coupling training platform <NUM>, content processing pipeline <NUM>, and user system <NUM> with system <NUM> via communication network <NUM>.

Furthermore, <FIG> shows input stream <NUM> to first processing node 152a of content processing pipeline <NUM>, as well as processed content streams received by system <NUM> as output streams <NUM>, <NUM>, and <NUM> from respective first processing node 152a, intermediate processing node 152b, and last processing node 152c of content processing pipeline <NUM>. Also shown in <FIG> is security assessment <NUM> of at least some portion of content processing pipeline <NUM>, generated by system <NUM>.

Although <FIG> depicts content processing pipeline <NUM> as including three processing nodes represented by first processing node 152a, intermediate processing node 152b, and last processing node 152c, that representation is merely exemplary. In other implementations, content processing pipeline <NUM> may include as few as one processing node, or more, or many more than three processing nodes. Thus, in some implementations, any one of first processing node 152a, intermediate processing node 152b, and last processing node 152c may be the first as well as the last processing node of content processing pipeline <NUM>, while in other implementations, intermediate processing node 152b may include a plurality of processing nodes linking first processing node 152a and last processing node 152c.

It is noted that first processing node 152a, intermediate processing node 152b, and last processing node 152c, can be implemented as any one of a variety of image processing devices. That is to say, one or more of first processing node 152a, intermediate processing node 152b, and last processing node 152c may take the form of a digital camera, a video editing workstation, a desktop, laptop, or tablet computer, a smartphone, or a cloud-based data storage service device, to name a few examples.

Software code <NUM>, when executed by hardware processor <NUM> of computing platform <NUM>, is configured to produce security assessment <NUM> of at least a portion of content processing pipeline <NUM> using synthesized test image <NUM>, as described in greater detail below. Although the present application refers to software code <NUM>, image database <NUM>, and content library <NUM> as being stored in system memory <NUM> for conceptual clarity, more generally, system memory <NUM> may take the form of any computer-readable non-transitory storage medium.

The expression "computer-readable non-transitory storage medium," as used in the present application, refers to any medium, excluding a carrier wave or other transitory signal that provides instructions to hardware processor <NUM> of computing platform <NUM>. Thus, a computer-readable non-transitory storage medium may correspond to various types of media, such as volatile media and non-volatile media, for example. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM), while non-volatile memory may include optical, magnetic, or electrostatic storage devices. Common forms of computer-readable non-transitory media include, for example, optical discs, RAM, programmable read-only memory (PROM), erasable PROM (EPROM), and FLASH memory.

Moreover, although <FIG> depicts training platform <NUM> and user system <NUM> as computer platforms remote from system <NUM>, that representation is also merely exemplary. More generally, system <NUM> may include one or more computing platforms, such as computer servers for example, which may form an interactively linked but distributed system, such as a cloud based system, for instance. As a result, hardware processor <NUM> and system memory <NUM> may correspond to distributed processor and memory resources within system <NUM>, while training platform <NUM> may be a component of system <NUM> or may be implemented as a software module stored in system memory <NUM>. Furthermore, in some implementations, user system <NUM> may be included as an element of system <NUM>.

In one implementation, computing platform <NUM> of system <NUM> may correspond to one or more web servers, accessible over a packet-switched network such as the Internet, for example. Alternatively, computing platform <NUM> may correspond to one or more computer servers supporting a wide area network (WAN), a local area network (LAN), or included in another type of limited distribution or private network.

It is noted that although user system <NUM> is shown as a desktop computer in <FIG>, that representation is also provided merely as an example. More generally, user system <NUM> may be any suitable system that implements data processing capabilities sufficient to provide a user interface, support connections to communication network <NUM>, and implement the functionality ascribed to user system <NUM> herein. For example, in other implementations, user system <NUM> may take the form of a laptop computer, tablet computer, or smartphone, for example.

It is also noted that, in various implementations, display <NUM> may be physically integrated with computing platform <NUM> or may be communicatively coupled to but physically separate from computing platform <NUM>. Moreover, display <NUM> of user system <NUM> may be physically integrated with user system <NUM> or may be communicatively coupled to but physically separate from user system <NUM>. For example, where user system <NUM> is implemented as a smartphone, laptop computer, or tablet computer, display <NUM> will typically be integrated with user system <NUM>. By contrast, where user system <NUM> is implemented as a desktop computer, display <NUM> may take the form of a monitor separate from user system <NUM> in the form of a computer tower. Display <NUM> of system <NUM> and display <NUM> of user system <NUM> may be implemented as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, or any other suitable display screens that perform a physical transformation of signals to light.

<FIG> shows exemplary synthesized test images <NUM> suitable for use in assessing content processing pipeline security, according to one implementation. As shown in <FIG>, synthesized test images <NUM> are "nonsense" images unrecognizable to a human observer. However, synthesized test images <NUM> are configured to be mistaken for recognizable objects by a malicious NN fully trained using popular and widely available training data sets and architectures or fine-tuned on top of a neural net initially trained with one of those datasets and architectures, thereby causing the malicious NN to "hallucinate" objects that are not actually depicted in synthesized test images <NUM>. It is noted that, as defined for the purposes of the present application, a malicious NN is characterized as "hallucinating" when the score of one or more neurons in the NN is substantially maximized in response to synthesized images <NUM>.

For example, synthesized test image 224a is configured to cause a malicious NN to identify the pattern depicted in that image as a tree, despite being unrecognizable as any specific object to a human observer. Analogously, synthesized test image 224b is configured to cause a malicious NN to hallucinate a human person, synthesized test image 224c is configured to cause a malicious NN to hallucinate a bicycle, and synthesized test image 224d is configured to cause a malicious NN to hallucinate a car, by way of example.

Synthesized test images <NUM> correspond in general to synthesized test image(s) <NUM> in <FIG>. As a result, synthesized test image(s) <NUM> may share any of the characteristics attributed to synthesized test images <NUM> by the present disclosure, and vice versa. Thus, like synthesized test images <NUM>, synthesized test image(s) <NUM> is/are unrecognizable to a human observer of synthesized test image(s) <NUM>, but is/are nevertheless configured to be identified as one or more recognizable objects by a malicious NN. The state-of-the-art in deep learning shows that it is possible to produce, i.e., synthesize, images that are completely unrecognizable to humans, but that state-of-the-art NNs believe to be recognizable objects with almost complete confidence.

As a result, synthesized test image(s) <NUM>/<NUM> may be used to detect and thwart so called "man in the middle" attacks. Man in the middle attacks can happen as a result of interception of data communications within content processing pipeline <NUM>. Alternatively, a man in the middle attack may occur as a result malware installed during a firmware update to one or more of first processing node 152a, intermediate processing node 152b, and last processing node 152c, for example due to a camera firmware update or a security breach in the cloud.

The types of synthesized test images <NUM>/<NUM> that will cause a malicious NN configured to produce a deepfake to instead hallucinate will depend on the architecture of the malicious NN, as well as the dataset used for its training. The present content processing pipeline security solution utilizes one or more synthesized test images configured to fool the most popular NN architectures trained based on commonly used training data sets, such as the ResNet, you only look once (YOLO), and Visual Geometry Group (VGG) architectures, trained with ImageNet or 'Labeled Faces in the Wild,' for example. As a result, synthesized test image(s) <NUM>/<NUM> is/are configured to activate one or more neurons of the malicious NN.

For example, if a malicious NN configured as a deepfake NN interprets synthesized test image(s) <NUM>/<NUM> to include a face, such as synthesized test image 224b, for instance, it will typically transform the synthesized test image in an unexpected way. By comparing a synthesized test image supplied as an input to content processing pipeline <NUM> after processing by at least a portion of the pipeline to an expected image corresponding to the synthesized test image, the present security solution enables detection of a man in the middle attack that uses a malicious NN.

<FIG> shows exemplary software code <NUM> suitable for use by system <NUM> in <FIG>, according to one implementation. As shown in <FIG>, software code <NUM> includes training module <NUM>, NN <NUM>, image insertion module <NUM>, and security assessment module <NUM>. In addition, <FIG> shows training data <NUM>, content stream <NUM>, one or more synthesized test images <NUM> (hereinafter "test image(s) <NUM>"), input stream <NUM> to content processing pipeline <NUM>, in <FIG>, and output streams <NUM>, <NUM>, and <NUM> received from content processing pipeline <NUM>. Also shown in <FIG> are synthesized test image(s) <NUM> after processing by at least a portion of content processing pipeline <NUM> (hereinafter "post-processed test image(s) <NUM>"), one or more expected images <NUM> corresponding respectively to synthesized test image(s) <NUM> (hereinafter "expected image(s) <NUM>"), and security assessment <NUM>.

Software code <NUM> including NN <NUM>, content stream <NUM>, input stream <NUM> to content processing pipeline <NUM>, output streams <NUM>, <NUM>, and <NUM> received from content processing pipeline <NUM>, and security assessment <NUM> correspond respectively in general to software code <NUM> including NN <NUM>, content stream <NUM>, input stream <NUM>, output streams <NUM>, <NUM>, and <NUM>, and security assessment <NUM>, in <FIG>. That is to say, software code <NUM>, NN <NUM>, content stream <NUM>, input stream <NUM>, output streams <NUM>, <NUM>, and <NUM>, and security assessment <NUM> may share any of the characteristics attributed to respective software code <NUM>, NN <NUM>, content stream <NUM>, input stream <NUM>, output streams <NUM>, <NUM>, and <NUM>, and security assessment <NUM> by the present disclosure, and vice versa. Thus, although not shown explicitly in <FIG>, software code <NUM> may include features corresponding to each of training module <NUM>, image insertion module <NUM>, and security assessment module <NUM>, as well as optional NN <NUM>/<NUM>.

In addition, synthesized test image(s) <NUM>, in <FIG>, correspond in general to synthesized test image(s) <NUM>/<NUM> in <FIG> and <FIG>. In other words, synthesized test image(s) <NUM> may share any of the characteristics attributed to synthesized test image(s) <NUM>/<NUM> by the present disclosure, and vice versa. Thus, like synthesized test image(s) <NUM>/<NUM>, synthesized test image(s) <NUM> may be unrecognizable to a human observer, but may nevertheless by configured to cause a malicious NN to hallucinate by identifying a synthesized test image as a recognizable object.

The functionality of system <NUM> including software code <NUM>/<NUM> will be further described by reference to <FIG> in combination with <FIG>, <FIG>, and <FIG>. <FIG> shows flowchart <NUM> presenting an exemplary method for securing a content processing pipeline, according to one implementation. With respect to the method outlined in <FIG>, it is noted that certain details and features have been left out of flowchart <NUM> in order to not obscure the discussion of the inventive features in the present application.

As a preliminary matter, two exemplary implementations of content processing pipeline <NUM> are herein described in greater detail in the interests of conceptual clarity. However, it is emphasized that the specific implementational details disclosed in the present application are merely by way of example, and are not to be interpreted as limiting the scope of the present inventive concepts in any way.

For example, in one implementation content processing pipeline <NUM> may include six nodes, i.e., first processing node 152a, four intermediate processing nodes 152b, and last processing node 152c. In one such implementation, first processing node 152a may be a mobile unit including a video camera. Content generated by first processing node 152a is transferred to intermediate processing node 152b in the form of a video transcoder, which, in turn, may output the transcoded video to another intermediate processing node 152b in the form of a segmenter configured to partition and package the content. The content output by the segmenter may be transferred to another intermediate processing node 152b in the form of a video production facility, where it is processed by another intermediate processing node 152b in the form of a color correction system. Finally, last processing node 152c may add additional text and/or graphic overlays to the content and broadcast the content to an audience of consumers of the video content.

Alternatively, in another implementation, first processing node 152a may be a camera capturing video of a green screen stage in a television studio. Content generated by first processing node 152a is transferred to intermediate processing node 152b in the form of a video transcoder, which, in turn, may transfer the transcoded video to another intermediate processing node 152b in the form of an online application for previsualization of the video content. The previsualized video content may then be transferred to another intermediate processing node 152b in the form of a color correction system, followed by transfer to last processing node 152c in the form of a playback system or device enabling user <NUM> to preview the video content prior to its distribution to consumers.

It is noted that in some implementations, content processing pipeline <NUM> may include a processing node, e.g., first processing node 152a, configured to create original content prior to processing of that original content by other processing nodes of content processing pipeline <NUM>. However, in other implementations, content processing pipeline <NUM> may receive content created outside of content processing pipeline <NUM>, for example by a third party content creator, such as an authorized field reporter using a mobile videography device, or from an amateur user submitting images captured using a smartphone video or still image camera, for instance.

Referring now to <FIG> in combination with <FIG>, <FIG>, and <FIG>, flowchart <NUM> begins with inserting synthesized test image(s) <NUM>/<NUM>/<NUM> into content stream <NUM>/<NUM>, synthesized test image(s) <NUM>/<NUM>/<NUM> being configured to activate one or more neurons of a malicious NN (action <NUM>). Content stream <NUM>/<NUM> may be a digital content stream including a sequence of digital photographs, or a video clip, for example. Synthesized test image(s) <NUM>/<NUM>/<NUM> may be inserted into content stream <NUM>/<NUM> by being appended to the beginning or end of content stream <NUM>/<NUM>, or by being inserted between photographic images or video frames of content stream <NUM>/<NUM>. In one implementation, synthesized test image(s) <NUM>/<NUM>/<NUM> may be inserted into content stream <NUM>/<NUM> when a hash value of content included in content stream <NUM>/<NUM> is embedded or appended to content stream <NUM>/<NUM>. Synthesized test image(s) <NUM>/<NUM>/<NUM> may be inserted into content stream <NUM>/<NUM> by software code <NUM>/<NUM>, executed by hardware processor <NUM>, and using image insertion module <NUM>.

It is noted that, in some implementations, software code <NUM>/<NUM> may obtain previously produced synthesized test image(s) <NUM>/<NUM>/<NUM> from image database <NUM>. However, in other implementations, software code <NUM>/<NUM> may include optional NN <NUM>/<NUM>. In those implementations, hardware processor <NUM> may execute software code <NUM>/<NUM> to produce synthesized test image(s) <NUM>/<NUM>/<NUM> using NN <NUM>/<NUM>. For example, in some implementations, NN <NUM>/<NUM> may be a GAN including generator <NUM> used to produce modifications to a random image, and including discriminator <NUM> implemented as a model of the target malicious NN architecture.

In implementations in which NN <NUM>/<NUM> is used to produce synthesized test image(s) <NUM>/<NUM>/<NUM>, NN <NUM>/<NUM> must be trained prior to action <NUM>. NN <NUM>/<NUM> may be trained using training platform <NUM>, training data <NUM>, and training module <NUM> of software code <NUM>/<NUM>. The goal of training is to synthesize test images that are unrecognizable to a human observer but are likely to be identified as a recognizable object by the target malicious NN.

During training, discriminator <NUM> of NN <NUM>/<NUM> may look at a synthesized test image output by generator <NUM> and determine whether it convincingly depicts a recognizable object. Validation of the learning process during training may be performed by user <NUM>, who may utilize user system <NUM> to evaluate synthesized test image(s) <NUM>/<NUM>/<NUM>. However, in some implementations, validation of the learning can be performed as an automated process using discriminator <NUM>. Once training is completed, software code <NUM>/<NUM> including NN <NUM>/<NUM> may be utilized in an automated process to produce synthesized test image(s) <NUM>/<NUM>/<NUM>.

In some implementation, a single synthesized test image <NUM>/<NUM>/<NUM> may be inserted into content stream <NUM>/<NUM> in action <NUM>. However, in other implementations, it may be advantageous or desirable to insert multiple synthesized test images <NUM>/<NUM>/<NUM> into content stream <NUM>/<NUM>. For example, because any one of synthesized test image <NUM>/<NUM>/<NUM> may or may not activate a malicious NN and cause it to hallucinate, the insertion of multiple synthesized test images <NUM>/<NUM>/<NUM> increases the likelihood that the presence of a malicious NN in content processing pipeline <NUM> will be detected. When multiple synthesized test images <NUM>/<NUM>/<NUM> are inserted into content stream <NUM>/<NUM>, those synthesized test images <NUM>/<NUM>/<NUM> may be grouped together, for example, at the beginning of content stream <NUM>/<NUM>, or may distributed throughout content stream <NUM>/<NUM>.

Flowchart <NUM> continues with providing content stream <NUM>/<NUM> including synthesized test image(s) <NUM>/<NUM>/<NUM> as input stream <NUM>/<NUM> to first processing node 152a of content processing pipeline <NUM> (action <NUM>). As shown by <FIG>, input stream <NUM>/<NUM> may be provided to first processing node 152a of content processing pipeline <NUM> by being transmitted to first processing node 152a via communication network <NUM> and network communication links <NUM>. Action <NUM> may be performed by software code <NUM>/<NUM>, executed by hardware processor <NUM>. In some implementations, input stream <NUM>/<NUM> including synthesized test image(s) <NUM>/<NUM>/<NUM> may be provided to content processing pipeline <NUM> periodically, such as one a day, or every few hours for example.

Flowchart <NUM> continues with receiving, after processing of input stream <NUM>/<NUM> including synthesized test image(s) <NUM>/<NUM>/<NUM> by at least one portion of content processing pipeline <NUM>, an output stream from one of first processing node 152a or a second processing node 152b of content processing pipeline <NUM>, the output stream including post-processed test image(s) <NUM> (action <NUM>). In some implementations, system <NUM> may produce security assessment <NUM>/<NUM> of first processing node 152a alone. In those implementations, hardware processor <NUM> may execute software code <NUM>/<NUM> to receive output stream <NUM>/<NUM> from first processing node 152a via communication network <NUM> and network communication links <NUM> in action <NUM>.

In other implementations, system <NUM> may produce security assessment <NUM>/<NUM> of first processing node 152a and one or more intermediate processing nodes 152b. In those implementations, hardware processor <NUM> may execute software code <NUM>/<NUM> to receive output stream <NUM>/<NUM> from a second processing node included in one or more intermediate processing nodes 152b via communication network <NUM> and network communication links <NUM> in action <NUM>.

In still other implementations, system <NUM> may produce security assessment <NUM>/<NUM> of the entirety of content processing pipeline <NUM>. In those more comprehensive implementations, hardware processor <NUM> may execute software code <NUM>/<NUM> to receive output stream <NUM>/<NUM> from last processing node 152c via communication network <NUM> and network communication links <NUM> in action <NUM>. Action <NUM> may be performed by software code <NUM>/<NUM>, executed by hardware processor <NUM>.

Flowchart <NUM> continues with comparing post-processed test image(s) <NUM> in the received output stream with expected image(s) <NUM> corresponding respectively to synthesized test image(s) <NUM>/<NUM>/<NUM> (action <NUM>). Expected image(s) <NUM> may be substantially identical to synthesized test image(s) <NUM>/<NUM>/<NUM>, or may be synthesized test image(s) <NUM>/<NUM>/<NUM> modified by predictable transformations imposed by the processing performed by one or more processing nodes of content processing pipeline <NUM>. Comparison of post-processed test image(s) <NUM> with expected image(s) <NUM> may be performed by software code <NUM>/<NUM>, executed by hardware processor <NUM>, and using security assessment module <NUM>.

In some implementations, as shown in <FIG>, flowchart <NUM> may conclude with validating the at least one portion of content processing pipeline <NUM> assessed for security as secure when post-processed test image(s) <NUM> matches/match expected image(s) <NUM> (action 475a). As noted above, in use cases in which a malicious NN is present in the portion of content processing pipeline <NUM> being assessed, the manipulation of synthesized test image(s) <NUM>/<NUM>/<NUM> by the malicious NN will result in post-processed test image(s) <NUM> having unexpected features not present in expected image(s) <NUM>. As a result, a match between post-processed test image(s) <NUM> and expected image(s) <NUM> advantageously establishes the absence of a malicious NN within the portion of content processing pipeline <NUM> assessed in actions <NUM>, <NUM>, <NUM>, <NUM>, and 475a, thereby validating data security in that portion of the pipeline.

In implementations in which a single synthesized test image <NUM>/<NUM>/<NUM> is inserted into content stream <NUM>/<NUM> in action <NUM>, a single post-processed test image <NUM> is compared to a single expected image <NUM>. However, in implementations in which multiple synthesized test image(s) <NUM>/<NUM>/<NUM> are inserted into content stream <NUM>/<NUM>, each of multiple post-processed test images <NUM> are compared to a corresponding one of expected images <NUM>. In those latter implementations, validation of the portion of content pipeline assessed in actions <NUM>, <NUM>, <NUM>, and <NUM> (hereinafter "actions <NUM>-<NUM>") may require that each of post-processed test images <NUM> match its respective expected image <NUM>.

In some implementations, it may be advantageous or desirable to assess the security of content processing pipeline <NUM> as a whole. For example, where output stream <NUM>/<NUM> is received from last processing node 152c of content processing pipeline <NUM> in action <NUM>, the entirety of content processing pipeline <NUM> can be validated as secure when post-processed test image(s) <NUM> matches/match expected image <NUM>. However, in other implementations, it may advantageous or desirable to assess the security of content processing pipeline by assessing the security of each processing node individually. In those latter implementations, for example, actions <NUM>-<NUM> may be performed for each of first processing node 152a, intermediate processing node 152b, and last processing node 152c.

Conversely, where post-processed test image(s) <NUM> fail to match expected image(s) <NUM>, the method outlined by flowchart <NUM> advantageously detects the likely presence of a malicious NN on at least one processing node of content processing pipeline <NUM>. In those situations, actions <NUM>, <NUM>, <NUM>, and <NUM> may be repeated for each processing node of the portion of content processing pipeline <NUM> providing the output stream received in action <NUM>, or for subsets of those processing nodes (action 475b). For example, where input stream <NUM>/<NUM> including synthesized test image(s) <NUM>/<NUM>/<NUM> is provided to first processing node 152a and post-processed test image(s) <NUM> included in output stream <NUM>/<NUM> received from last processing node 152c fails/fail to match expected image(s) <NUM>, input stream <NUM>/<NUM> may be provided anew to first processing node 152a, but this time post-processed test image(s) <NUM> included in output stream <NUM>/<NUM> received from intermediate processing node 152b could be compared with expected image(s) <NUM>. If post-processed test image(s) <NUM> included in output stream <NUM>/<NUM> matches/match expected image(s) <NUM> but post-processed test image(s) <NUM> included in output stream <NUM>/<NUM> does/do not, first processing node 152a and intermediate processing node 152b can by validated as secure, while the image manipulations performed by the malicious NN are advantageously identified as being isolated to last processing node 152c.

It is noted that, in some implementations, hardware processor <NUM> may execute software code <NUM>/<NUM> to perform the actions outlined in flowchart <NUM> to produce security assessment <NUM>/<NUM> in an automated process from which human involvement may be omitted. That is to say, in those implementations, system <NUM> is configured to produce security assessment <NUM>/<NUM> automatically. Moreover, in some implementations, hardware processor <NUM> may further execute software code <NUM>/<NUM> to render security assessment <NUM>/<NUM> on display <NUM> of system <NUM>. Alternatively, in some implementations, hardware processor <NUM> may execute software code <NUM>/<NUM> to transmit security assessment <NUM>/<NUM> to user system <NUM>, via communication network <NUM> and network communication links <NUM>, for rendering on display <NUM> of user system <NUM>.

Thus, the present application discloses systems and methods for securing a content processing pipeline that overcome the drawbacks and deficiencies in the conventional art. As described above, by inserting a synthesized test image configured to activate one or more neurons of a malicious NN into a content stream provided as an input stream to a content processing pipeline, the present security solution advantageously enables detection of the malicious NN when it is present in the pipeline. In addition, by comparing the synthesized test image included in an output stream received from one or more content processing nodes of the pipeline to an expected image corresponding to the synthesized test image, the present solution enables identification of the node or nodes of the pipeline infected by the malicious NN.

Claim 1:
A method for producing a security assessment of at least one portion of a content processing pipeline including a plurality of processing nodes, the method comprising:
inserting a synthesized test image into a content stream as a test image, the synthesized test image being configured to activate one or more neurons of a malicious neural network;
providing the content stream including the synthesized test image as an input stream to a first processing node of the plurality of processing nodes of the content processing pipeline;
receiving an output stream from one of the first processing node or a second processing node of the plurality of processing nodes of the content processing pipeline, the output stream including a post-processed test image;
comparing the post-processed test image in the received output stream with an expected image corresponding to the synthesized test image; and
validating at least the one portion of the content processing pipeline as secure when the post-processed test image in the received output stream matches the expected image.