System and method with masking for certified defense against adversarial patch attacks

A computer-implemented system and method relate to certified defense against adversarial patch attacks. A set of one-mask images is generated using a first mask at a set of predetermined regions of a source image. The source image is obtained from a sensor. A set of one-mask predictions is generated, via a machine learning system, based on the set of one-mask images. A first one-mask image is extracted from the set of one-mask images. The first one-mask image is associated with a first one-mask prediction that is identified as a minority amongst the set of one-mask predictions. A set of two-mask images is generated by masking the first one-mask image using a set of second masks. The set of second masks include at least a first submask and a second submask in which a dimension of the first submask is less than a dimension of the first mask. A set of two-mask predictions is generated based on the set of two-mask images. Class data, which classifies the source image, is selected based on the set of two-mask predictions.

TECHNICAL FIELD

This disclosure relates generally to machine learning systems, and more particularly to certified defenses against patch attacks.

BACKGROUND

Image classifiers may generate false predictions when small changes, such as perturbations, are applied to the input. For example, adversarial perturbations can either apply to all pixels of an image bounded by a norm constraint so the perturbed image remains visually the same or apply to a small sub-region of the image in which pixel values within this sub-region are changed to an allowable value approximating the addition of a small poster or a small object in a scene. The latter is known as patch attack, which has drawn attention recently for being produced in the physical world without requiring direct access to the input of the classifier. Although PatchCleanser provides a certified defense against such patch attacks, PatchCleanser applies a masking strategy to an image in a manner that loses significant image information and affects classification accuracy.

SUMMARY

The following is a summary of certain embodiments described in detail below. The described aspects are presented merely to provide the reader with a brief summary of these certain embodiments and the description of these aspects is not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be explicitly set forth below.

According to at least one aspect, a computer-implemented method includes obtaining a source image. The method includes generating a set of one-mask images using a first mask at a set of predetermined regions of the source image. The method includes generating, via a machine learning system, a set of one-mask predictions based on the set of one-mask images. The method includes extracting a first one-mask image from the set of one-mask images. The first one-mask image being associated with a first one-mask prediction that is identified as a minority amongst the set of one-mask predictions. The method includes generating a set of two-mask images by masking the first one-mask image using a set of second masks. The set of second masks include at least a first submask and a second submask in which a dimension of the first submask is less than a dimension of the first mask. The method includes generating, via the machine learning system, a set of two-mask predictions based on the set of two-mask images. The method includes selecting class data that classifies the source image based on the set of two-mask predictions.

According to at least one aspect, a system includes a processor and a memory. The memory is in data communication with the processor. The memory has computer readable data including instructions stored thereon that, when executed by the processor, cause the processor to perform a method for defending against adversarial patch attacks. The method includes obtaining a source image. The method includes generating a set of one-mask images using a first mask at a set of predetermined regions of the source image. The method includes generating, via a machine learning system, a set of one-mask predictions based on the set of one-mask images. The method includes extracting a first one-mask image from the set of one-mask images. The first one-mask image being associated with a first one-mask prediction that is identified as a minority amongst the set of one-mask predictions. The method includes generating a set of two-mask images by masking the first one-mask image using a set of second masks. The set of second masks include at least a first submask and a second submask in which a dimension of the first submask is less than a dimension of the first mask. The method includes generating, via the machine learning system, a set of two-mask predictions based on the set of two-mask images. The method includes selecting class data that classifies the source image based on the set of two-mask predictions.

According to at least one aspect, a non-transitory computer readable medium has computer readable data including instructions stored thereon that, when executed by a processor, cause the processor to perform a method for defending against patch attacks. The method includes obtaining a source image. The method includes generating a set of one-mask images using a first mask at a set of predetermined regions of the source image. The method includes generating, via a machine learning system, a set of one-mask predictions based on the set of one-mask images. The method includes extracting a first one-mask image from the set of one-mask images. The first one-mask image being associated with a first one-mask prediction that is identified as a minority amongst the set of one-mask predictions. The method includes generating a set of two-mask images by masking the first one-mask image using a set of second masks. The set of second masks include at least a first submask and a second submask in which a dimension of the first submask is less than a dimension of the first mask. The method includes generating, via the machine learning system, a set of two-mask predictions based on the set of two-mask images. The method includes selecting class data that classifies the source image based on the set of two-mask predictions.

These and other features, aspects, and advantages of the present invention are discussed in the following detailed description in accordance with the accompanying drawings throughout which like characters represent similar or like parts.

DETAILED DESCRIPTION

The embodiments described herein, which have been shown and described by way of example, and many of their advantages will be understood by the foregoing description, and it will be apparent that various changes can be made in the form, construction, and arrangement of the components without departing from the disclosed subject matter or without sacrificing one or more of its advantages. Indeed, the described forms of these embodiments are merely explanatory. These embodiments are susceptible to various modifications and alternative forms, and the following claims are intended to encompass and include such changes and not be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling with the spirit and scope of this disclosure.

FIG.1is a diagram of a non-limiting example of a system100, which is configured to defend against adversarial patch attacks. The system100includes at least a processing system110with at least one processing device. For example, the processing system110includes at least an electronic processor, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), any suitable processing technology, or any number and combination thereof. The processing system110is operable to provide the functionality as described herein.

The system100includes a memory system120, which is operatively connected to the processing system110. In an example embodiment, the memory system120includes at least one non-transitory computer readable storage medium, which is configured to store and provide access to various data to enable at least the processing system110to perform the operations and functionality, as disclosed herein. In an example embodiment, the memory system120comprises a single memory device or a plurality of memory devices. The memory system120can include electrical, electronic, magnetic, optical, semiconductor, electromagnetic, or any suitable storage technology that is operable with the system100. For instance, in an example embodiment, the memory system120can include random access memory (RAM), read only memory (ROM), flash memory, a disk drive, a memory card, an optical storage device, a magnetic storage device, a memory module, any suitable type of memory device, or any number and combination thereof. With respect to the processing system110and/or other components of the system100, the memory system120is local, remote, or a combination thereof (e.g., partly local and partly remote). For example, the memory system120can include at least a cloud-based storage system (e.g. cloud-based database system), which is remote from the processing system110and/or other components of the system100.

The memory system120includes at least a patch masker130, a machine learning system140, a computer vision application150, and other relevant data160, which are stored thereon. The patch masker130includes computer readable data with instructions, which, when executed by the processing system110, is configured to defend against patch attacks. The computer readable data can include instructions, code, routines, various related data, any software technology, or any number and combination thereof. In an example embodiment, the machine learning system140includes at least one artificial neural network model and/or any suitable machine learning model, which is configured to perform a classification task. In this regard, for example, the machine learning system140includes a classifier (e.g., a convolutional neural network (CNN), ResNet, vision transformer (ViT), or any suitable classification model). Also, the computer vision application150is configured to apply the output (e.g., class data) of the machine learning system140to computer vision technology. Meanwhile, the other relevant data160provides various data (e.g. operating system, etc.), which enables the system100to perform the functions as discussed herein.

The system100is configured to include at least one sensor system170. The sensor system170includes one or more sensors. For example, the sensor system170includes an image sensor, a camera, a radar sensor, a light detection and ranging (LIDAR) sensor, a thermal sensor, an ultrasonic sensor, an infrared sensor, a motion sensor, an audio sensor (e.g., microphone), any suitable sensor, or any number and combination thereof. The sensor system170is operable to communicate with one or more other components (e.g., processing system110and memory system120) of the system100. For example, the sensor system170may provide sensor data, which is then used by the processing system110to generate digital image data based on the sensor data. In this regard, the processing system110is configured to obtain the sensor data as digital image data directly or indirectly from one or more sensors of the sensor system170. The sensor system170is local, remote, or a combination thereof (e.g., partly local and partly remote). Upon receiving the sensor data, the processing system110is configured to process this sensor data (e.g. image data) in connection with the patch masker130, the machine learning system140, the computer vision application150, the other relevant data160, or any number and combination thereof.

In addition, the system100may include at least one other component. For example, as shown inFIG.1, the memory system120is also configured to store other relevant data160, which relates to operation of the system100in relation to one or more components (e.g., sensor system170, I/O devices180, and other functional modules190). In addition, the system100is configured to include one or more I/O devices180(e.g., display device, keyboard device, speaker device, etc.), which relate to the system100. Also, the system100includes other functional modules190, such as any appropriate hardware, software, or combination thereof that assist with or contribute to the functioning of the system100. For example, the other functional modules190include communication technology (e.g. wired communication technology, wireless communication technology, or a combination thereof) that enables components of the system100to communicate with each other as described herein. In this regard, the system100is operable to execute the patch masker130to defend against adversarial patch attacks with respect to the machine learning system140, as described herein.

FIG.2AandFIG.2Billustrate a flow diagram of a process200for defending against adversarial patch attacks according to an example embodiment. The process200comprises a computer-implemented method, which is stored in the memory system120as the patch defense system130and which is executed via one or processors of the processing system110. The process200may include more steps or less steps than those steps discussed with respect toFIG.2AandFIG.2Bprovided that such modifications provide the same functions and/or objectives as the process200ofFIG.2AandFIG.2B.

At step202, according to an example, the processing system110obtains an input image, which may be referred to as the source image300(denoted as “x”). The processing system110may obtain the source image300directly or indirectly from the sensor system170. The processing system110may obtain the source image300from the memory system120. For example, inFIG.3, the source image300is a digital image, which has a height, denoted as “h,” and a width, denoted as “w.” The source image300may be a clean image with no adversarial pixels. Alternatively, the source image300may include an adversarial patch, which includes adversarial pixels. In any event, the processing system110is configured to defend against an adversarial attack involving an adversarial patch, having a height, denoted as “hp,” and a width, denoted as “wp,” on the source image300.

At step204, according to an example, the processing system110generates a set of one-mask images using a first mask302with respect to the source image300. In an example embodiment, the first mask302comprises a rectangular shape having a height, denoted as hM1, and a width, denoted as wM1. In this example, the first mask302is sized to cover at most a 3% adversarial patch of a square shape on the source image300. The first mask302serves to mask or block an area of pixels on the source image300so that those masked pixels are not visible/available for processing by the machine learning system140.

The set of one-mask images may be denoted by {x1, x2, . . . , xN}, where N represents an integer value and the total number of one-mask images. The set of one-mask images comprise the first mask302at a set of predetermined regions that are identified based on a set of predetermined locations of the source image300. In an example, each predetermined region is defined with respect to the first mask302such that an area of a predetermined region is equal to an area of the first mask302. The set of predetermined regions collectively cover every pixel of the source image300. Also, each one-mask image includes the first mask302at a single predetermined region within the set of predetermined regions. For instance, as a non-limiting example,FIG.3illustrates a set of one-mask images, which include nine different one-mask images using the first mask302. As shown inFIG.3, the first mask302covers a different predetermined region of the source image300in each one-mask image.

At step206, according to an example, the processing system110generates, via the machine learning system140, a set of one-mask predictions based on the set of one-mask images. More specifically, the processing system110provides each one-mask image as input data to the machine learning system140. In response to a given one-mask image as input data, the machine learning system140is configured to generate prediction data, which may be referred to as a one-mask prediction. For example, when the machine learning system140is a classifier, the machine learning system140is configured to generate class data as the one-mask prediction that classifies the given one-mask image upon receiving that given one-mask image as input data. Upon receiving the set of one-mask images, the machine learning system140generates the set of one-mask predictions.

Referring toFIG.3as a non-limiting example, the processing system110is configured to generate a set of one-mask predictions that include (1) a first one-mask prediction when the machine learning system140receives the first one-mask image304as input data, (2) a second one-mask prediction when the machine learning system140receives the second one-mask image306as input data, (3) a third one-mask prediction when the machine learning system140receives the third one-mask image308as input data, (4) a fourth one-mask prediction when the machine learning system140receives the fourth one-mask image310as input data, (5) a fifth one-mask prediction when the machine learning system140receives the fifth one-mask image312as input data, (6) a sixth one-mask prediction when the machine learning system140receives the sixth one-mask image314as input data, (7) a seventh one-mask prediction when the machine learning system140receives the seventh one-mask image316as input data, (8) an eighth one-mask prediction when the machine learning system140receives the eighth one-mask image318as input data, and (9) a ninth one-mask prediction when the machine learning system140receives the ninth one-mask image320as input data.

At step208, according to an example, the processing system110determines whether or not there is disagreement among the set of one-mask predictions. In this regard, for example, the processing system110identifies a disagreement when there is at least one one-mask prediction that is different from the other one-mask predictions. As a non-limiting example, for instance, the processing system110may determine that there is disagreement when the machine learning system140classifies at least one one-mask image as a “stop sign” while classifying other one-mask images as a “tree.” In this regard, this disagreement occurs between at least one one-mask prediction (e.g., stop sign) and at least one other one-mask prediction (e.g., tree). Additionally or alternatively, the processing system110determines whether or not there is unanimous agreement among the set of one-mask predictions. As each one-mask image comprises the same source image300with the first mask302at a different predetermined region of the source image, the processing system110determines that the one-mask prediction is valid when each one of the set of one-mask predictions agree with each other (i.e. when they all have the same one-mask prediction). Accordingly, upon determining that all of one-mask predictions within the set of one-mask predictions are the same or in unanimous agreement, then the processing system110performs step210. Alternatively, upon determining that there is disagreement among the set of one-mask predictions, then the processing system100performs step212.

At step210, according to an example, the processing system110outputs the unanimous one-mask prediction f(xi) as the predication data for the source image300. As a non-limiting example, for instance, if all of the one-mask predictions for the set of one-mask images are the same (e.g., all class data is tree for the one-mask images), then the processing system110selects and outputs any one of the one-mask predictions as the class data to classify the source image300. The processing system110is then enabled to use this class data with respect to the computer vision application150.

At step212, according to an example, the processing system110extracts each one-mask image with a one-mask prediction that is a minority from among the set of one-mask predictions. As a non-limiting example, for instance, when the machine learning system140classifies a single one-mask image as a stop sign and classifies several other one-mask images as a tree, then the processing system110determines that the one-mask prediction of “stop sign” is a minority one-mask prediction within the set of one-mask predictions. The processing system110also determines that the one-mask prediction of “tree” is a majority one-mask prediction within the set of one-mask predictions when “tree” occurs the most amongst the set of one-mask predictions or constitutes a greatest number of the one-mask predictions amongst the set of one-mask predictions. For instance, if seven one-mask images are classified as “tree,” then the one-mask prediction of “tree” is considered to be a majority one-mask prediction for the set of one-mask images (e.g., nine one-mask images). Also, there may be more than one one-mask prediction that may be considered a minority one-mask prediction. The processing system110extracts a one-mask image (e.g., the first one-mask image304), which generated the minority one-mask prediction (e.g., the stop sign), and processes this extracted one-mask image at step214. If there are more than one one-mask images are identified as generating minority one-mask predictions, then each of these one-mask images are extracted and processed at step214.

At step214, according to an example, the processing system110generates a set of two-mask images for each extracted one-mask image using a set of second masks400/700. The set of second masks may include the set of second masks400ofFIG.4, the set of second masks700ofFIG.7, or the like. Each two-mask image includes (i) the first mask302at a fixed location (i.e., one of the predetermined regions) with respect to the source image300via the extracted one-mask image and (ii) a submask at one of the predetermined regions within the set of predetermined regions. For instance,FIG.5illustrates a non-limiting example of a set of two-mask images. In this case, the set of two-mask images includes thirty-three different two-mask images, which are generated based on the extracted one-mask image (e.g. the first one-mask image304) using the set of second masks400(FIG.4).

At step216, according to an example, the processing system110generates, via the classifier, a set of two-mask predictions based on the set of two-mask images for each extracted one-mask image. More specifically, the processing system110provides each two-mask image as input data to the machine learning system140. In response to a given two-mask image as input data, the machine learning system140is configured to generate prediction data, which may be referred to as a two-mask prediction. For example, when the machine learning system140is a classifier, the machine learning system140is configured to generate class data as the two-mask prediction that classifies the given two-mask image upon receiving that given two-mask image as input data. Upon receiving the set of two-mask images, the machine learning system140generates the set of two-mask predictions. Referring toFIG.5, as a non-limiting example, the processing system110is configured to generate thirty-three two-mask predictions when the machine learning system140receives the thirty-three two-mask images as input data.

At step218, according to an example, the processing system110determines whether or not any extracted one-mask image has unanimous agreement for its set of two-mask predictions. Referring toFIG.5, as an example, for a given extracted one-mask image, the processing system110determines whether or not there is unanimous agreement (or any disagreement) among the thirty-three two-mask predictions, which was generated by the machine learning system140in response to the thirty-three two-mask images. When there is unanimous agreement among the set of two-mask predictions (e.g., all thirty-three two-mask predictions are the same) for the thirty-three two-mask images, which were based on that extracted one-mask image, the processing system110performs step220. The processing system110performs this evaluation for each extracted one-mask image to determine if any extracted one-mask image has unanimous agreement among its set of two-mask predictions. When none of the extracted one-mask images have unanimous agreement among its corresponding set of two-mask predictions, the processing performs step222.

At step220, according to an example, the processing system110selects the unanimous two-mask prediction as the class data to classify the source image300. As a non-limiting example, for instance, if all of the two-mask predictions for the set of one-mask predictions are the same (e.g., all classified as stop sign), then the processing system110selects and outputs any one of the one-mask predictions (e.g., “stop sign”) as the class data to classify the source image300. The processing system110is then enabled to use this class data with respect to the computer vison application150.

At step222, according to an example, the processing system100selects the one-mask prediction, which is a majority within the set of one-mask predictions, as the class data to classify the source image300. As a non-limiting example, for instance, upon determining that there is disagreement amongst the set of two-mask predictions, then the processing system110selects and outputs a majority one-mask prediction, which has a majority of votes, from among the set of one-mask predictions as the class data to classify the source image300. For instance, in the aforementioned non-limiting example, the processing system110outputs “tree” as the class data for the source image300since “tree” is deemed the majority one-mask prediction. That is, the processing system110does not select any of the two-mask predictions at least due to the disagreement among them. The processing system110is then enabled to use the majority one-mask prediction as the class data with respect to the computer vision application150.

FIG.3is a diagram of a non-limiting example of a set of one-mask images according to an example embodiment. The set of one-mask images includes a first mask302at a set of predetermined regions of the source image300. More specifically, in this example, the set of one-mask images include (1) a first one-mask image304that comprises the first mask302at a first predetermined region of the source image300, (2) a second one-mask image306that comprises the first mask302at a second predetermined region of the source image300, (3) a third one-mask image308that comprises the first mask302at a third predetermined region of the source image300, (4) a fourth one-mask image310that comprises the first mask302at a fourth predetermined region of the source image300, (5) a fifth one-mask image312that comprises the first mask302at a fifth predetermined region of the source image300, (6) a sixth one-mask image314that comprises the first mask302at a sixth predetermined region of the source image300, (7) a seventh one-mask image316that comprises the first mask302at a seventh predetermined region of the source image300, (8) an eighth one-mask image318that comprises the first mask302at an eighth predetermined region of the source image300, and (9) a ninth one-mask image320that comprises the first mask302at a ninth predetermined region of the source image300. As shown inFIG.3, the first mask302is associated with every pixel of the source image300collectively across the set of one-mask images via the set of predetermined regions. Accordingly, this set of one-mask images is generated such that at least one of the one-mask images has the first mask302covering an adversarial patch if present on the source image300.

FIG.4is a diagram of an example of the first mask302in relation to an example of a set of second masks400according to an example embodiment. In this case, the set of second masks400include a first submask402, a second submask404, a third submask406, and a fourth submask408. Also, as shown inFIG.4, each submask is smaller in size (e.g., surface area and dimensions) than the first mask302. In this regard,FIG.4illustrates a boundary of each submask in relation to a boundary of the first mask302for comparison. As a non-limiting example, for instance, the first mask comprises 100×100 pixels. In contrast, the first submask402covers 70×70 pixels, the second submask404covers 70×70 pixels, a third submask406covers 70×70 pixels, and a fourth submask408covers 70×70 pixels. Also, each submask is configured to cover at most a putative 3% adversarial patch, at most, of a square shape on the source image300.

Each submask may be generated directly. Alternatively, each submask may be generated by demasking a copy of the first mask302. The demasking may comprise performing an L-demasking operation on a copy of the first mask302. In this regard, for example, copies of the first mask302may serve as a basis for generating each submask. For example, the first submask402is generated by removing an L-shape region410from a copy of the first mask302at a top, right corner region. The second submask404is generated by removing an L-shape region410from a copy of the first mask302at a top, left corner region. The third submask406is generated by removing an L-shape region410from a copy of the first mask302at a bottom, left corner region. The fourth submask408is generated by removing an L-shape region410from a copy of the first mask302at a bottom, right corner region. The L-shape region410comprises the same dimensions and area with respect to each of the submasks of the set of second masks400, but the L-shape region410may be oriented differently (e.g., 90 degree rotation) with respect to each of the submasks of the set of second masks400.

In the above examples, when generating each submask, the processing system110performs at least one demasking operation that satisfies equation 1, equation 2, equation 3, and equation 4. These equations are applied when generating the first submask402. These equations are applied when generating the second submask404. These equations are applied when generating the third submask406. These equations are applied when generating the fourth submask408. In these equations, hmask1represents a height of the first mask302, wmask1represents a width of the first mask302, hLdemaskrepresents a demasking height, wLdemaskrepresents a demasking width, hprepresents the height of an adversarial patch, and wprepresents a width of the adversarial patch, as indicated inFIG.4.
hLdemask=[(hmask1−hp)/2]  [1]
hsubmask=hmask1−hLdemask[2]
wLdemask=[(wmask1−wp)/2]  [3]
wsubmask=wmask1−wLdemask[4]

As shown inFIG.4, the first mask302provides a greater masking area than any submask from the set of second masks400. Also, a combined masking area of the first mask302and any submask is less than a combined masking area of the first mask302and another first mask302on the source image300. In this regard, by using a submask, the patch masker130is enabled to provide the machine learning system140with more pixels and thus more discriminative image features to classify a two-mask image such that classification accuracy is improved.

FIG.5is a diagram of a non-limiting example of a set of two-mask images according to an example embodiment. In this case, the set of two-mask images is generated based on the extracted one-mask image304, which in this non-limiting example is determined to generate a one-mask prediction that has a minority of votes from among the set of one-mask predictions. More specifically,FIG.5includes (1) a first representation500that represents a 1sttwo-mask image that comprises the first one-mask image304in which the first mask302occludes the subset of second masks400at the first predetermined region, (2) a second representation502that represents a subset of two-mask images (2nd, 3rd, 4th, and 5thtwo-mask images) with the subset of second masks400at the second predetermined region of the extracted one-mask image304, (3) a third representation504that represents a subset of two-mask images (6th, 7th, 8th, and 9thtwo-mask images) with the subset of second masks400at the third predetermined region of the extracted one-mask image304, (4) a fourth representation506that represents a subset of two-mask images (10th, 11th, 12th, and 13thtwo-mask images) with the subset of second masks400at the fourth predetermined region of the extracted one-mask image304, (5) a fifth representation508that represents a subset of two-mask images (14th, 15th, 16th, and 17thtwo-mask images) with the subset of second masks400at the fifth predetermined region of the extracted one-mask image304, (6) a sixth representation510that represents a subset of two-mask images (18th, 19th, 20th, and 21sttwo-mask images) with the subset of second masks400at the sixth predetermined region of the extracted one-mask image304, (7) a seventh representation512that represents a subset of two-mask images (22nd, 23rd, 24th, and 25thtwo-mask images) with the subset of second masks400at the seventh predetermined region of the extracted one-mask image, (8) an eighth representation514that represents a subset of two-mask images (26th, 27th, 28th, and 29thtwo-mask images) with the subset of second masks400at the eighth predetermined region of the extracted one-mask image, and (9) a ninth representation516that represents a subset of two-mask images (30th, 31st, 32nd, and 33rdtwo-mask images) with the subset of second masks400at the ninth predetermined region of the extracted one-mask image. As shown inFIG.5, the set of second masks400is associated with every pixel of the source image300(or the extracted one-mask image) via the set of predetermined regions when taken collectively across the set of two-mask images.

FIG.6is a diagram of a non-limiting example of a subset taken from the set of two-mask images according to an example embodiment. More specifically,FIG.6illustrates a selected example600of a subset of two-mask images. In this case, for discussion purposes, the ninth representation516is randomly chosen to show each two-mask image within its subset of two-mask images. As shown inFIG.5andFIG.6, this selected example600includes a subset of two-mask images in which the first mask302is fixed at the first predetermined region of the source image300and the set of second masks400are located at the ninth predetermined region of the source image300(or the ninth predetermined region of the extracted 1stone-mask image). More specifically, this subset of two-mask images include (1) a two-mask image602that comprises a first submask402at the ninth predetermined region of the extracted one-mask image, which includes the first mask302at the first predetermined region, (2) a two-mask image604that comprises a second submask404at the ninth predetermined region of the extracted one-mask image, which includes the first mask302at the first predetermined region, (3) a two-mask image606that comprises a third submask406at the ninth predetermined region of the extracted one-mask image, which includes the first mask302at the first predetermined region, and (4) a two-mask image608that comprises a fourth submask408at the ninth predetermined region of the extracted one-mask image, which includes the first mask302at the first predetermined region.

In addition,FIG.6shows a boundary of a predetermined region to illustrate the creation of a masked region and one or more unmasked regions when a given submask is applied that predetermined region. More specifically, for example,FIG.6shows that the ninth predetermined region includes an unmasked region corresponding to the L-shaped region410. Each unasked region exposes pixels of the source image300. The unmasked region is disposed between an edge of the source image300and an edge of any one of the submasks via the L-shape region410. In this example, a size of a predetermined region is associated with a size (and/or boundary) of the first mask302. As such, the processing system110generates some two-mask images, which reveal image features at edge portions and/or corner portions of the source image300and which contribute to improved classification accuracy.

FIG.7is a diagram of an example of the first mask302in relation to another example of a set of second masks700according to an example embodiment. In this case, the set of second masks700include a first submask702, a second submask704, a third submask706, a fourth submask708, and a fifth submask710. In this regard, the processing system110generates a greater number of two-mask images when applying the set of second masks700to the extracted one-mask image than when applying the set of second masks400to the extracted one-mask image. The set of second masks700are configured such that at least one submask is configured to mask an adversarial patch, which occupies 1% of the area of the source image300, when applied across the set of predetermined regions. Also, as shown inFIG.7, this set of second masks700collectively provide a sliding mask effect across a surface area that is equivalent or similar to the masking area of the first mask302. In this regard, each submask is structured as a vertical masking strip. Alternatively, each submask may be structured as a horizontal masking strip. In addition, this set of second masks400is also configured such that each submask is smaller in size (e.g., surface area, height dimension, width dimension, etc.) than the first mask.

Each submask may be generated directly. Alternatively, each submask may be generated by demasking a copy of the first mask302. In this regard, for example, the first mask302may serve as a basis for generating each submask. For example, the first submask702is generated by demasking of a copy of the first mask302, whereby a rectangular region712is removed from the copy of the first mask302such that the first submask702comprises a rectangular masking strip from a remaining portion of the first mask302. The second submask704is generated by demasking of a copy of the first mask302, whereby a first rectangular region714and a second rectangular region716are removed from the copy of the first mask302such that the second submask704comprises a rectangular masking strip between the first rectangular region714and the second rectangular region716. The third submask706is generated by demasking of a copy of the first mask302, whereby a first rectangular region718and a second rectangular region720are removed from the copy of the first mask302such that the third submask706comprises a masking strip between rectangular regions718and720. The fourth submask708is generated by demasking of a copy the first mask302, whereby a first rectangular region722and a second rectangular region724are removed from the copy of the first mask302such that the fourth submask708comprises a masking strip between rectangular regions722and724. The fifth submask710is generated by demasking of a copy of the first mask302, whereby rectangular region726is removed from the copy of the first mask302such that the fifth submask710comprises a rectangular masking strip from a remaining portion of the first mask302.

Also, as shown inFIG.7, the first mask302provides a greater masking area than any one of the submasks of the set of the second masks700. As a non-limiting example, for instance, a submask from this set of second masks700is 40% of the size of the first mask302. In this regard,FIG.7illustrates each submask in relation to a boundary of the first mask302for comparison. Also, a combined masking area of the first mask302and any one of the submasks of the set of the second masks700is less than a combined masking area of the first mask302and another first mask302on the source image300. In this regard, the patch masker130is enabled to provide the machine learning system140with more pixels and thus more discriminative image features to classify a two-mask image, thereby improving classification accuracy.

FIG.8is a diagram of a system800, which includes the patch masker130. The system800is configured to also include at least a sensor system810, a control system820, and an actuator system830. The system800is configured such that the control system820controls the actuator system830based on sensor data from the sensor system810. More specifically, the sensor system810includes one or more sensors and/or corresponding devices to generate sensor data. For example, the sensor system810includes an image sensor, a camera, a radar sensor, a light detection and ranging (LIDAR) sensor, a thermal sensor, an ultrasonic sensor, an infrared sensor, a motion sensor, a satellite-based navigation sensor (e.g., Global Positioning System (GPS) sensor), an optical sensor, an audio sensor, any suitable sensor, or any number and combination thereof. Upon obtaining detections from the environment, the sensor system810is operable to communicate with the control system820via an input/output (I/O) system870and/or other functional modules850, which includes communication technology.

The control system820is configured to obtain the sensor data directly or indirectly from one or more sensors of the sensor system810. In this regard, the sensor data may include sensor data from a single sensor or sensor-fusion data from a plurality of sensors. Upon receiving input, which includes at least sensor data, the control system820is operable to process the sensor data via the processing system840. In this regard, the processing system840includes at least one processor. For example, the processing system840includes an electronic processor, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), processing circuits, any suitable processing technology, or any combination thereof. Upon processing at least this sensor data, the processing system840is configured to extract, generate, and/or obtain proper input data (e.g., digital image data) for the patch masker130. In addition, the processing system840is operable to generate output data (e.g., class data) via the patch masker130and the machine learning system140based on communications with the memory system860. In addition, the processing system840is operable to provide actuator control data to the actuator system830based on the output data (e.g., class data), which is generated via the patch masker130and the machine learning system140.

The memory system860is a computer or electronic storage system, which is configured to store and provide access to various data to enable at least the operations and functionality, as disclosed herein. The memory system860comprises a single device or a plurality of devices. The memory system860includes electrical, electronic, magnetic, optical, semiconductor, electromagnetic, any suitable memory technology, or any combination thereof. For instance, the memory system860may include random access memory (RAM), read only memory (ROM), flash memory, a disk drive, a memory card, an optical storage device, a magnetic storage device, a memory module, any suitable type of memory device, or any number and combination thereof. In an example embodiment, with respect to the control system820and/or processing system840, the memory system860is local, remote, or a combination thereof (e.g., partly local and partly remote). For example, the memory system860may include at least a cloud-based storage system (e.g., cloud-based database system), which is remote from the processing system840and/or other components of the control system820.

The memory system860includes at least the patch masker130, which is executed via the processing system840. The patch masker130is configured to receive or obtain input data, which includes at least one digital image. In this regard, the patch masker130, via the machine learning system140and the processing system840, is configured to generate output data (e.g., class data) based on the input data (e.g., source image).

Furthermore, as shown inFIG.8, the system800includes other components that contribute to operation of the control system820in relation to the sensor system810and the actuator system830. For example, as shown inFIG.8, the memory system860is also configured to store other relevant data880, which relates to the operation of the system800in relation to one or more components (e.g., sensor system810, the actuator system830, etc.). Also, as shown inFIG.8, the control system820includes the I/O system870, which includes one or more interfaces for one or more I/O devices that relate to the system800. For example, the I/O system870provides at least one interface to the sensor system810and at least one interface to the actuator system830. Also, the control system820is configured to provide other functional modules850, such as any appropriate hardware technology, software technology, or any combination thereof that assist with and/or contribute to the functioning of the system800. For example, the other functional modules850include an operating system and communication technology that enables components of the system800to communicate with each other as described herein. With at least the configuration discussed in the example ofFIG.8, the system800is applicable in various technologies.

FIG.9is a diagram of the system800with respect to mobile machine technology900according to an example embodiment. As a non-limiting example, the mobile machine technology900includes at least a partially autonomous vehicle or robot. InFIG.9, the mobile machine technology900is at least a partially autonomous vehicle, which includes a sensor system810. The sensor system810includes an optical sensor, an image sensor, a video sensor, an ultrasonic sensor, a position sensor (e.g. GPS sensor), a radar sensor, a LIDAR sensor, any suitable sensor, or any number and combination thereof. One or more of the sensors may be integrated with respect to the vehicle. The sensor system810is configured to provide sensor data to the control system820.

The control system820is configured to obtain image data, which is based on sensor data or sensor-fusion data from the sensor system810. In addition, the control system820is configured to pre-process the sensor data to provide input data of a suitable form (e.g., digital image data) to the patch masker130. In this regard, the patch masker130is advantageously configured to provide one-mask images and two-mask images to the machine learning system140. In this regard, the patch masker130is advantageously configured provide a certified defense against patch attacks when the machine learning system140generates class data for a given digital image.

Upon receiving class data from the patch masker130, the control system820is configured to generate actuator control data based on the class data, which classifies a digital image and/or classifies a target object in the digital image. By using class data that is provided by the patch masker130(and the machine learning system140), the control system820is configured to generate actuator control data that allows for safer and more accurate control of the actuator system830of the vehicle as the patch masker130defends against adversarial patch attacks. The actuator system830may include a braking system, a propulsion system, an engine, a drivetrain, a steering system, or any number and combination of actuators of the vehicle. The actuator system830is configured to control the vehicle so that the vehicle follows rules of the roads and avoids collisions based at least on the class data that is provided by the patch masker130(and the machine learning system140).

FIG.10is a diagram of the system800with respect to security technology1000according to an example embodiment. As a non-limiting example, the security technology1000includes at least a monitoring system, a control access system, a surveillance system, or any suitable type of security apparatus. For instance, as one example,FIG.10relates to security technology1000, which is configured to physically control a locked state and an unlocked state of a lock of the door1002and display an enhanced image/video on the display1004. The sensor system810includes at least an image sensor that is configured to provide image/video data.

FIG.11is a diagram of the system800with respect to imaging technology1100according to an example embodiment. As a non-limiting example, the imaging technology1100includes a magnetic resonance imaging (MRI) apparatus, an x-ray imaging apparatus, an ultrasonic apparatus, a medical imaging apparatus, or any suitable type of imaging apparatus. InFIG.11, the sensor system810includes at least one imaging sensor. The control system820is configured to obtain image data from the sensor system810. The control system820is also configured to generate class data that classifies the image data via the patch masker130and the machine learning system140. In addition, the control system820is configured to provide more accurate medical information by using the class data, which is generated via the patch masker130and the machine learning system140. In addition, the control system820is configured to display the any relevant data (e.g., any data relating to the patch masker130, the machine learning system140, the computer vision application150, or any number and combination thereof) on the display1102.

As described in this disclosure, the embodiments provide several advantages and benefits. For example, the embodiments include a two-masking technique that reduces the amount of pixels that mask a digital image, thereby providing more pixels (and more image features) for a classifier to perform a classification task and thereby improving classification accuracy. The embodiments also leverage a certification process that requires greater agreement among the two-mask predictions and improves the robustness of the certified defense. The embodiments are advantageous in being able to certify digital images that have unanimous agreement among the two-mask predictions while not certifying digital images that do not have unanimous agreement. The embodiments certify digital images against patch proportions of at most 3% adversarial pixels. The embodiments are advantageous in providing a guaranteed lower bound of robust accuracy and improved certified robustness.

In addition, the embodiments provide a number of improvements over PatchCleanser. For example, the embodiments provide a certified defense with a set of two-mask images in which the amount of pixels that are masked in each two-mask image is reduced compared to PatchCleanser, thereby achieving better classification accuracy compared to PatchCleanser. In this regard, the embodiments provide a certified defense while revealing a greater amount of image content that might contain discriminative image features of an object compared to the amount of image content that is revealed by PatchCleaner. More specifically, for instance, with respect to a 224×224 image for certifying at most a 3% adversarial patch (or a 39×39 adversarial patch), the embodiments provide a combined masking area that is 29.69% of the digital image while PatchCleanser provides a combined masking area that is 39.85% of the digital image. Also, the embodiments require agreement or a unanimous vote over at least thirty-three or more while PatchCleanser requires agreement or a unanimous vote over nine, thereby providing improved certified robustness compared to PatchCleanser. Also, the embodiments are able to provide the machine learning system140with at least four times more variations of two-mask images than PatchCleanser. Overall, the embodiments are advantageous in defending against adversarial patch attacks with improved certified robustness and classification accuracy.

That is, the above description is intended to be illustrative, and not restrictive, and provided in the context of a particular application and its requirements. Those skilled in the art can appreciate from the foregoing description that the present invention may be implemented in a variety of forms, and that the various embodiments may be implemented alone or in combination. Therefore, while the embodiments of the present invention have been described in connection with particular examples thereof, the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the described embodiments, and the true scope of the embodiments and/or methods of the present invention are not limited to the embodiments shown and described, since various modifications will become apparent to the skilled practitioner upon a study of the drawings, specification, and following claims. Additionally or alternatively, components and functionality may be separated or combined differently than in the manner of the various described embodiments, and may be described using different terminology. These and other variations, modifications, additions, and improvements may fall within the scope of the disclosure as defined in the claims that follow.