P USHING THE L IMITS OF F EWSHOT A NOMALY D ETECTION IN I NDUSTRY V ISION : G RAPHCORE
Anonymous authors
Paper under double-blind review
A BSTRACT
In the area of fewshot anomaly detection (FSAD), efficient visual feature plays an essential role in memory bank M -based methods.
However, these methods do not account for the relationship between the visual feature and its rotated visual feature, drastically limiting the anomaly detection performance.
To push the limits, we reveal that rotation-invariant feature property has a significant impact in industrial-based FSAD.
Specifically, we utilize graph representation in FSAD and provide a novel visual isometric invariant feature (VIIF) as anomaly measurement feature.
As a result, VIIF can robustly improve the anomaly discriminating ability and can further reduce the size of redundant features stored in M by a large amount.
Besides, we provide a novel model GraphCore via VIIFs that can fast implement unsupervised FSAD training and can improve the performance of anomaly detection.
A comprehensive evaluation is provided for comparing GraphCore and other SOTA anomaly detection models under our proposed fewshot anomaly detection setting, which shows GraphCore can increase average AUC by 5.8%, 4.1%, 3.4%, and 1.6% on MVTec AD and by 25.5%, 22.0%, 16.9%, and 14.1% on MPDD for 1, 2, 4, and 8-shot cases, respectively.
I NTRODUCTION
With the rapid development of deep vision detection technology in artificial intelligence, the detection of anomalies/defects on the surface of industrial products (including scratches, stains, missing, etc.) has received unprecedented attention.
Changeover in manufacturing refers to the process of converting a line or machine from processing one product to another.
Since the equipment has not been completely fine-tuned after the starting of the production line, changeover frequently results in unsatisfactory anomaly detection (AD) performance.
How to achieve rapid training of industrial product models in the changeover scenario while assuring accurate anomaly detection is a critical issue in the actual production process.
The current state of AD in the industry is as follows: (1) In terms of detection accuracy, the performance of stateof-the-art (SOTA) AD models degrades dramatically during changeover.
Current mainstream work utilizes a considerable amount of training data as input to train the model, as shown in Fig. 1(a).
However, this will make data collecting challenging, even for unsupervised learning.
As a result, many approaches based on fewshot learning at the price of accuracy have been proposed.
For instance, Huang et al.
(2022) employ meta learning, as shown in Fig. 1(b).
While due to complicated settings, it is impossible to flexibly migrate to the new product during changeover, and the detection accuracy cannot be guaranteed.
In terms of training speed, when a large amount of data is utilized for training, the training progress for new goods is slowed in actual production line.
As is well-known, vanilla unsupervised AD requires to collect a large amount of information.
Even though meta learning works in fewshot learning, as shown in Fig. 1(b), it is still necessary to train a massive portion of previously collected data.
We state that AD of industrial products requires just a small quantity of data to achieve performance comparable to a large amount of data, i.e. a small quantity of image data can contain sufficient information to represent a large number of data.
Due to the fact that industrial products are manufactured with high stability (no evident distortion of shape and color cast), the taken images lack the diversity of natural images, and there is a problem with the shooting angle or rotation.
Therefore, it is more essential and necessary to extract rotation-invariant structural features.
As graph
[Figure]
neural networks (GNNs) are capable of robustly extracting non-serialized structural features (Han et al.
(2022), Bruna et al.
(2013), Hamilton et al.
(2017), Xu et al.
, they are more suited than convolution neural networks (CNNs) to handle the problem of extracting rotation-invariant features.
For this reason, the core idea of the proposed GraphCore method in this paper is to use the visual isometric invariant features (VIIFs) as the anomaly measurement features.
In the method using memory bank ( M ) as the AD paradigm, PatchCore (Roth et al.
(2022)) uses ResNet (He et al.
as the feature extractor.
However, since their features obtained by CNNs do not have rotation invariance (Dieleman et al.
, a large number of redundant features are stored in M .
Note that these redundant features maybe come from multiple rotation features of the same patch structure.
It will hence require a huge quantity of training data to ensure the high accuracy of the test set.
To avoid these redundant features, we propose VIIFs, which not only produce more robust visual features, but also dramatically lower the size of M and accelerate detection.
Based on the previous considerations, the goal of our work is to handle the cold start of the production line during the changeover.
As shown in Fig.
1(c), a new FSAD method, called GraphCore , is developed that employs a small number of normal samples to accomplish fast training and competitive AD accuracy performance of the new product.
On the one hand, by utilizing a small amount of data, we would rapidly train and accelerate the speed of anomaly inference.
On the other hand, because we directly train new product samples, adaptation and migration of anomalies from the old product to the new product do not occur.
Contributions.
In summary, the main contributions of this work are as follows:
• We present a feature-augmented method for FSAD in order to investigate the property of visual features generated by CNNs.
• We propose a novel anomaly detection model, GraphCore, add a new VIIF into the memory bank-based AD paradigm, which can drastically reduce the quantity of redundant visual features.
• The experimental results show that the proposed VIIFs are effective and can greatly enhance the FSAD performance on MVTec AD and MPDD .
Related Work.
Fewshot anomaly detection (FSAD) is an attractive research topic.
However, there are only a few of papers devoted to the industrial image FSAD.
Some works (Liznerski et al.
Pang et al.
; Ding et al.
experiment with fewshot abnormal images in the test set, which contradicts our assumptions where no abnormal images existed.
And others (Wu et al.
Huang et al.
(2022)) conduct experiments in a meta-learning setting.
This configuration has the disadvantage of requiring a high number of base class images and being incapable of addressing the shortage of data under cold-start conditions in industrial applications.
PatchCore (Roth et al.
SPADE (Cohen & Hoshen (2020)) and PaDiM (Defard et al. investigated AD performance on MVTec AD in a fewshot setting.
However, these approaches are not intended for changeoverbased fewshot settings, thus their performance cannot satisfy the requirements of manufacturing changeover.
In this research, we propose a feature augmentation method for FSAD that can rapidly finish the training of anomaly detection models with a small quantity of data and meet manufacturing changeover requirements.
A PPROACH
Problem Setting.
Fig.
1(c) outlines the formal definition of the problem setting for the proposed
FSAD.
Given a training set of only n normal samples, where n ≤ 8 , from a certain category.
At test time, given a normal or abnormal sample from a target category, the anomaly detection model should predict whether or not the image is anomalous and localize the anomaly region if the prediction result is anomalous.
Challenges.
For the FSAD proposed in Fig.
1(c), we attempt to detect anomalies in the test sample using only a small number of normal images as the training dataset.
The key challenges consist of:
(1) Each category’s training dataset contains only normal samples, i.e there are no annotations at the image or pixel level.
(2) There are few normal samples of the training set available.
In our proposed setting, there are less than 8 training samples.
Motivation.
In the realistic industrial image dataset (Bergmann et al.
; Jezek et al.
, the images under certain categories are extremely similar.
Most of them can be converted to one another with simple data augmentation, such as the meta nut (Fig. 2) and the screw (Fig. 6).
For instance, rotation augmentation can effectively provide a new screw dataset.
Consequently, when faced with the challenges stated in Section 2, our natural inclination is to acquire additional data through data augmentation.
Then, the feature memory bank (Fig. 4) can store more useful features.
[Figure]
2. A UGMENTATION +P ATCH C ORE
As a means of validating our insight, we have adapted PatchCore (Roth et al.
to our model.
We denote augmentation (rotation) with PatchCore as Aug.(R).
The architecture is depicted in detail in Fig.
Before extracting features from the ImageNet pretrained model, we augment the data (e.g., by rotating the data).
In training phase, the aim of training phase is to build up a memory bank, in which store the neighbourhood aware features from all of normal samples.
At test time, the test image are predicted as anomalies if at least one patch is anomalous, and pixel-level anomaly segmentation is computed via the score of each patch feature.
The feature memory construction method is shown in Algorithm 1.
In default, we set ResNet18 (He et al.
as the feature extraction model.
Conceptually, coreset sampling (Sener & Savarese (2018)) for memory bank aims to balance the size of memory bank with the performance of anomaly detection.
And the size of memory bank has a huge impact on the inference speed.
In Section 3.3, we discuss the effect of sampling rate in detail.
In testing phase, with the normal patches feature bank M , the image-level anomaly score s for the test image x test is computed by the maximum score s ∗ between the test image’s patch feature
P ( x test ) and its respective nearest neighbour m ∗ in M :
[Equation]
From Table 2 and Table 3, we can easily observed that the performance of Aug.(R) greatly outperforms the SOTA models under the proposed fewshot setting.
2. V ISION I SOMETRIC I NVARIANT F EATURE
In Section 2.1, we demonstrate heuristically that Augmentation+PatchCore outperforms SOTA models in the fewshot anomaly detection context proposed.
Essentially, the data augmentation method immediately incorporates the features of normal samples into the memory bank.
In other words, Augmentation+PatchCore improves the probability of locating a subset feature, allowing the anomaly score of the test image to be calculated with greater precision using Equ. (1) and Equ. (2).
Therefore, we question whether it is possible to extract the invariant representational features from a small number of normal samples and add them to the feature memory bank.
As demonstrated in Fig.
3, we propose a new model for feature extraction: vision isometric invariant graph neural network (VIIG).
The proposed model is motivated by Section 2 and tries to extract visual isometric invariant feature (VIIF) from each patch of the normal sample.
As previously stated, the majority of [Figure] [Figure] industrial visual anomaly detection datasets are transformable via rotation, translation, and flipping.
Thus, the isomorphism of GNN suited industrial visual anomaly detection excellently.
2. G RAPH R EPRESENTATION OF I MAGE
Fig.
4 shows that the feature extraction process of GraphCore.
Specifically, for a normal sample image with a size of H × W × 3 , we evenly separate it as an N patch.
In addition, each patch is transformed into a feature vector f i ∈ R D .
So we have the features F =
[ f 1 , f 2 , · · · , f N ] , where D is the feature dimension and i = 1 , 2 , · · · , N .
We view these features as unordered nodes
V = { v 1 , v 2 , · · · , v N } .
For certain each node v i , we denote the K nearest neighbours N ( v i ) and add an edge e ij directed from v j to v i for all v j ∈ N ( v i ) .
Hence, each patch of normal samples can be denoted as a graph G =
( V , E ) .
E refers all the edges of Graph G .
2. G RAPH F EATURE P ROCESSING
Fig.
4 shows the architecture of the proposed vision isometric invariant GNN.
To be specific, we set the feature extraction as GCN (Kipf & Welling (2017)).
We aggregate features for each node by exchanging information with its neighbour nodes.
In specific, the feature extraction operates as follows:
[Equation]
(3) where W aggregate and W update denote the weights of the aggregation and update operations.
Both of them can be optimized in an end-to-end manner.
Specifically, the aggregation operation for each node is calculated by aggregating neighboring nodes’ features:
[Equation]
[Equation]
where h is the node feature update function and g is the node feature aggregate feature function.
N ( f li ) denotes the set of neighbor nodes of f li at the l -th layer.
Specifically, we employ max-relative graph convolution (Li et al.
as our operator.
So g and h are defined as:
[Equation]
[Figure] [Figure]
In Equations 5 and 6, g ( · ) is a max-pooling vertex feature aggregator that aggregates the difference in features between node v i and all of its neighbors.
h ( · ) is an MLP layer with batch normalization and ReLU activation.
2. G RAPH C ORE A RCHITECTURE
Fig.
5 shows the whole architecture of GraphCore.
In the training phase, the most significant difference between GraphCore and Augmentation+PatchCore is the feature memory bank construction algorithm.
The feature construction algorithm is the same with Aug.(R) memory bank in Algorithm 1.
Note that we use vision isometric invariant GNN as feature extractor P without data augmentation.
In the testing phase, the computation of anomaly score s ∗ for GraphCore is highly similar to the one in Augmentation+PatchCore.
The only difference is the feature extraction method for each normal patch sample.
2. A U NIFIED V IEW OF A UGMENTATION +P ATCH C ORE AND G RAPH C ORE
Fig.
6 depicts a unified view of both Augmentation+PatchCore and GraphCore.
GraphCore is prompted by Augmentation+PatchCore to obtain the isometric invariant feature.
Therefore, GraphCore is able to improve the probability of locating a feature subset and it allows the anomaly score of a test image to be calculated most precisely and rapidly.
Table 1 shows the difference among
PatchCore, Augmentation+PatchCore and GraphCore in terms of architecture details.
[Table]
E XPERIMENT 3. E XPERIMENT SETTING
Datasets.
To demonstrate the generalization of our proposed method, we conduct experiments on three datasets, namely MVTec AD (Bergmann et al.
, MPDD (Jezek et al.
and
MVTec LOCO AD (Bergmann et al.
Competing Methods.
RegAD (Huang et al.
(2022)) is the SOTA FSAD method.
It works under meta-learning setting: aggregated training on multiple categories and adapting to unseen categories, [Figure] using fewshot unseen images as a support set.
However, our proposed fewshot setting utilizes only a few images as a training set, and not several categories.
Taking into account the fairness of the experiments, we reimplement the classical and SOTA approaches in the field of unsupervised anomaly detection, such as SPADE (Cohen & Hoshen (2020)), STPM (Wang et al.
, RD4AD (Deng & Li (2022)), CFA (Lee et al.
(2022)), and PatchCore (Roth et al.
, using the official source code for comparison under our fewshot setting.
PatchCore-1 is the result of our reimplementation with a 1% sampling rate, PatchCore-10 and PatchCore-25 are the results at 10% and 25% sampling rates, respectively, and RegAD-L is the RegAD experiment with our fewshot setting.
3. C OMPARISON WITH THE SOTA M ETHODS
The comparative findings between MVTec and MPDD are shown in Table 2.
Especially, the performance of RegAD under the meta-learning setting is also listed in the table.
In comparison to
SOTA models, GraphCore improves average AUC by 5.8%, 4.1%, 3.4%, and 1.6% on MVTec and by 25.5%, 22.0%, 16.9%, and 14.1% on MPDD for 1, 2, [Table] and 8-shot cases, respectively.
From
Fig.
7, it can be easily observed that GraphCore significantly outperform the SOTA approah at the image and pixel level from 1-shot to 8-shot.
As can be seen, the performance of GraphCore and
Augmentation+PatchCore surpass the other methods when using only a few samples (no more than 8 samples) for training.
[Table]
Considering that RegAD only shows detailed results of various categories above 2-shot, we only show the detailed results of 2-shot in the main text [Table] and the results of 1-shot, 4-shot [Table]
and 8-shot are in the appendix. As shown in Table 3, GraphCore outperforms all other baseline methods in 12 out of the 15 categories at the image level [Table] and outperforms all other baselines in 11 out of the 15 categories at the pixel level on MVTec AD.
Moreover, results in Table 4 show that GraphCore outperforms all other baselines in 5 out of the 6 categories at the image level, and outperforms all other baselines in all categories at the pixel level on MPDD.
3. A BLATION S TUDIES
Sampling Rate.
When demonstrated in Fig.
8, our technique significantly improves as the sample rate increases from 0.0001 to 0.001, after which the increase in sampling rate has a flattening effect [Table] [Table] on the performance gain.
In other words, as the sampling rate steadily increases, the performance of
GraphCore is insensitive to the sampling rate.
Nearest Neighbour.
In Fig.
8, the green color represents the performance of GraphCore’s 9 nearest neighbour search and the blue color represents the performance of GraphCore’s 3 nearest neighbour search.
As can be seen, increasing the number of neighbors from 3 to 9 greatly increases performance at the pixel level when the sampling rate is low [Table] but does not enhance performance at the image level.
As the sampling rate increases, the gain of the number of pixels’ neighbors approaches zero.
Augmentation Methods.
Fig.
9 demonstrates that the performance of PatchCore on MVTec AD and MPDD is relatively weak, but Aug.(R) demonstrates higher performance.
It demonstrates 0.
0.1 Sampling Rate 0.
A cc u r [Figure]
[Figure] [Figure]
heuristically that our enhancement to feature rotation is significantly effective.
Moreover, GraphCore outperforms Aug.(R) by a large margin, confirming our assumption that GraphCore can extract the isometric invariant feature from industrial-based anomaly detection images
3. V ISUALIZATION
Fig.
10 shows the visualization results obtained by our method on MVTec AD and MPDD with sampling rates of 0.01 and 1 shot, respectively.
Each column represents a different item type, and the four rows, from top to bottom, are the detection image, anomaly score map, anomaly map on detection image, and ground truth.
According to the results, our method can produce a satisfactory impact of anomaly localization on a variety of objects, indicating that it has strong generalizability even in the 1-shot case.
C ONCLUSION
In this study, we introduce a new approach, GraphCore, for industrial-based fewshot visual anomaly detection.
Initially, by investigating the CNN-generated feature space, we present a simple pipeline - Augmentation+PatchCore - for obtaining rotation-invariant features.
It turns out that this simple baseline can significantly improve anomaly detection performance.
We further propose GraphCore to capture the isometric invariant features of industrial normal samples.
It outperforms the SOTA models by a large margin using only a few normal samples ( ≤ 8 ) for training.
The majority of industrial datasets for anomaly detection possess isomorphism, which is a property perfectly suited to GraphCore.
We will continue to push the limits of industrial-based fewshot anomaly detection in the future.