Fast Adaptive Anomaly Detection

1 INTRODUCTION . Anomaly detection is an important area of study in the field of artificial intelligence . It has found utility in computer vision applications such as industrial inspection ( Bergmann et al. , 2019 ) and video surveillance ( Liu et al. , 2018 ; Zhao et al. , 2011 ; Nguyen & Meunier , 2019 ) , in the context of abuse prevention such as misinformation , fraud and network intrusion detection ( Zhang et al. , 2019 ; Bolton & Hand , 2002 ; Mukherjee et al. , 1994 ) , and others such as system health monitoring and fault detection ( Bao et al. , 2019 ; Purarjomandlangrudi et al. , 2014 ) . In this paper , we propose an approach for detecting anomaly in images , where we have carefully designed steps to handle some of the bigger issues that have prevented the deployment of image anomaly detection in the real-world . Image anomaly detection can generally be defined as the identification of abnormalities in a given image . An exact definition of abnormality in this case is elusive because abnormality can be derived from any unknown distribution outside of a normal population . Many studies have hence focused on modeling the normal population instead of learning irregularities , where the goal is to capture the shared concept among all of the normal data as one or several reference models . This process usually will require investing significant efforts in curating a large enough set of normal samples for each task , after which anomaly is detected as deviations from the reference model ( s ) ( An & Cho , 2015 ; Xu et al. , 2018 ) . Recent work from ( Sheynin et al. , 2021 ) provides algorithms that utilize only a few normal samples to train models from scratch . However , the models still have to be provisioned for each new task , which requires considerable human efforts and expertise , and thus lack the fast deployment criterion that is often time critical for real-world applications . In view of these challenges , our goals for this work are threefold . We are interested in designing an anomaly detection system that is capable of : ( G1 ) modeling the normal population while at the same time has a principled approach towards modeling the abnormalities ; ( G2 ) quickly adapting to a new task at inference time ; and ( G3 ) requiring only a few normal shots to update itself to the new task at hand . For ( G1 ) , we introduce the class of Energy Based Model ( EBM ) , which is an important family of generative models ( Zhao et al. , 2016 ; Du & Mordatch , 2019 ; Xie et al. , 2016 ) . EBMs have been shown to demonstrate superior capability on modeling data density and localizing anomaly ( Genc et al. , 2021 ) . For our purpose , the EBM we adopted learn to assign low energy to normal samples but high energy to abnormal samples . More importantly , the abnormal samples are generated with a procedure known as Langevin Dynamics ( LD ) ( Welling & Teh , 2011 ) , which , in its original form , starts with a noise image ( see App . Fig C ) and gradually samples from the distribution along the direction of lower energy . This lends itself gracefully to utilizing the generated intermediate samples as negative/abnormal . The LD procedure is then coupled with a contrastive divergence loss ( Hinton , 2002 ) that aims to maximize the energy differences between the normal and abnormal samples . To achieve ( G2 ) , we propose an adaptive sparse coding layer that is attached to the deep feature extractor in the EBM . The deep features are projected into a set of feature vectors along the spatial axes , after which each vector is forwarded to the sparse coding layer , where the dictionary is constructed with the features of a few normal samples of the given task . In essence , the input representation has been decomposed into a linear combination of normal features with the sparsity constraint imposed . The final energy score is measured by the distance between the original and the reconstructed features ( after the sparse coding layer ) . Under this scheme , the dictionary for a particular task is not obtained by learning , but instead is constructed by the feature representations of a few normal samples during inference . As a result , this simple “ plug-and-play ” trick allows the model to be adapted to novel tasks promptly without re-training . Further , we expect that the dictionary , which is formed by normal features , will not be able to explain the abnormal samples well , causing relatively high reconstruction error that lends itself for subsequent detection . As a bonus , a backward pass from the reconstruction error to the image is also additionally useful for localizing the abnormal regions . Towards ( G3 ) , we utilize meta learning ( Vilalta & Drissi , 2002 ; Finn et al. , 2017 ) to simulate the scenario of being given a new task with a few normal shots to update the dictionary , followed by training the EBM . This is accomplished by episodic training , where in each episode the model is adapted to a held back task that is given a few normal samples . To accelerate the EBM training , we introduce “ learning from inpainting ” , a simple yet effective strategy for synthesizing hard abnormal samples quicker by starting the LD procedure with a synthesized image that is simply a normal sample with a noise patch injected as opposed to a noise image that is traditionally what is used . We show the proposed framework is able to efficiently adapt to a novel task ( e.g. , a new object category or scenes from a new camera ) with a few normal samples on both industrial inspection and video surveillance tasks . We provide both qualitative and quantitative results to demonstrate that our method outperforms other adaptive frameworks and is comparable to methods that rely on large amount of normal samples . 2 BACKGROUNDS . We briefly introduce two key ingredients of the proposed method : EBMs and sparse coding . Energy-based Model In EBMs , the goal is to learn an energy function Eθ ( x ) : Rd → R which parametrizes the data density pθ ( x ) as : pθ ( x ) = exp ( −Eθ ( x ) ) ∫ x exp ( −Eθ ( x ) ) , ( 1 ) where θ is the parameter of the energy function and Zθ = ∫ x exp ( −Eθ ( x ) ) is the partition function . Approximating the true data distribution pdata ( x ) is equivalent to minimizing the expected negative log-likelihood function over the data distribution , defined by the loss function : LML = Ex∼pdata ( x ) [ − log pθ ( x ) ] = Ex∼pdata ( x ) [ Eθ ( x ) + logZθ ] . ( 2 ) As the computation of LML involves an intractable term Zθ , the common practice is to represent the gradient of LML as , ∇θLML = Ex+∼pdata ( x ) [ ∇θEθ ( x+ ) ] − Ex−∼pθ ( x ) [ ∇θEθ ( x− ) ] . ( 3 ) This objective is commonly referred to as the contrastive divergence ( Hinton , 2002 ) , which decreases the energy of positive data samples x+ from the true distribution ( normal samples in our use case ) and increases the energy of negative samples x− from the model pθ ( synthesized abnormal samples ) . In practice , the synthesized negative samples are achieved through Langevin dynamics ( Welling & Teh , 2011 ) , which samples along the direction of decreasing energy score , typically starting from a given noise image : x̃k = x̃k−1 − β 2 ∇xEθ ( x̃k−1 ) + ωk , ωk ∼ N ( 0 , βI ) , ( 4 ) where β is the step size . The synthesizing ability of EBMs enables generating abnormal samples to help in learning a more accurate data density , and is often touted as the one of the advantages of using an EBM . Sparse coding . Approximating a signal z ∈ Rd with the sparse linear combination over a dictionary D ∈ Rd×k can be expressed as : min α 1 2 ||z−Dα||22 + λ||α||1 , ( 5 ) where α is the sparse coefficients , with its sparsity ( l1 norm ) and λ is the weight of the sparsity constraint . Dα is a sparse approximation to the original signal z . In practice , finding the dictionary atoms and the sparse coefficients is usually formulated as an optimization problem . In this paper , we adopt Iterative Soft Thresholding Continuation ( ISTC ) ( Jiao et al. , 2017 ) to convert this optimization problem into linear operations with a non-linear shrinkage function , which allows sparse coding to be seamlessly integrated into the deep neural networks . To compute a sparse coefficient α , ISTC performs iterations of gradient steps on reconstruction ||z −Dα||2 and a proximal projection step to increase coefficient sparsity . Formally , initializing the coefficients at the first step α0 with all zeros , each step of ISTC refines the sparse code with descending values of λ from λmax to λ ? : each step of ISTC is expressed as : αn+1 = σ ( αn + D > ( z−Dαn ) , λn ) , with λn = λmax λmax λ ? −n/N , ( 6 ) where σ ( · , · ) here is a shrinkage function that truncates small values ( lower than λ ) of the coefficients to 0 to enforce sparsity , and can be easily implemented by a customized ReLU activation function : σ ( z , λ ) = sgn ( z ) ( max ( |z| − λ , 0 ) ) = sgn ( z ) ReLU ( |z| − λ ) . ( 7 ) 3 PROPOSED METHOD . The objective of our proposed method , when given an input image or video frame , is to output an anomaly score indicating how likely this input deviates from the normal , and additionally , a pixel map grounding abnormal regions . In this section , we describe the proposed fast adaptive anomaly detection framework in details . In Section 3.1 , we introduce the adaptive EBM which consists of a deep feature extractor followed by an adaptive sparse coding layer . From there , we further show that utilizing larger receptive field in the sparse coding could improve training robustness ( Section 3.1.1 ) , and applying smoothed shrinkage functions could help speed up convergence ( Section 3.1.2 ) . In Section 3.2 , we describe the episodic training regime on various anomaly detection tasks that mimics few-shot adaptation in the meta-testing stage while learning common knowledge across tasks . Finally , Instead of synthesizing negative samples ( anomaly ) directly from noise , we introduce a simple but effective “ learning from inpainting ” operation to accelerate the training in Section 3.3 . 3.1 ADAPTIVE ENERGY-BASED MODEL . An EBM is a form of generative model and it is widely used for modeling data density and sampling . While there has been recent work ( Genc et al. , 2021 ) applying EBM to anomaly detection , it still requires re-training for each new task . To efficiently adapt the EBM to novel tasks , we introduce an adaptive sparse coding layer which is conditioned on the dictionary constructed by the features of normal samples . Specifically , as illustrated in Fig 1 , given an input image , x ∈ R3×h×w , we first obtain the corresponding feature z ∈ Rd×h′×w′ from the deep feature extractor Ψ with parameters θ , so that z = Ψ ( x ; θ ) . All feature vectors along spatial axes of z are then sparsely decomposed through the sparse coding layer over a task-specific dictionary D ∈ Rd×Kh′w′ , which contains the features of K normal samples of the current task as shown in the Fig 1 ( a ) . Each feature vector of the normal sample feature is then directly used as an atom in the task dictionary . The decomposed coefficients are α = S ( z ; D ) , where α ∈ RKh′w′×h′×w′ and S denotes the iterative sparse decomposition process of ( 6 ) . By multiplying the coefficient α with the dictionary D , we obtain the reconstructed features z′ = Dα . The sparsity regularization to α is important , as it encourages input features to be reconstructed by simple combinations of dictionary atoms ( normal features ) , so that it would be difficult for features of abnormal samples to be well-approximated , therefore producing higher reconstruction errors that make it conducive for detecting anomalies . From here , the final energy score is formulated as the mean squared error ( MSE ) between the original and the reconstructed features : Eθ ( x ; D ) = MSE ( z , z′ ) = ||Ψ ( x ; θ ) −DS ( Ψ ( x ; θ ) ; D ) ||2 . ( 8 ) In effect , Eqn . 8 depicts a conditional EBM , which is conditioned on the task-specific D formed by normal features . In the following sections , we will discuss how to make the training of this adaptive structure more robust .

Summary: This paper proposes an image classification system, where a set of normal patterns are stored as a "dictionary," and the degree of deviation is used as the anomaly score.    The overall architecture seems to feature a conventional deep encoder and a pattern matching module that compares the encoded latent vector against the pattern dictionary. The pattern matching is done by solving the lasso regression problem.   The authors propose a certain online learning approach, following existing few-shot learning methods, and also a synthetic sample generation approach using random perturbation combined with a gradient method.