Fast Adaptive Anomaly Detection

1 INTRODUCTION . Anomaly detection is an important area of study in the field of artificial intelligence . It has found utility in computer vision applications such as industrial inspection ( Bergmann et al. , 2019 ) and video surveillance ( Liu et al. , 2018 ; Zhao et al. , 2011 ; Nguyen & Meunier , 2019 ) , in the context of abuse prevention such as misinformation , fraud and network intrusion detection ( Zhang et al. , 2019 ; Bolton & Hand , 2002 ; Mukherjee et al. , 1994 ) , and others such as system health monitoring and fault detection ( Bao et al. , 2019 ; Purarjomandlangrudi et al. , 2014 ) . In this paper , we propose an approach for detecting anomaly in images , where we have carefully designed steps to handle some of the bigger issues that have prevented the deployment of image anomaly detection in the real-world . Image anomaly detection can generally be defined as the identification of abnormalities in a given image . An exact definition of abnormality in this case is elusive because abnormality can be derived from any unknown distribution outside of a normal population . Many studies have hence focused on modeling the normal population instead of learning irregularities , where the goal is to capture the shared concept among all of the normal data as one or several reference models . This process usually will require investing significant efforts in curating a large enough set of normal samples for each task , after which anomaly is detected as deviations from the reference model ( s ) ( An & Cho , 2015 ; Xu et al. , 2018 ) . Recent work from ( Sheynin et al. , 2021 ) provides algorithms that utilize only a few normal samples to train models from scratch . However , the models still have to be provisioned for each new task , which requires considerable human efforts and expertise , and thus lack the fast deployment criterion that is often time critical for real-world applications . In view of these challenges , our goals for this work are threefold . We are interested in designing an anomaly detection system that is capable of : ( G1 ) modeling the normal population while at the same time has a principled approach towards modeling the abnormalities ; ( G2 ) quickly adapting to a new task at inference time ; and ( G3 ) requiring only a few normal shots to update itself to the new task at hand . For ( G1 ) , we introduce the class of Energy Based Model ( EBM ) , which is an important family of generative models ( Zhao et al. , 2016 ; Du & Mordatch , 2019 ; Xie et al. , 2016 ) . EBMs have been shown to demonstrate superior capability on modeling data density and localizing anomaly ( Genc et al. , 2021 ) . For our purpose , the EBM we adopted learn to assign low energy to normal samples but high energy to abnormal samples . More importantly , the abnormal samples are generated with a procedure known as Langevin Dynamics ( LD ) ( Welling & Teh , 2011 ) , which , in its original form , starts with a noise image ( see App . Fig C ) and gradually samples from the distribution along the direction of lower energy . This lends itself gracefully to utilizing the generated intermediate samples as negative/abnormal . The LD procedure is then coupled with a contrastive divergence loss ( Hinton , 2002 ) that aims to maximize the energy differences between the normal and abnormal samples . To achieve ( G2 ) , we propose an adaptive sparse coding layer that is attached to the deep feature extractor in the EBM . The deep features are projected into a set of feature vectors along the spatial axes , after which each vector is forwarded to the sparse coding layer , where the dictionary is constructed with the features of a few normal samples of the given task . In essence , the input representation has been decomposed into a linear combination of normal features with the sparsity constraint imposed . The final energy score is measured by the distance between the original and the reconstructed features ( after the sparse coding layer ) . Under this scheme , the dictionary for a particular task is not obtained by learning , but instead is constructed by the feature representations of a few normal samples during inference . As a result , this simple “ plug-and-play ” trick allows the model to be adapted to novel tasks promptly without re-training . Further , we expect that the dictionary , which is formed by normal features , will not be able to explain the abnormal samples well , causing relatively high reconstruction error that lends itself for subsequent detection . As a bonus , a backward pass from the reconstruction error to the image is also additionally useful for localizing the abnormal regions . Towards ( G3 ) , we utilize meta learning ( Vilalta & Drissi , 2002 ; Finn et al. , 2017 ) to simulate the scenario of being given a new task with a few normal shots to update the dictionary , followed by training the EBM . This is accomplished by episodic training , where in each episode the model is adapted to a held back task that is given a few normal samples . To accelerate the EBM training , we introduce “ learning from inpainting ” , a simple yet effective strategy for synthesizing hard abnormal samples quicker by starting the LD procedure with a synthesized image that is simply a normal sample with a noise patch injected as opposed to a noise image that is traditionally what is used . We show the proposed framework is able to efficiently adapt to a novel task ( e.g. , a new object category or scenes from a new camera ) with a few normal samples on both industrial inspection and video surveillance tasks . We provide both qualitative and quantitative results to demonstrate that our method outperforms other adaptive frameworks and is comparable to methods that rely on large amount of normal samples . 2 BACKGROUNDS . We briefly introduce two key ingredients of the proposed method : EBMs and sparse coding . Energy-based Model In EBMs , the goal is to learn an energy function Eθ ( x ) : Rd → R which parametrizes the data density pθ ( x ) as : pθ ( x ) = exp ( −Eθ ( x ) ) ∫ x exp ( −Eθ ( x ) ) , ( 1 ) where θ is the parameter of the energy function and Zθ = ∫ x exp ( −Eθ ( x ) ) is the partition function . Approximating the true data distribution pdata ( x ) is equivalent to minimizing the expected negative log-likelihood function over the data distribution , defined by the loss function : LML = Ex∼pdata ( x ) [ − log pθ ( x ) ] = Ex∼pdata ( x ) [ Eθ ( x ) + logZθ ] . ( 2 ) As the computation of LML involves an intractable term Zθ , the common practice is to represent the gradient of LML as , ∇θLML = Ex+∼pdata ( x ) [ ∇θEθ ( x+ ) ] − Ex−∼pθ ( x ) [ ∇θEθ ( x− ) ] . ( 3 ) This objective is commonly referred to as the contrastive divergence ( Hinton , 2002 ) , which decreases the energy of positive data samples x+ from the true distribution ( normal samples in our use case ) and increases the energy of negative samples x− from the model pθ ( synthesized abnormal samples ) . In practice , the synthesized negative samples are achieved through Langevin dynamics ( Welling & Teh , 2011 ) , which samples along the direction of decreasing energy score , typically starting from a given noise image : x̃k = x̃k−1 − β 2 ∇xEθ ( x̃k−1 ) + ωk , ωk ∼ N ( 0 , βI ) , ( 4 ) where β is the step size . The synthesizing ability of EBMs enables generating abnormal samples to help in learning a more accurate data density , and is often touted as the one of the advantages of using an EBM . Sparse coding . Approximating a signal z ∈ Rd with the sparse linear combination over a dictionary D ∈ Rd×k can be expressed as : min α 1 2 ||z−Dα||22 + λ||α||1 , ( 5 ) where α is the sparse coefficients , with its sparsity ( l1 norm ) and λ is the weight of the sparsity constraint . Dα is a sparse approximation to the original signal z . In practice , finding the dictionary atoms and the sparse coefficients is usually formulated as an optimization problem . In this paper , we adopt Iterative Soft Thresholding Continuation ( ISTC ) ( Jiao et al. , 2017 ) to convert this optimization problem into linear operations with a non-linear shrinkage function , which allows sparse coding to be seamlessly integrated into the deep neural networks . To compute a sparse coefficient α , ISTC performs iterations of gradient steps on reconstruction ||z −Dα||2 and a proximal projection step to increase coefficient sparsity . Formally , initializing the coefficients at the first step α0 with all zeros , each step of ISTC refines the sparse code with descending values of λ from λmax to λ ? : each step of ISTC is expressed as : αn+1 = σ ( αn + D > ( z−Dαn ) , λn ) , with λn = λmax λmax λ ? −n/N , ( 6 ) where σ ( · , · ) here is a shrinkage function that truncates small values ( lower than λ ) of the coefficients to 0 to enforce sparsity , and can be easily implemented by a customized ReLU activation function : σ ( z , λ ) = sgn ( z ) ( max ( |z| − λ , 0 ) ) = sgn ( z ) ReLU ( |z| − λ ) . ( 7 ) 3 PROPOSED METHOD . The objective of our proposed method , when given an input image or video frame , is to output an anomaly score indicating how likely this input deviates from the normal , and additionally , a pixel map grounding abnormal regions . In this section , we describe the proposed fast adaptive anomaly detection framework in details . In Section 3.1 , we introduce the adaptive EBM which consists of a deep feature extractor followed by an adaptive sparse coding layer . From there , we further show that utilizing larger receptive field in the sparse coding could improve training robustness ( Section 3.1.1 ) , and applying smoothed shrinkage functions could help speed up convergence ( Section 3.1.2 ) . In Section 3.2 , we describe the episodic training regime on various anomaly detection tasks that mimics few-shot adaptation in the meta-testing stage while learning common knowledge across tasks . Finally , Instead of synthesizing negative samples ( anomaly ) directly from noise , we introduce a simple but effective “ learning from inpainting ” operation to accelerate the training in Section 3.3 . 3.1 ADAPTIVE ENERGY-BASED MODEL . An EBM is a form of generative model and it is widely used for modeling data density and sampling . While there has been recent work ( Genc et al. , 2021 ) applying EBM to anomaly detection , it still requires re-training for each new task . To efficiently adapt the EBM to novel tasks , we introduce an adaptive sparse coding layer which is conditioned on the dictionary constructed by the features of normal samples . Specifically , as illustrated in Fig 1 , given an input image , x ∈ R3×h×w , we first obtain the corresponding feature z ∈ Rd×h′×w′ from the deep feature extractor Ψ with parameters θ , so that z = Ψ ( x ; θ ) . All feature vectors along spatial axes of z are then sparsely decomposed through the sparse coding layer over a task-specific dictionary D ∈ Rd×Kh′w′ , which contains the features of K normal samples of the current task as shown in the Fig 1 ( a ) . Each feature vector of the normal sample feature is then directly used as an atom in the task dictionary . The decomposed coefficients are α = S ( z ; D ) , where α ∈ RKh′w′×h′×w′ and S denotes the iterative sparse decomposition process of ( 6 ) . By multiplying the coefficient α with the dictionary D , we obtain the reconstructed features z′ = Dα . The sparsity regularization to α is important , as it encourages input features to be reconstructed by simple combinations of dictionary atoms ( normal features ) , so that it would be difficult for features of abnormal samples to be well-approximated , therefore producing higher reconstruction errors that make it conducive for detecting anomalies . From here , the final energy score is formulated as the mean squared error ( MSE ) between the original and the reconstructed features : Eθ ( x ; D ) = MSE ( z , z′ ) = ||Ψ ( x ; θ ) −DS ( Ψ ( x ; θ ) ; D ) ||2 . ( 8 ) In effect , Eqn . 8 depicts a conditional EBM , which is conditioned on the task-specific D formed by normal features . In the following sections , we will discuss how to make the training of this adaptive structure more robust .

Summary: The paper proposes a framework for anomaly detection and localization that allows fast adaptation to new tasks.  Specifically, the authors propose an energy-based model (EBM) with an adaptive sparse coding layer directly trained with normal features of a target task.  A meta-learning process is followed to extract common knowledge across tasks enabling few shots adaptation.  Shrinkage functions, sparse coding with large receptive fields, and learning by inpainting are introduced to improve and accelerate the EBM training.