  Recently, people tried to use a few anomalies for video anomaly detection
(VAD) instead of only normal data during the training process. A side effect of
data imbalance occurs when a few abnormal data face a vast number of normal
data. The latest VAD works use triplet loss or data re-sampling strategy to
lessen this problem. However, there is still no elaborately designed structure
for discriminative VAD with a few anomalies. In this paper, we propose a
DiscRiminative-gEnerative duAl Memory (DREAM) anomaly detection model to take
advantage of a few anomalies and solve data imbalance. We use two shallow
discriminators to tighten the normal feature distribution boundary along with a
generator for the next frame prediction. Further, we propose a dual memory
module to obtain a sparse feature representation in both normality and
abnormality space. As a result, DREAM not only solves the data imbalance
problem but also learn a reasonable feature space. Further theoretical analysis
shows that our DREAM also works for the unknown anomalies. Comparing with the
previous methods on UCSD Ped1, UCSD Ped2, CUHK Avenue, and ShanghaiTech, our
model outperforms all the baselines with no extra parameters. The ablation
study demonstrates the effectiveness of our dual memory module and
discriminative-generative network.
