Lucia MADDALENA et Alfredo PETROSINO : A self-organizing approach to background subtraction for visual surveillance applications. *IEEE Trans Image Process*, 17:1168-1177, 2008.

Mahshid MAJD et Reza SAFABAKHSH : Correlational convolutional lstm for human action recognition. *Neurocomputing*, 396:224-229, 2020.

Snehashis MAJHI, Srijan DAS et François BRÉMOND : Dam: Dissimilarity attention module for weakly-supervised video anomaly detection. In *2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)*, pages 1-8. IEEE, 2021.

Pierre-Etienne MARTIN, Jenny BENOIS-PINEAU, Renaud PÉTERI et Julien MORLIER : Fine grained sport action recognition with twin spatio-temporal convolutional neural networks: Application to table tennis. *Multimedia Tools and Applications*, 79:20429-20447, 2020.

Jonathan MASCI, Ueli MEIER, Dan CIREŞAN et Jürgen SCHMIDHUBER : Stacked convolutional auto-encoders for hierarchical feature extraction. In *International conference on artificial neural networks*, pages 52-59. Springer, 2011.

Dierck MATERN, Alexandru Paul CONDURACHE et Alfred MERTINS : Automated intrusion detection for video surveillance using conditional random fields. In *MVA*, pages 298-301, 2013.

MATHWORKS : https://fr.mathworks.com/discovery/lstm.html, 2022.

Warren S MCCULLOCH et Walter PITTS : A logical calculus of the ideas immanent in nervous activity. *The bulletin of mathematical biophysics*, 5(4):115-133, 1943.

Jefferson Ryan MEDEL et Andreas SAVAKIS : Anomaly detection in video using predictive convolutional long short-term memory networks. *arXiv e-prints*, pages arXiv:1612.2016.

Ramin MEHRAN, Alexis OYAMA et Mubarak SHAH : Abnormal crowd behavior detection using social force model. In *2009 IEEE conference on computer vision and pattern recognition*, pages 935-942. IEEE, 2009.

Lili MENG, Bo ZHAO, Bo CHANG, Gao HUANG, Wei SUN, Frederick TUNG et Leonid SIGAL : Interpretable spatio-temporal attention for video action recognition. In *Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops*, pages 0-0, 2019.

V MENON et K STEPHEN : Re learning memory guided normality for anomaly detection. In *ML Reproducibility Challenge 2020*, 2021. URL https://openreview.net/forum?id=vvLWTXkJ2Zv.

Gaurav MITTAL, Sushrutha LOCHARAM, Sreela SASI, Glenn R SHAFFER et Ajith K KUMAR : An efficient video enhancement method using LA*B* analysis. In *AVSS*, pages 66-66, 2006.

Bahram MOHAMMADI, Mahmood FATHY et Mohammad SABOKROU : Image/video deep anomaly detection: A survey. *arXiv preprint arXiv:2103.01739*, 2021.
R NAYAK, U C PATI et S K DAS : A comprehensive review on deep learning-based methods for video anomaly detection. *Image Vis Comput*, 106:104078, 2021.

Rashmiranjan NAYAK, Mohini Mohan BEHERA, Umesh Chandra PATI et Santos Kumar DAS : Video-based real-time intrusion detection system using deep-learning for smart city applications. *In ANTS*, pages 1–6, 2019.

Daniel NEIMARK, Omri BAR, Maya ZOHAR et Dotan ASSELMANN : Video transformer network. *In Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 3163–3172, 2021.

Trong-Nguyen NGUYEN et Jean MEUNIER : Anomaly detection in video sequence with appearance-motion correspondence. *In Proceedings of the IEEE/CVF international conference on computer vision*, pages 1273–1283, 2019.

Michael NIELSEN : http://neuralnetworksanddeeplearning.com, 2017. Accessed: 2022-11-20.

Hyeonwoo NOH, Seunghoon HONG et Bohyung HAN : Learning deconvolution network for semantic segmentation. *In Proceedings of the IEEE international conference on computer vision*, pages 1520–1528, 2015.

Genevieve ORR, Nici SCHRAUDOLPH et Fred CUMMINS : https://www.willamette.edu/~gorr/classes/cs449/momrate.html, 1999.

Yuqi OUYANG et Victor SANCHEZ : Video anomaly detection by estimating likelihood of representations. *In 2020 25th International Conference on Pattern Recognition (ICPR)*, pages 8984–8991. IEEE, 2021.

C PARK, M CHO, M LEE et S LEE : FastAno: Fast anomaly detection via spatio-temporal patch transformation. *In WACV*, pages 2249–2259, 2022.

H PARK, J NOH et B HAM : Learning memory-guided normality for anomaly detection. *In CVPR*, pages 14372–14381, 2020.

Luis PATINO, Jonathan BOYLE, James FERRYMAN, Jonas AUER, Julian PEGORARO, Roman PFLUGFELDER, Mertcan COKBAS, Janusz KONRAD, Prakash ISHWAR, Giulia SLAVIC et al. : PETS2021: Through-foliage detection and tracking challenge and evaluation. *In AVSS*, pages 1–10, 2021.

Toby PERRETT et Dima DAMEN : Ddirstm: dual-domain lstm for cross-dataset action recognition. *In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 7852–7861, 2019.

Oluwatoyin P POPOOLA et Kejun WANG : Video-based abnormal human behavior recognition—a review. *IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)*, 42(6):865–878, 2012.

Ning QIAN : On the momentum term in gradient descent learning algorithms. *Neural networks*, 12(1):145–151, 1999.

Zhaofan QIU, Ting YAO et Tao MEI : Learning spatio-temporal representation with pseudo-3d residual networks. *In proceedings of the IEEE International Conference on Computer Vision*, pages 5533–5541, 2017.
B RAMACHANDRA et M JONES : Street scene: A new dataset and evaluation protocol for video anomaly detection. *In WACV*, pages 2569–2578, 2020.

Bharathkumar RAMACHANDRA, Michael J JONES et Ranga Raju VATSAVAI : A survey of single-scene video anomaly detection. *IEEE Transactions on Pattern Analysis & Machine Intelligence*, 44(05):2293–2312, 2022.

Yunbo RAO et Leiting CHEN : A survey of video enhancement techniques. *J Inf Hiding Multimedia Signal Process*, 3:71–99, 2012.

Yunbo RAO, Zhongho CHEN, Ming-Ting SUN, Yu-Feng HSU et Zhengyou ZHANG : An effective night video enhancement algorithm. *In VCIP*, pages 1–4, 2011.

Mahdyar RAVANBAKHSH, Moin NABI, Enver SANGINETO, Lucio MARCENARO, Carlo REGAZZONI et Nicu SEBE : Abnormal event detection in videos using generative adversarial nets. *In ICIP*, pages 1577–1581, 2017.

Joseph REDMON et Ali FARHADI : YOLO9000: better, faster, stronger. *In CVPR*, pages 7263–7271, 2017.

Joseph REDMON et Ali FARHADI : Yolov3: An incremental improvement. *arXiv preprint arXiv:1804.02767*, 2018.

Shaoqing REN, Kaiming HE, Ross GIRSHICK et Jian SUN : Faster R-CNN: towards real-time object detection with region proposal networks. *In NIPS*, pages 91–99, 2015.

M RIBEIRO, AE LAZZARETTI et HS LOPES : A study of deep convolutional auto-encoders for anomaly detection in videos. *Pattern Recognit Lett*, 105:13–22, 2018.

N-C RISTEA, N MADAN, R T IONESCU, K NASROLLAHI, F S KHAN, T B MOESLUND et M SHAH : Self-supervised predictive convolutional attentive block for anomaly detection. *In CVPR*, pages 13576–13586, 2022.

Lionel ROBINAULT : *Détection et suivi d'objets mobiles par caméras fixes*. Habilitation à diriger des recherches, Université Lumière Lyon 2, juin 2021. URL https://hal.univ-lyon2.fr/tel-03274532.

Olaf RONNEBERGER, Philipp FISCHER et Thomas BROX : U-net: Convolutional networks for biomedical image segmentation. *In International Conference on Medical image computing and computer-assisted intervention*, pages 234–241. Springer, 2015.

Frank ROSENBLATT : *The perceptron, a perceiving and recognizing automaton Project Para*. Cornell Aeronautical Laboratory, 1957.

Lukas RUFF, Robert A VANDERMEULEN, Nico GÖRNITZ, Alexander BINDER, Emmanuel MÜLLER, Klaus-Robert MÜLLER et Marius KLOFT : Deep semi-supervised anomaly detection. *In International Conference on Learning Representations*, 2019.

David E RUMELHART, Geoffrey E HINTON et Ronald J WILLIAMS : Learning representations by back-propagating errors. *nature*, 323(6088):533–536, 1986.
Olga RUSSAKOVSKY, Jia DENG, Hao SU, Jonathan KRAUSE, Sanjeev SATHEESH, Sean MA, Zhiheng HUANG, Andrej KARPATHY, Aditya KHOSLA, Michael BERNSTEIN et al. : Imagenet large scale visual recognition challenge. *International journal of computer vision*, 115(3):211–252, 2015.

Mohammad SABOKROU, Mohammad KHALOOEI, Mahmood FATHY et Ehsan ADELI : Adversarially learned one-class classifier for novelty detection. *In Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 3379–3388, 2018.

Takaya SAITO et Marc REHMSMEIER : The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. *PLoS ONE*, 10 (3):e0118432, 2015.

Haşim SAK, Andrew SENIOR, Kanishka RAO et Françoise BEAUFAYS : Fast and accurate recurrent neural network acoustic models for speech recognition. *arXiv preprint arXiv:1507.06947*, 2015.

V SALIGRAMA, J KONRAD et P-M JODOIN : Video anomaly identification. *IEEE Signal Process Mag*, 27:18–33, 2010.

Kamal SEHAIRI, Fatima CHOUIREB et Jean MEUNIER : Comparative study of motion detection methods for video surveillance systems. *J Electron Imaging*, 26:023025, 2017.

Ling SHAO, Jungong HAN, Pushmeet KOHLI et Zhengyou ZHANG : *Computer vision and machine learning with RGB-D sensors*, volume 20. Springer, 2014.

Shikhar SHARMA, Ryan KIROS et Ruslan SALAKHUTDINOV : Action recognition using visual attention. *In International Conference on Learning Representations: Workshop*, 2016.

Xingjian SHI, Zhourong CHEN, Hao WANG, Dit-Yan YEUNG, Wai-Kin WONG et Wang-chun WOO : Convolutional lstm network: A machine learning approach for precipitation nowcasting. *Advances in neural information processing systems*, 28, 2015.

Yemin SHI, Yonghong TIAN, Yaowei WANG, Wei ZENG et Tiejun HUANG : Learning long-term dependencies for action recognition with a biologically-inspired deep network. *In Proceedings of the IEEE International Conference on Computer Vision*, pages 716–725, 2017.

Karen SIMONYAN et Andrew ZISSERMAN : Two-stream convolutional networks for action recognition in videos. *In Advances in neural information processing systems*, pages 568–576, 2014.

P SINHA et R RUSSELL : A perceptually based comparison of image similarity metrics. *Perception*, 40:1269–1281, 2011.

A A SODEMANN, M P ROSS et B J BORGHETTI : A review of anomaly detection in automated surveillance. *IEEE Trans Syst Man Cybern C Appl Rev*, 42:1257–1272, 2012.

Marina SOKOLOVA, Nathalie JAPKOWICZ et Stan SZPAKOWICZ : Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation. *In AJCAI*, pages 1015–1021, 2006.
Nitish SRIVASTAVA, Elman MANSIMOV et Ruslan SALAKHUDINOV: Unsupervised learning of video representations using lstms. *In International conference on machine learning*, pages 843–852. PMLR, 2015.

Chris STAUFFER et W Eric L GRIMSON: Adaptive background mixture models for real-time tracking. *In CVPR*, volume 2, pages 246–252, 1999.

Christoph STILLER et Janusz KONRAD: Estimating motion in image sequences. *IEEE Signal Process Mag*, 16(4):70–91, 1999.

Swathikiran SUDHAKARAN, Sergio ESCALERA et Oswald LANZ: Lsta: Long short-term attention for egocentric action recognition. *In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 9954–9963, 2019.

Swathikiran SUDHAKARAN, Sergio ESCALERA et Oswald LANZ: Gate-shift networks for video action recognition. *In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 1102–1111, 2020.

Waqas SULTANI, Chen CHEN et Mubarak SHAH: Real-world anomaly detection in surveillance videos. *In Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 6479–6488, 2018.

Che SUN, Yunde JIA, Hao SONG et Yuwei WU: Adversarial 3d convolutional autoencoder for abnormal event detection in videos. *IEEE Transactions on Multimedia*, 23:3292–3305, 2020.

Lin SUN, Kui JIA, Kevin CHEN, Dit-Yan YEUNG, Bertram E SHI et Silvio SAVARESE: Lattice long short-term memory for human action recognition. *In Proceedings of the IEEE international conference on computer vision*, pages 2147–2156, 2017.

S SZYMANOWICZ, J CHARLES et R CIPOLLA: Discrete neural representations for explainable anomaly detection. *In WACV*, pages 148–156, 2022.

Y TANG, L ZHAO, S ZHANG, C GONG, G LI et J YANG: Integrating prediction and reconstruction for anomaly detection. *Pattern Recognit Lett*, 129:123–130, 2020.

David THIRDE, Longzhen LI et F FERRYMAN: Overview of the PETS2006 challenge. *In PETS*, pages 47–50, 2006.

Yu TIAN, Guansong PANG, Yuanhong CHEN, Rajvinder SINGH, Johan W VERJANS et Gustavo CARNEIRO: Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. *In Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 4975–4986, 2021.

Tijmen TIELEMAN et Geoffrey HINTON: Rmsprop: Divide the gradient by a running average of its recent magnitude. coursera: Neural networks for machine learning. *COURSERA Neural Networks Mach. Learn*, 2012.

Du TRAN, Lubomir BOURDEV, Rob FERGUS, Lorenzo TORRESANI et Manohar PALURI: Learning spatiotemporal features with 3d convolutional networks. *In Proceedings of the IEEE international conference on computer vision*, pages 4489–4497, 2015.
Du TRAN, Heng WANG, Lorenzo TORRESANI, Jamie RAY, Yann LECUN et Manohar PALURI: A closer look at spatiotemporal convolutions for action recognition. *In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition*, pages 6450–6459, 2018.

Hanh TM TRAN et David HOGG: Anomaly detection using a convolutional winner-take-all autoencoder. *In Proceedings of the British Machine Vision Conference 2017*. BMVC, 2017.

Vassilios TSAKANIKAS et Tasos DAGIUKLAS: Video surveillance systems-current status and future trends. *Computers & Electrical Engineering*, 70:736–753, 2018.

Amin ULLAH, Jamil AHMAD, Khan MUHAMMAD, Muhammad SAJJAD et Sung Wook BAIK: Action recognition in video sequences using deep bi-directional lstm with cnn features. *IEEE access*, 6:1155–1166, 2017.

Antoine VACAVANT, Thierry CHATEAU, Alexis WILHELM et Laurent LEQUIEVRE: A benchmark dataset for outdoor foreground/background extraction. *In Computer Vision-ACCV 2012 Workshops: ACCV 2012 International Workshops, Daejeon, Korea, November 5-6, 2012, Revised Selected Papers, Part I 11*, pages 291–300. Springer, 2013.

Maria VALERA et Sergio A VELASTIN: Intelligent distributed surveillance systems: a review. *IEE Proc Vis Image Sign Process*, 152:192–204, 2005.

Leslie G VALIANT: A theory of the learnable. *Communications of the ACM*, 27(11):1134–1142, 1984.

Koen VAN DE SANDE, Theo GEVERS et Cees SNOEK: Evaluating color descriptors for object and scene recognition. *IEEE Trans Pattern Anal Mach Intell*, 32:1582–1596, 2009.

Gül VAROL, Ivan LAPTEV et Cordelia SCHMID: Long-term temporal convolutions for action recognition. *IEEE transactions on pattern analysis and machine intelligence*, 40 (6):1510–1517, 2017.

Ashish VASWANI, Noam SHAZEER, Niki PARMAR, Jakob USZKOREIT, Llion JONES, Aidan N GOMEZ, Łukasz KAISER et Illia POLOUKHIN: Attention is all you need. *Advances in neural information processing systems*, 30, 2017.

Julien A VIJVERBERG, Roel TM JANSSEN, Remco de ZWART et Peter HN de WITH: Perimeter-intrusion event classification for on-line detection using multiple instance learning solving temporal ambiguities. *In ICIP*, pages 2408–2412, 2014.

Julien A VIJVERBERG, Cornelis J KOELEMAN et Peter HN de WITH: Towards real-time and low-latency video object tracking by linking tracklets of incomplete detections. *In AVSS*, pages 300–305, 2013.

Julien A VIJVERBERG, Marijn JH LOOMANS, Cornelis J KOELEMAN et al.: Two novel motion-based algorithms for surveillance video analysis on embedded platforms. *In Proc. SPIE 7724, Real-Time Image and Video Processing*, volume 7724, page 77240I, 2010.
Michael VILLAMIZAR, Angel MARTÍNEZ-GONZÁLEZ, Olivier CANÉVET et Jean-Marc ODOBEZ: WatchNet: Efficient and depth-based network for people detection in video surveillance systems. In AVSS, pages 1–6, 2018.

Paul VIOLA et Michael JONES: Rapid object detection using a boosted cascade of simple features. In CVPR, volume 1, pages I–I, 2001.

Vinay VISHWAKARMA, Chittaranjan MANDAL et Shamik SURAL: Automatic detection of human fall in video. In International conference on pattern recognition and machine intelligence, pages 616–623. Springer, 2007.

J WANG, J JIAO et Y-H LIU: Self-supervised video representation learning by pace prediction. In ECCV, pages 504–521, 2020.

Limin WANG, Zhe WANG, Yuanjun XIONG et Yu QIAO: Cuhk&siat submission for thumos15 action recognition challenge. THUMOS Action Recognition challenge, pages 1–3, 2015a.

Limin WANG, Yuanjun XIONG, Zhe WANG et Yu QIAO: Towards good practices for very deep two-stream convnets. arXiv preprint arXiv:1507.02159, 2015b.

Limin WANG, Yuanjun XIONG, Zhe WANG, Yu QIAO, Dahua LIN, Xiaoou TANG et Luc Van GOOL: Temporal segment networks: Towards good practices for deep action recognition. In European conference on computer vision, pages 20–36. Springer, 2016a.

Wenguan WANG, Hongmei SONG, Shuyang ZHAO, Jianbing SHEN, Sanyuan ZHAO, Steven CH HOI et Haibin LING: Learning unsupervised video object segmentation through visual attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3064–3074, 2019.

Xuanhan WANG, Lianli GAO, Peng WANG, Xiaoshuai SUN et Xianglong LIU: Two-stream 3-d convnet fusion for action recognition in videos with arbitrary size and length. IEEE Transactions on Multimedia, 20(3):634–644, 2017.

Xue WANG, Sheng WANG et Daowei BI: Distributed visual-target-surveillance system in wireless sensor networks. IEEE Trans Syst Man Cybern B Cybern, 39:1134–1146, 2009.

Yasi WANG, Hongxun YAO et Sicheng ZHAO: Auto-encoder based dimensionality reduction. Neurocomputing, 184:232–242, 2016b.

WIKIPEDIA CONTRIBUTORS: Autoencoder — Wikipedia, the free encyclopedia, 2022. URL https://en.wikipedia.org/w/index.php?title=Autoencoder&oldid=1120107812. [Online; accessed 23-December-2022].

Christopher Richard WREN, Ali AZARBAYEJANI, Trevor DARRELL et Alex Paul PENTLAND: Pfinder: Real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell, 19:780–785, 1997.

Zuxuan WU, Caiming XIONG, Yu-Gang JIANG et Larry S DAVIS: Liteeval: A coarse-to-fine framework for resource efficient video recognition. Advances in Neural Information Processing Systems, 32, 2019a.
Zuxuan WU, Caiming XIONG, Chih-Yao MA, Richard SOCHER et Larry S DAVIS : Adaframe: Adaptive frame selection for fast video recognition. *In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 1278–1287, 2019b.

Saining XIE, Chen SUN, Jonathan HUANG, Zhuowen TU et Kevin MURPHY : Re-thinking spatiotemporal feature learning for video understanding. *arXiv preprint arXiv:1712.04851*, 1(2):5, 2017a.

Saining XIE, Chen SUN, Jonathan HUANG, Zhuowen TU et Kevin MURPHY : Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. *In Proceedings of the European conference on computer vision (ECCV)*, pages 305–321, 2018.

Xuemei XIE, Chenye WANG, Shu CHEN, Guangming SHI et Zhifu ZHAO : Real-time illegal parking detection system based on deep learning. *In ICDLT*, pages 23–27, 2017b.

Dan XU, Elisa RICCI, Yan YAN, Jingkuan SONG et Nicu SEBE : Learning deep representations of appearance and motion for anomalous event detection. *arXiv preprint arXiv:1510.01553*, 2015a.

Kelvin XU, Jimmy BA, Ryan KIROS, Kyunghyun CHO, Aaron COURVILLE, Ruslan SALAKHUDINOV, Rich ZEMEL et Yoshua BENGIO : Show, attend and tell: Neural image caption generation with visual attention. *In International conference on machine learning*, pages 2048–2057. PMLR, 2015b.

Yong XU, Jixiang DONG, Bob ZHANG et Daoyun XU : Background modeling methods in video analysis: A review and comparative evaluation. *CAAI Trans Intell Technol*, 1:43–60, 2016.

Y YAO, C LIU, D LUO, Y ZHOU et Q YE : Video playback rate perception for self-supervised spatio-temporal representation learning. *In CVPR*, pages 6548–6557, 2020.

Mu YE, X PENG, W GAN, W WU et Y QIAO : AnoPCN: Video anomaly detection via deep predictive coding network. *In ACM Int Conf Multimed*, pages 1805–1813, 2019.

W YONGHUI, Mike SCHUSTER, Zhifeng CHEN, Quoc V LE, Mohammad NOROUZI, Wolfgang MACHEREY, Maxim KRIKUN, Yuan CAO, Qin GAO, Klaus MACHEREY et al. : Bridging the gap between human and machine translation. *arXiv preprint arXiv:1609.08144*, 2016.

Guang YU, Siqi WANG, Zhiping CAI, En ZHU, Chuanfu XU, Jianping YIN et Marius KLOFT : Cloze test helps: Effective video anomaly detection via learning to complete video events. *In Proceedings of the 28th ACM International Conference on Multimedia*, pages 583–591, 2020.

Joe YUE-HEI NG, Matthew HAUSKNECHT, Sudheendra VIJAYANARASIMHAN, Oriol VINYALS, Rajat MONGA et George TODERICI : Beyond short snippets: Deep networks for video classification. *In Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 4694–4702, 2015.
Sergey ZAGORUYKO et Nikos KOMODAKIS : Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. *arXiv preprint arXiv:1612.03928*, 2016.

Matthew D ZEILER : Adadelta: an adaptive learning rate method. *arXiv preprint arXiv:1212.5701*, 2012.

Matthew D ZEILER, Dilip KRISHNAN, Graham W TAYLOR et Rob FERGUS : Deconvolutional networks. *In CVPR*, pages 2528–2535, 2010.

Gloria ZEN et Elisa RICCI : Earth mover’s prototypes: A convex learning approach for discovering activity patterns in dynamic scenes. *In CVPR 2011*, pages 3225–3232.
IEEE, 2011.

Yanhong ZENG, Jianlong FU et Hongyang CHAO : Learning joint spatial-temporal transformations for video inpainting. *In European Conference on Computer Vision*, pages 528–543. Springer, 2020.

R ZHANG : Making convolutional networks shift-invariant again. *In ICML*, pages 7324–7334, 2019.

Han ZHAO et Xinyu JIN : Human action recognition based on improved fusion attention cnn and rnn. *In 2020 5th International Conference on Computational Intelligence and Applications (ICCIA)*, pages 108–112. IEEE, 2020.

Yiru ZHAO, Bing DENG, Chen SHEN, Yao LIU, Hongtao LU et Xian-Sheng HUA : Spatio-temporal autoencoder for video anomaly detection. *In ACM Int Conf Multimed*, pages 1933–1941, 2017.

Jia-Xing ZHONG, Nannan LI, Weijie KONG, Shan LIU, Thomas H LI et Ge LI : Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. *In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 1237–1246, 2019.

Bolei ZHOU, Alex ANDONIAN, Aude OLIVA et Antonio TORRALBA : Temporal relational reasoning in videos. *In Proceedings of the European conference on computer vision (ECCV)*, pages 803–818, 2018.

Joey Tianyi ZHOU, Le ZHANG, Zhiwen FANG, Jiawei DU, Xi PENG et Yang XIAO : Attention-driven loss for anomaly detection in video surveillance. *IEEE transactions on circuits and systems for video technology*, 30(12):4639–4647, 2019.

Linchao ZHU, Du TRAN, Laura SEVILLA-LARA, Yi YANG, Matt FEISZLI et Heng WANG : Faster recurrent networks for efficient video classification. *In Proceedings of the AAAI Conference on Artificial Intelligence*, volume 34, pages 13098–13105, 2020.

Zoran ZIVKOVIC : Improved adaptive gaussian mixture model for background subtraction. *In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004.*, volume 2, pages 28–31. IEEE, 2004.

Bo ZONG, Qi SONG, Martin Renqiang MIN, Wei CHENG, Cristian LUMEZANU, Daeki CHO et Haifeng CHEN : Deep autoencoding gaussian mixture model for unsupervised anomaly detection. *In International conference on learning representations*, 2018.
Zhengxia ZOU, Zhenwei SHI, Yuhong GUO et Jieping YE: Object detection in 20 years: A survey. *arXiv:1905.05055*, 2019.
List of Figures

<table>
  <tbody>
    <tr>
      <td>1</td>
      <td>Venn diagram to illustrate the taxonomy of tasks in anomaly detection.</td>
      <td>1</td>
    </tr>
    <tr>
      <td>1.1</td>
      <td>Issues with the infrared camera: low visibility in rain (left) and insect on camera (right).</td>
      <td>7</td>
    </tr>
    <tr>
      <td>1.2</td>
      <td>Data acquisition on a site by color (left) and thermal camera (right). Each camera captures two scenes at the same instant, one during day (top row) and the other during night (bottom row).</td>
      <td>8</td>
    </tr>
    <tr>
      <td>1.3</td>
      <td>Effect of perspective: three people (with blue bounding boxes) in the same image with different apparent size. Image source: ShanghaiTech dataset (Luo et al., 2017a).</td>
      <td>11</td>
    </tr>
    <tr>
      <td>1.4</td>
      <td>Camouflaged image with three persons to detect. Source: (Robinault, 2021).</td>
      <td>12</td>
    </tr>
    <tr>
      <td>1.5</td>
      <td>Example frames from the UCSD Ped2 dataset. First row contains frames without any anomaly, with increasing crowd volume (left to right order). Second row shows anomalous frames with anomalies highlighted in red bounding boxes.</td>
      <td>14</td>
    </tr>
    <tr>
      <td>1.6</td>
      <td>Example frames from the CUHK Avenue dataset. First row contains frames without any anomaly but with different scene dynamics. Second row shows the following anomalies in red bounding boxes (left to right order): bicycle, throwing bag and running person, loitering and person walking in wrong direction.</td>
      <td>15</td>
    </tr>
    <tr>
      <td>1.7</td>
      <td>Some frames from the ShanghaiTech dataset, each from different scene. First row contains frames without any anomaly but with different scene dynamics. Second row depicts the following anomalies in red bounding boxes (left to right order): chasing, small truck, brawling and two people jumping.</td>
      <td>15</td>
    </tr>
    <tr>
      <td>1.8</td>
      <td>Example frames drawn from i-LIDS sterile zone dataset with various intrusion (intruders in red boxes) and non-intrusion frames. The color and black and white frames belong to view 1 and view 2 respectively. The time of the day in four columns are dawn, dusk, night, and day. The distractions (in blue boxes) in non-intrusion row (left to right order) are a rabbit, birds with rain, fox and an insect on camera respectively. In intrusion row (left to right order), we have two intruders, intruder with ladder during snow, log rolling intruder and crawling intruder respectively.</td>
      <td>17</td>
    </tr>
    <tr>
      <td>2.1</td>
      <td>Pictorial representation of a feed-forward neural network. Source: (Nielsen, 2017).</td>
      <td>24</td>
    </tr>
    <tr>
      <td>2.2</td>
      <td>Common activation functions.</td>
      <td>25</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>2.3</td>
      <td>SGD optimization with and without momentum. On the left, without momentum the optimization process bounces between the ravine's slopes whereas, on the right, with momentum, optimization is smoother. Source: (Orr et al., 1999).</td>
      <td>27</td>
    </tr>
    <tr>
      <td>2.4</td>
      <td>Convolution of an image <i>I</i> with kernel <i>K</i> using stride 1 × 1.</td>
      <td>28</td>
    </tr>
    <tr>
      <td>2.5</td>
      <td>Examples of max pooling and average pooling operations with filter size and stride as two.</td>
      <td>29</td>
    </tr>
    <tr>
      <td>2.6</td>
      <td>Pictorial representation of a simple RNN and an LSTM. Source: (Math-Works, 2022).</td>
      <td>30</td>
    </tr>
    <tr>
      <td>2.7</td>
      <td>Classic autoencoder architecture. Source: (Wikipedia contributors, 2022).</td>
      <td>32</td>
    </tr>
    <tr>
      <td>2.8</td>
      <td>Schematic representation of pooling, unpooling, convolution and deconvolution. Source: (Noh et al., 2015).</td>
      <td>33</td>
    </tr>
    <tr>
      <td>2.9</td>
      <td>Illustrating Generative adversarial network (GAN). Source: (Hayes et al., 2017).</td>
      <td>34</td>
    </tr>
    <tr>
      <td>2.10</td>
      <td>Illustrative example of 2D CNN for spatial feature extraction and an LSTM to model this spatial information over time, applied to action recognition task.</td>
      <td>35</td>
    </tr>
    <tr>
      <td>2.11</td>
      <td>Illustration of Two-stream model for video action recognition proposed by Simonyan et Zisserman (2014).</td>
      <td>36</td>
    </tr>
    <tr>
      <td>2.12</td>
      <td>Comparison of 2D convolution (left image) and 3D convolution (right image) operations. Figure reproduced from (Tran et al., 2015).</td>
      <td>38</td>
    </tr>
    <tr>
      <td>2.13</td>
      <td>Illustration of 3D CNN for video action recognition.</td>
      <td>38</td>
    </tr>
    <tr>
      <td>2.14</td>
      <td>The architecture of the convolutional autoencoder proposed in Hasan et al. (2016) for video anomaly detection.</td>
      <td>44</td>
    </tr>
    <tr>
      <td>2.15</td>
      <td>The future frame prediction architecture proposed by Liu et al. (2018) for video anomaly detection.</td>
      <td>45</td>
    </tr>
    <tr>
      <td>2.16</td>
      <td>Comparison of VAD tasks: reconstruction (left), frame prediction (middle) and self-supervised: video event completion (right) (Yu et al., 2020). DNN refers to a deep neural network and video event is the VCO input.</td>
      <td>46</td>
    </tr>
    <tr>
      <td>2.17</td>
      <td>One-class classification VAD proposed by Sabokrou et al. (2018). <i>R</i> and <i>D</i> are generator and discriminator modules of the adversarially learned GAN. <i>X</i>, <i>X̃</i>, <i>Z</i> and <i>X'</i> refers to input, input with noise, latent space and reconstruction respectively, where target class 1 represents normal (non-anomaly) class.</td>
      <td>47</td>
    </tr>
    <tr>
      <td>2.18</td>
      <td>Network architecture proposed by Zhao et al. (2017) for reconstructing present frames and predicting future frames.</td>
      <td>48</td>
    </tr>
    <tr>
      <td>2.19</td>
      <td>Autoencoder based VAD work of Astrid et al. (2021a) with normal input and pseudo anomaly input, where probability <i>p</i> regulates quantity of pseudo anomalous input.</td>
      <td>49</td>
    </tr>
    <tr>
      <td>2.20</td>
      <td>Proposed schema of MemAE (Gong et al., 2019). The encoder takes the input <i>x</i> to produce encoding <i>z</i>, which is taken as a query by the memory addressing unit to obtain the soft addressing weights <i>w</i>. The memory slots can be used to model the whole encoding or the features on one pixel (as shown in the figure). The updated encoding <i>ẑ</i> passes via decoder to produce the reconstruction <i>x̂</i>.</td>
      <td>51</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>2.21</td>
      <td>Typical pipeline of a PIDS, composed of an optional offline training (left part) and an online detection (middle part). An illustrative example of different steps in online detection is shown in right part of the figure (best viewed in color).</td>
      <td>54</td>
    </tr>
    <tr>
      <td>2.22</td>
      <td>Pre-processing proposed by Buch et Velastin (2014), applied on the i-LIDS dataset. After normalization of video frame, patches of 16 × 16 pixels with 20% overlap are constructed on the region of interest, i.e., ground and fence. These patches are used as input for their PIDS instead of the whole video frame.</td>
      <td>55</td>
    </tr>
    <tr>
      <td>2.23</td>
      <td>Moving object detection using background subtraction proposed by Kim et al. (2018b). The first column represents frames from a thermal and color camera respectively, with objects of interest in white bounding boxes. The second column demonstrates the output after background subtraction and image binarization, where white regions represent detected moving objects.</td>
      <td>57</td>
    </tr>
    <tr>
      <td>2.24</td>
      <td>Examples of incorrect detections using background subtraction method proposed by Vijverberg et al. (2013).</td>
      <td>58</td>
    </tr>
    <tr>
      <td>2.25</td>
      <td>Moving object classification architecture proposed by Kim et al. (2018b). Input frame of size 224 × 224 × 3 goes through a series of 2D convolutions, local response normalization and max-pooling units, followed by fully connected layers to finally give predictions of length ten, where each unit depicts probability of a particular class from a list of human (intruder) and wild animal classes.</td>
      <td>58</td>
    </tr>
    <tr>
      <td>2.26</td>
      <td>Working illustration of the PIDS proposed by Nayak et al. (2019) on CAVIAR (Crowley et al., 2005) (first row) and a private dataset (Nayak et al., 2019) (second row). The first column presents video frames from two different sites with green zones depicting the areas to protect. In the second column, potential intruder objects detected by YOLO detector are in yellow bounding boxes and since some of them are inside the protected zone, the alarm is raised (shown with red zone).</td>
      <td>61</td>
    </tr>
    <tr>
      <td>3.1</td>
      <td>Video with n frames of height H, width W and channels D = 1. Two objects, shown in orange and green bounding boxes, are defined as o<sub>1</sub> = ({b<sub>1,1</sub>, b<sub>1,2</sub>, b<sub>1,3</sub>}, c<sub>1</sub>) and o<sub>2</sub> = ({b<sub>2,3</sub>}, c<sub>2</sub>), where c<sub>1</sub>, c<sub>2</sub> ∈ C are the object classes. Here, {b<sub>1,1</sub>, b<sub>1,2</sub>, b<sub>1,3</sub>} are bounding boxes of object 1 on first three frames and {b<sub>2,3</sub>} represents bounding box of object 2 at frame 3.</td>
      <td>71</td>
    </tr>
    <tr>
      <td>3.2</td>
      <td>Illustration of an intrusion event caused by a single object. Video with n frames, where S<sub>na</sub> is shown with yellow surface and object with a green bounding box plus center. The object causes an intrusion event for four frames from frame I<sub>4</sub> to I<sub>7</sub>, colored in red.</td>
      <td>72</td>
    </tr>
    <tr>
      <td>3.3</td>
      <td>Illustration of intrusion events of the video and intrusion intervals. Objects o<sub>1</sub>, o<sub>2</sub> and o<sub>3</sub> cause intrusion events I<sub>E</sub>(o<sub>1</sub>), I<sub>E</sub>(o<sub>2</sub>) and I<sub>E</sub>(o<sub>3</sub>), marked with value 1. I<sub>E</sub>(V) is collection of all the frames with value 1. Two intrusion intervals I<sub>I1</sub> and I<sub>I2</sub> are shown in red intervals.</td>
      <td>73</td>
    </tr>
  </tbody>
</table>
<table>
  <tr>
    <td>3.4</td>
    <td>Illustration of i-LIDS evaluation protocol, highlighting its drawback on an intrusion example starting at 9<sup>th</sup> second. Since no alarm has been raised in the first 10 seconds of the intrusion, a false negative (FN) is counted. Following alarms, at 22<sup>nd</sup> and 37<sup>th</sup> second, are marked as false positive (FP) because they do not occur within 10 seconds from the beginning of the intrusion.</td>
    <td>75</td>
  </tr>
  <tr>
    <td>3.5</td>
    <td>The top subfigure is an illustration of the definitions of edge-level evaluation terms on ground truth, with time in abscissa and non-intrusion (0) and intrusion (1) class for each frame of the video in ordinate. The next two subfigures represent examples of alarms and possible outcomes (TP, FP, FN) for two different PIDSs evaluated by the edge-level protocol.</td>
    <td>76</td>
  </tr>
  <tr>
    <td>3.6</td>
    <td>Illustration of ROC (left sub figure) and PR curve (right sub figure). Each point in dotted lines represent values ( (TPR, FPR) for ROC and (Precision, Recall) for PR) corresponding to a threshold. The area under the ROC or PR curve, i.e., AUROC or AUPR is depicted with colored areas.</td>
    <td>80</td>
  </tr>
  <tr>
    <td>4.1</td>
    <td>Learning normality: A sliding window (in yellow) extracts video clips of fixed length. Each video clip is fed to S3DCAE, which reconstructs it. The mean squared error (MSE) between input and reconstructed clip is backpropagated to update learnable parameters of the autoencoder.</td>
    <td>85</td>
  </tr>
  <tr>
    <td>4.2</td>
    <td>S3DCAE architecture for PID with encoder and decoder. Conv3D and De-conv3D refer to 3D convolution and 3D deconvolution respectively, where s is stride, and the number that follows - denotes number of filters. The rightmost column refers to the data shape (channels, temporal length, height, width).</td>
    <td>86</td>
  </tr>
  <tr>
    <td>4.3</td>
    <td>Detecting intrusion using AE-Adapt: an input video clip is reconstructed after going through the S3DCAE. The reconstruction error is calculated between these clips and fed to a moving z-score module, the output of which is thresholded to raise an intrusion alarm.</td>
    <td>86</td>
  </tr>
  <tr>
    <td>4.4</td>
    <td>Example of per-frame predictions of the three methods on portions of video taken from i-LIDS test dataset. The intrusion intervals (ground truth) are shown in light red strips, the abscissa represents frames and ordinate shows prediction, where 1 signifies intrusion and 0 otherwise.</td>
    <td>90</td>
  </tr>
  <tr>
    <td>4.5</td>
    <td>The results on view 1 of i-LIDS test set for three methods: GOFPID (Barthélemy, 2022) (in green), Nayak et al. (2019) (in blue) and our method (in orange). The abscissa represents three evaluation protocols: frame-level (FL) (on left subfigure), i-LIDS and edge-level (EL) (on right subfigure). i-LIDS is evaluated by default for 10 s whereas, for EL, we show results for 1, 2 and 10 s. The ordinate represents values of three metrics: precision, recall and F<sub>1</sub> score.</td>
    <td>91</td>
  </tr>
  <tr>
    <td>4.6</td>
    <td>The results on view 2 of i-LIDS test set for three methods: GOFPID (Barthélemy, 2022) (in green), Nayak et al. (2019) (in blue) and our method (in orange). The abscissa represents three evaluation protocols: frame-level (FL) (on left subfigure), i-LIDS and edge-level (EL) (on right subfigure). i-LIDS is evaluated by default for 10 s whereas, for EL, we show results for 1, 2 and 10 s. The ordinate represents values of three metrics: precision, recall and F<sub>1</sub> score.</td>
    <td>93</td>
  </tr>
</table>
<table>
  <tbody>
    <tr>
      <td>4.7</td>
      <td>Qualitative comparison of thresholding strategies on a video, with ground truth (GT) in blue, thresholds in green and alarms in red. Fixed threshold on reconstruction error of S3DCAE (top); fixed threshold on min-max (MM) rescaled reconstruction error (middle); adaptive threshold by moving z-score (MZ) of reconstruction error (bottom).</td>
      <td>94</td>
    </tr>
    <tr>
      <td>4.8</td>
      <td>AE-Adapt working illustration on a portion of video of i-LIDS test set. In the curve, reconstruction error (RE) is shown in black, threshold in green and alarm in red. GT stands for ground truth per frame, where blue region indicates intrusion. Some illustrative frames are shown above and below the plot, where the blue bounding box shows intruder.</td>
      <td>96</td>
    </tr>
    <tr>
      <td>5.1</td>
      <td>Overall schema of the proposed LUSS-AE method.</td>
      <td>101</td>
    </tr>
    <tr>
      <td>5.2</td>
      <td>Overall schema of the proposed LUSS-AE method during testing. A window of T + 1 consecutive frames is drawn sequentially from the test video. The first T frames from this window are the input of the system and the output for each task is computed on it. The anomaly score is determined for each task, i.e., A<sub>VCR</sub>, A<sub>FFP</sub> and A<sub>PRP</sub>, and the final anomaly score is their weighted sum.</td>
      <td>105</td>
    </tr>
    <tr>
      <td>5.3</td>
      <td>Autoencoder architecture for VAD with encoder and decoder. Conv3D and Deconv3D refers to 3D convolution and 3D deconvolution respectively, and the number that follows them, denotes amount of filters. The rightmost column refers to the data shape (channels or filters, temporal length, frame height, frame width).</td>
      <td>107</td>
    </tr>
    <tr>
      <td>5.4</td>
      <td>Architecture of FFP head i.e., H<sub>FFP</sub>, for VAD task. Conv3D refers to 3D convolution, where k and s are kernel size and stride respectively. The rightmost column refers to the data shape.</td>
      <td>107</td>
    </tr>
    <tr>
      <td>5.5</td>
      <td>Architecture of PRP head i.e., H<sub>PRP</sub>, for VAD task. The different layers are depicted in middle column with violet color and the number that follows them, denotes amount of filters or units. Kernel size and stride are denoted by k and s, wherever applicable. The rightmost column refers to the data shape (channels, temporal length, frame height, frame width).</td>
      <td>108</td>
    </tr>
    <tr>
      <td>5.6</td>
      <td>Architecture of FFP head i.e., H<sub>FFP</sub>, for perimeter intrusion detection. Conv3D refers to 3D convolution, where k and s are kernel size and stride respectively. The rightmost column refers to the data shape (channels, temporal length, frame height, frame width).</td>
      <td>109</td>
    </tr>
    <tr>
      <td>5.7</td>
      <td>Architecture of PRP head i.e., H<sub>PRP</sub>, for perimeter intrusion detection. The different layers are depicted in middle column with violet color. Kernel size and stride are denoted by k and s, wherever applicable. The rightmost column refers to the data shape.</td>
      <td>109</td>
    </tr>
    <tr>
      <td>5.8</td>
      <td>LUSS-AE working illustration on video 11_0176 of Shanghai test set. Anomaly scores (A<sub>VCR</sub>, A<sub>FFP</sub>, A<sub>PRP</sub>, A) are plotted per video frame; red regions depict anomalous events and some illustrative frames are shown above the plot, where the yellow and red bounding boxes exhibit objects of interest and anomalies respectively.</td>
      <td>112</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>5.9</td>
      <td>LUSS-AE working illustration on video 05_0023 of Shanghai test set. Anomaly scores ($A_{VCR}$, $A_{FFP}$, $A_{PRP}$, A) are plotted per video frame; red regions depict ground truth anomalous events, and some illustrative frames are shown above the plot, where the yellow and red bounding boxes exhibit objects of interest and anomalies respectively.</td>
      <td>112</td>
    </tr>
    <tr>
      <td>5.10</td>
      <td>Illustration of different error measures on a test frame of the Avenue dataset. From left to right: actual frame (ground truth), predicted frame, error frame for MSE, for PSE and for BPE. (Best viewed in color)</td>
      <td>115</td>
    </tr>
    <tr>
      <td>A.1</td>
      <td>Example frames from the Sheeps dataset without intrusions. The images show video frames from different time of the day. The sheeps present are moving in random manner. Also, observe the sheeps at the top-right end of the third image.</td>
      <td>128</td>
    </tr>
    <tr>
      <td>A.2</td>
      <td>Example frames from the Sheeps dataset with intrusions shown in red- bounding boxes. The images are shown in increasing level of intrusion detection difficulty.</td>
      <td>128</td>
    </tr>
  </tbody>
</table>
List of Tables

<table><tr><td>1.1</td><td>Description of video anomaly detection datasets.</td><td>16</td></tr><tr><td>1.2</td><td>i-LIDS Sterile Zone dataset description.</td><td>18</td></tr><tr><td>2.1</td><td>Review of major video anomaly detection methods.</td><td>52</td></tr><tr><td>2.2</td><td>PIDS reviewed in chronological order, where columns represent steps of the pipeline and model training needs. <x> denotes unavailability of the step, whereas ✓ denotes that the step is available but not detailed.</td><td>62</td></tr><tr><td>4.1</td><td>Quantitative comparison of thresholding strategies, on View 1 of i-LIDS dataset. S3DCAE stands for proposed autoencoder with frame reconstruction error, MM for min-max scaling, and MZ for moving z-score. EL (10 s) is used as evaluation protocol.</td><td>95</td></tr><tr><td>5.1</td><td>Quantitative comparison with the existing state of the art methods: AUROC (%) for VAD is computed on Ped2, Avenue and Shanghai datasets. Numbers in bold indicate the best performance, and * indicate non-reproducible results.</td><td>111</td></tr><tr><td>5.2</td><td>Influence of different tasks (VCR, FFP and FRP) used during training and testing of AE on Avenue and Shanghai datasets, in terms of AUROC (%).</td><td>113</td></tr><tr><td>5.3</td><td>Influence of tasks (VCR, FFP and PRP) used during training of autoencoder on Avenue and Shanghai datasets, when tested only with VCR, in terms of AUROC (%).</td><td>114</td></tr><tr><td>5.4</td><td>Influence of error measure (MSE, PSE and BPE) on each task (VCR and FFP) during testing of AE on Avenue dataset, in terms of AUROC (%).</td><td>115</td></tr><tr><td>5.5</td><td>PID methods comparison based on edge-level (EL) evaluation at 1, 2 and 10 seconds for View 1 of i-LIDS dataset.</td><td>116</td></tr><tr><td>5.6</td><td>PID methods comparison based on edge-level (EL) evaluation at 1, 2 and 10 seconds for View 2 of i-LIDS dataset.</td><td>117</td></tr><tr><td>5.7</td><td>Comparison (EL (10 s)) of proposed unsupervised PID methods on i-LIDS dataset, taking into account the effect of moving z-score.</td><td>117</td></tr><tr><td>5.8</td><td>PID methods comparison based on edge-level (EL) evaluation at 10 seconds in the Sheeps dataset.</td><td>129</td></tr><tr><td>5.9</td><td>PID methods comparison at different difficulty levels in the Sheeps dataset, where edge-level (EL) evaluation at 10 seconds is used.</td><td>130</td></tr></table>