A Study of Crowd Abnormal Events Understanding in Surveillance Videos
Abstract
Crowd abnormal events detection in surveillance videos is a common topic in computer vision. For better security and safety, automatic video surveillance systems can detect and record abnormal activities at public and private places. However, traditional methods based on optical flow or segmentation cannot show good detection performance. On the other hand, deep learning based solutions for crowd unusual events detection showed better performance than those of conventional machine learning. This paper includes the latest deep learning models for crowd abnormal events detection in surveillance videos and their overall performance study.
References
. N. Ihaddadene, M. H. Sharif, and C. Djeraba, “Crowd behaviour monitoring”, in Proceedings of the 16th International Conference on Multimedia, Vancouver, British Columbia, Canada, October 26-31, 2008, pp. 1013–1014..
. M. H. Sharif, N. Ihaddadene, and C. Djeraba, "Covariance matrices for crowd behaviour monitoring on the escalator exits", in Advances in Visual Computing, 4th International Symposium (ISVC), Las Vegas, NV, USA, vol. 5359, 2008, pp. 470–481.
. M. H. Sharif and C. Djeraba, "A simple method for eccentric event espial using Mahalanobis metric", in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, 14th Iberoamerican Conference on Pattern Recognition (CIARP), Guadalajara, Jalisco, Mexico, vol. 5856, 2009, pp. 417–424.
. S. Mahmoudi, M. Sharif, N. Ihaddadene, and C. Djeraba, "Abnormal event detection in real time video", in International Workshop on Multimodal Interactions Analysis of Users in a Controlled Environment (ICMI), 2008, pp. 1–4.
. M. H. Sharif, N. Ihaddadene, and C. Djeraba, "Covariance matrices for crowd behaviour monitoring on the escalator exits", in Advances in Visual Computing, 4th International Symposium (ISVC), Las Vegas, NV, USA, vol. 5359, 2008, pp. 470–481.
. R. Mehran, A. Oyama, and M. Shah, "Abnormal crowd behavior detection using social force model", in CVPR, Miami, Florida, USA, 2009, pp. 935–942.
. M. H. Sharif and C. Djeraba, "Exceptional motion frames detection by means of spatiotemporal region of interest features", in Proceedings of the International Conference on Image Processing (ICIP), Cairo, Egypt, 2009, pp. 981–984.
. M. H. Sharif, S. Uyaver, and C. Djeraba, "Crowd behavior surveillance using Bhattacharyya distance metric", in Second International Symposium on Computational Modeling of Objects Represented in Images (CompIMAGE), Buffalo, NY, USA, May 5-7, 2010, pp. 311–323.
. M. H. Sharif, N. Ihaddadene, and C. Djeraba, "Finding and indexing of eccentric events in video emanates", J. Multim., vol. 5, no. 1, pp. 22–35, 2010.
. M. H. Sharif and C. Djeraba, "An entropy approach for abnormal activities detection in video streams", Pattern Recognition, vol. 45, no. 7, pp. 2543-2561, 2012.
. M. H. Sharif, "An eigenvalue approach to detect flows and events in crowd videos", Journal of Circuits, Systems, and Computers, vol. 26, no. 7, pp. 1750110:1-50, 2017.
. M. H. Sharif and C. Djeraba, "PedVed: Pseudo Euclidian distances for video events detection", in Advances in Visual Computing, 5th International Symposium (ISVC), ser. LNCS, vol. 5876, 2009, pp. 674–685.
. M. H. U. Sharif, A. K. Saha, K. S. Arefin, and M. H. Sharif, "Event detection from video streams", International Journal of Computer and Information Technology, vol. 1, no. 1, pp. 108–114, 2011.
. M. H. U. Sharif, S. Uyaver, and M. H. Sharif, "Ordinary video events detection", in Computational Modelling of Objects Represented in Images - Fundamentals, Methods and Applications III, Third International Symposium (CompIMAGE), Rome, Italy. CRC Press, 2012, pp. 19–24.
. F. Galip, M. Caputcu, R. H. Inan, M. H. Sharif, A. Karabayir, S. Kaplan, M. Ozuysal, B. Sengoz, A. Guler, and S. Uyaver, "A novel approach to obtain trajectories of targets from laser scanned datasets", in 18th International Conference on Computer and Information Technology (ICCIT), 2015, pp. 231–236.
. F. Galip, M. H. Sharif, M. Caputcu, and S. Uyaver, "Recognition of objects from laser scanned data points using SVM" in the First International Conference on Multimedia and Image Processing (ICMIP), 2016, pp. 28-35.
. M. Sharif, H. Shehu, F. Galip, I. Ince, and H. Kusetogullari, "Object tracking from laser scanned dataset", International Journal of Computer Science Engineering Techniques, vol. 3, no. 6, pp. 19–27, 2019.
. M. H. Sharif, "Particle filter for trajectories of movers from laser scanned dataset", in Third Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI), ser. Communications in Computer and Information Science, vol. 1144. Springer, 2019, pp. 133–148.
. Z. Su, S. Li, H. Liu, and Z. He, "Tree skeleton extraction from laser scanned points", in IGARSS - IEEE International Geoscience and Remote Sensing Symposium, 2019, pp. 6091–6094.
. M. H. Sharif, "Laser-based algorithms meeting privacy in surveillance: A survey", IEEE Access, vol. 9, pp. 92 394–92 419, 2021.
. M. H. Sharif, F. Galip, A. Guler, and S. Uyaver, "A simple approach to count and track underwater fishes from videos", in 2015 18th International Conference on Computer and Information Technology (ICCIT), 2015, pp. 347–352.
. M. H. Sharif, "A numerical approach for tracking unknown number of individual targets in videos", Digital Signal Processing, vol. 57, pp. 106–127, 2016.
. H. Ahn and H. J. Cho, "Research of multi-object detection and tracking using machine learning based on knowledge for video surveillance system", Pers. Ubiq. Comput., vol. 26, no. 2, pp. 385–394, 2022.
. M. H. Sharif, Sundry Applications and Computations of sin() & cos(). Richardson, TX 75081, USA.: PLOMS LLC., 2021, ISBN 978-1-63802-003-5.
. A. A. Afiq et al., "A review on classifying abnormal behavior in crowd scene", J. Vis. Commun. Image Represent., vol. 58, pp. 285–303, 2019.
. K. Khan,W. Albattah, R. U. Khan, A. M. Qamar, and D. Nayab, "Advances and trends in real time visual crowd analysis", Sensors, vol. 20, no. 18, p. 5073, 2020.
. J. J. P. Suarez and P. C. N. Jr., "A survey on deep learning techniques for video anomaly detection", CoRR, vol. abs/2009.14146, 2020.
. M. B. Braham, J. Weber, G. Forestier, L. Idoumghar, and P. A. Muller, "Recent trends in crowd analysis: A review", Machine Learning with Applications, vol. 4, pp. 1–30, 2021.
. A. B. Chan, Z. J. Liang, and N. Vasconcelos, "Privacy preserving crowd monitoring: Counting people without people models or tracking", in CVPR, Anchorage, Alaska, USA, 2008.
. U. of Minnesota, “Detection of unusual crowd activities in both indoor and outdoor scenes,” 2021, http://mha.cs.umn.edu/proj_events.shtml#crowd.
. A. Adam, E. Rivlin, I. Shimshoni, and D. Reinitz, "Robust real-time unusual event detection using multiple fixed-location monitors", IEEE Trans. PAMI, vol. 30, no. 3, pp. 555–560, 2008.
. J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. F. Fei, "Imagenet: A large-scale hierarchical image database", in CVPR, Miami, Florida, USA, 2009, pp. 248–255.
. C. Lu, J. Shi, and J. Jia, "Abnormal event detection at 150 FPS in MATLAB", in ICCV, Sydney, Australia, 2013, pp. 2720–2727.
. W. Luo, W. Liu, and S. Gao, "A revisit of sparse coding based anomaly detection in stacked RNN framework", in International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 341–349.
. W. Sultani, C. Chen, and M. Shah, "Real-world anomaly detection in surveillance videos", in Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 2018, pp. 6479–6488.
. Y. Hao, J. Li, N. Wang, X. Wang, and X. Gao, "Spatiotemporal consistency-enhanced network for video anomaly detection" Pattern Recognit., vol. 121, p. 108232, 2022.
. K. Lloyd, P. L. Rosin, A. D. Marshall, and S. C. Moore, "Detecting violent and abnormal crowd activity using temporal analysis of grey level co-occurrence matrix (glcm)-based texture measures", Mach. Vis. Appl., vol. 28, no. 3-4, pp. 361–371, 2017.
. F. L. Sanchez, I. Hupont, S. Tabik, and F. Herrera, "Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects", Inf. Fusion, vol. 64, pp. 318–335, 2020.
. F. Zhou, L. Wang, Z. Li, W. Zuo, and H. Tan, "Unsupervised learning approach for abnormal event detection in surveillance video by hybrid autoencoder", Neural Process. Lett., vol. 52, no. 2, pp. 961–975, 2020.
. U. of Minnesota, "ShanghaiTech Campus dataset (Anomaly Detection)", 2021, https://svip-lab.github.io/dataset/campus_dataset.html.
. K. Doshi and Y. Yilmaz, "Rethinking video anomaly detection - a continual learning approach", in IEEE Winter Conference on Applications of Computer Vision (WACV), USA, 2022, pp. 3961–3970.
. U. of Central Florida, "Video Anomaly Dection Dataset", 2021, https://svip-lab.github.io/dataset/campus_dataset.html.
. M. H. Sharif, N. Ihaddadene, and C. Djeraba, "Crowd behaviour monitoring on the escalator exits", in 2008 11th International Conference on Computer and Information Technology, 2008, pp. 194–200.
. V. Mahadevan, W. Li, V. Bhalodia, and N. Vasconcelos, "Anomaly detection in crowded scenes", in CVPR, San Francisco, CA, USA, 2010, pp. 1975–1981.
. G. Tripathi, K. Singh, and D. K. Vishwakarma, "Crowd emotion analysis using 2d convnets", in 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT), 2020, pp. 969–974.
. R. F. Mansour, J. Escorcia-Gutierrez, M. Gamarra, J. A. Villanueva, and N. Leal, "Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model", Image Vis. Comput., vol. 112, p. 104229, 2021.
. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization", in International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2015.
. K. Singh, S. Rajora, D. K. Vishwakarma, G. Tripathi, S. Kumar, and G. S. Walia, "Crowd anomaly detection using aggregation of ensembles of finetuned convnets", Neurocomputing, vol. 371, pp. 188–198, 2020.
. Y. Zahid, M. A. Tahir, N. M. Durrani, and A. Bouridane, "Ibaggedfcnet: An ensemble framework for anomaly detection in surveillance videos", IEEE Access, vol. 8, pp. 220 620–220 630, 2020.
. C. Szegedy,W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions", in CVPR, Boston, MA, USA, 2015, pp. 1–9.
. D. Tran, L. D. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3d convolutional networks", in ICCV, Santiago, Chile, 2015, pp. 4489–4497.
. Z. Hu, L. Zhang, S. Li, and D. Sun, "Parallel spatial-temporal convolutional neural networks for anomaly detection and location in crowded scenes", J. Vis. Commun. Image Represent., vol. 67, p. 102765, 2020.
. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", in 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2015.
. K. Hara, H. Kataoka, and Y. Satoh, "Learning spatio-temporal features with 3d residual networks for action recognition", in ICCV Workshops 2017, Venice, Italy, 2017, pp. 3154–3160.
. F. A. Gers, J. Schmidhuber, and F. A. Cummins, "Learning to forget: Continual prediction with LSTM", Neural Comput., vol. 12, no. 10, pp. 2451–2471, 2000.
. L. Xia and Z. Li, "A new method of abnormal behavior detection using LSTM network with temporal attention mechanism", J. Supercomput., vol. 77, no. 4, pp. 3223–3241, 2021.
. S. Hochreiter and J. Schmidhuber, "Long short-term memory", Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.
. A. N. Moustafa and W. Gomaa, “Gate and common pathway detection in crowd scenes and anomaly detection using motion units and LSTM predictive models,” Multim. Tools Appl., vol. 79, no. 29-30, pp. 20689--20728, 2020.
. H. Shi, X. Xu, Y. Fan, C. Zhang, and Y. Peng, "An auto encoder network based method for abnormal behavior detection", in The 4th International Conference on Software Engineering and Information Management, Yokohama Japan, Y. Li and H. Nishi, Eds., 2021, pp. 243–251.
. M. Hasan, J. Choi, J. Neumann, A. K. R. Chowdhury, and L. S. Davis, "Learning temporal regularity in video sequences", in CVPR, Las Vegas, NV, USA, 2016, pp. 733–742.
. D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. van den Hengel, "Memorizing normality to detect anomaly: Memoryaugmented deep autoencoder for unsupervised anomaly detection", in ICCV, Seoul, South Korea, 2019, pp. 1705–1714.
. F. Yang, Z. Yu, L. Chen, J. Gu, Q. Li, and B. Guo, "Human-machine cooperative video anomaly detection", Proc. ACM Hum. Comput. Interact., vol. 4, no. CSCW3, pp. 1–18, 2020.
. M. Asad, J. Yang, E. Tu, L. Chen, and X. He, "Anomaly3d: Video anomaly detection based on 3d-normality clusters", J. Vis. Commun. Image Represent., vol. 75, p. 103047, 2021.
. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, "Generative adversarial nets", in Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems, Quebec, Canada, 2014, pp. 2672–2680.
. X. Xia, X. Pan, N. Li, X. He, L. Ma, X. Zhang, and N. Ding, "Gan-based anomaly detection: A review", Neurocomputing, 2022.
. Z. Wang, Z. Yang, and Y. Zhang, "A promotion method for generation error-based video anomaly detection", Pattern Recognit. Lett., vol. 140, pp. 88–94, 2020.
. D. Chen, L. Yue, X. Chang, M. Xu, and T. Jia, "NM-GAN: noisemodulated generative adversarial network for video anomaly detection", Pattern Recognit., vol. 116, p. 107969, 2021.
. Y. Tang, L. Zhao, S. Zhang, C. Gong, G. Li, and J. Yang, "Integrating prediction and reconstruction for anomaly detection", Pattern Recognit. Lett., vol. 129, pp. 123–130, 2020.
. P. Isola, J. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks", in CVPR, Honolulu, HI, USA, 2017, pp. 5967–5976.
. Y. Zhong, X. Chen, J. Jiang, and F. Ren, "A cascade reconstruction model with generalization ability evaluation for anomaly detection in videos", Pattern Recognit., vol. 122, p. 108336, 2022.
. W. Liu,W. Luo, D. Lian, and S. Gao, "Future frame prediction for anomaly detection - A new baseline", in CVPR, Salt Lake City, UT, USA, 2018, pp. 6536–6545.
. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation", in CVPR 2015, Boston, MA, USA, 2015, pp. 3431–3440.
. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation", in International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich, Germany, vol. 9351, 2015, pp. 234–241.
. D. Chen, P. Wang, L. Yue, Y. Zhang, and T. Jia, "Anomaly detection in surveillance video based on bidirectional prediction", Image Vis. Comput., vol. 98, p. 103915, 2020.
. C. Park, M. Cho, M. Lee, and S. Le, "Fastano: Fast anomaly detection via spatio-temporal patch transformation", in IEEE Winter Conference on Applications of Computer Vision (WACV),Waikoloa, HI, USA, 2022, pp. 2249–2259.
. J. Redmon, S. K. Divvala, R. B. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection", in CVPR, Las Vegas, NV, USA, 2016, pp. 779–788.
. K. Doshi and Y. Yilmaz, "Continual learning for anomaly detection in surveillance videos", in Conference on Computer Vision and Pattern Recognition, CVPRWorkshops, Seattle,WA, USA, 2020, pp. 1025–1034.
. K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks", in European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, vol. 9908, 2016, pp. 630–645.
. J. Redmon and A. Farhadi, "Yolov3: An incremental improvement", CoRR, vol. abs/1804.02767, 2018.
. L. Shine, V. M. A, and C. V. Jiji, "Fractional data distillation model for anomaly detection in traffic videos", in Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2020, Seattle, WA, USA, 2020, pp. 2581–2589.
. K. Doshi and Y. Yilmaz, "A modular and unified framework for detecting and localizing video anomalies", in IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2022, pp. 3982–3991.
. W. Zhang, G. Wang, M. Huang, H. Wang, and S. Wen, "Generative adversarial networks for abnormal event detection in videos based on selfattention mechanism", IEEE Access, vol. 9, pp. 124 847–124 860, 2021.
. J. T. Zhou, L. Zhang, Z. Fang, J. Du, X. Peng, and Y. Xiao, "Attentiondriven loss for anomaly detection in video surveillance", IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 12, pp. 4639–4647, 2020.
. L. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille, "Attention to scale: Scale-aware semantic image segmentation", in Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 3640–3649.
. F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, "Residual attention network for image classification", in Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 6450–6458.
. A. Vaswani et al., "Attention is all you need", in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, USA, 2017, pp. 5998–6008.
. X. Feng et al., "Convolutional transformer based dual discriminator generative adversarial networks for video anomaly detection", in ACM Multimedia Conference, Virtual Event, 2021, pp. 5546–5554.
. H. Yuan, Z. Cai, H. Zhou, Y. Wang, and X. Chen, "Transanomaly: Video anomaly detection using video vision transformer", IEEE Access, vol. 9, pp. 123 977–123 986, 2021.
. A. Dosovitskiy et al., "An image is worth 16x16 words: Transformers for image recognition at scale", CoRR, vol. abs/2010.11929, 2020.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 PLOMS AI
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PLOMS Journals Copyright Statement
PLOMS LLC. grants you a non-exclusive, royalty-free, revocable license to:
- Academic Journals licenses all works published under the Creative Commons Attribution 4.0 International License. This license grants anybody the right to reproduce, redistribute, remix, transmit, and modify the work, as long as the original work and source are properly cited.
- PLOMS LLC. grants you no further rights in respect to this website or its content.
Without the prior consent of PLOMS LLC, this website and its content (in any form or medium) may not be changed or converted in any manner. To avoid doubt, you must not modify, edit, alter, convert, publish, republish, distribute, redistribute, broadcast, rebroadcast, display, or play in public any of the content on this website (in any form or medium) without PLOMS LLC's prior written approval.
Permissions
Permission to use the copyright content on this website may be obtained by emailing to:
PLOMS LLC. takes copyright protection very seriously. If PLOMS LLC. discovers that you have violated the license above by using its copyright materials, PLOMS LLC. may pursue legal action against you, demanding monetary penalties and an injunction to prevent you from using such materials. Additionally, you may be required to pay legal fees.
If you become aware of any unauthorized use of PLOMS LLC. copyright content that violates or may violate the license above, please contact :
Infringing content
If you become aware of any content on the website that you feel violates your or another person's copyright, please notify [email protected].
