Handbook on soft computing for video surveillance Preface Video surveillance is the area of computer science devoted to real time acquisition processing and management of videos coming from cameras installed in public and private areas in order to automatically understand events happening at the monitored sites eventually setting up an alarm. Because of the rapidly increasing number of surveillance cameras it has become a key technology for security and safety with applications ranging from fight against terrorism and crime to private and public safety (e.g. in private buildings transport networks town centres schools and hospitals) and to the efficient management of transport networks and public facilities (e.g. traffic lights and railroad crossings). Video surveillance is an extremely interdisciplinary area embracing the study of methods and algorithms for computer vision and pattern recognition but also hardware for sensors and acquisition tools computer architectures wired and wireless communication infrastructures and middleware. From an algorithmical standpoint the general problem can be broken down into several steps including motion detection object classification tracking activity understanding and semantic description each of which poses its own challenges and hurdles for the system designers. Moreover the scope of video surveillance is being extended to off line multimedia analysis systems related to security and safety thus entailing disciplines such as content based video retrieval for visual data similarity retrieval and video mining for knowledge extraction; typical applications are in forensic video analysis and human behaviour analysis