The understanding of crowd behaviour in semi-confined spaces is an important part of the design of new pedestrian facilities, for major layout modifications to existing areas and for the daily management of sites subject to crowd traffic. Conventional manual measurement techniques are not suitable for comprehensive data collection of patterns of site occupation and movement. Real-time monitoring is tedious and tiring, but safety-critical. This article presents some image processing techniques which, using existing closed-circuit television systems, can support both data collection and on-line monitoring of crowds. The application of these methods could lead to a better understanding of crowd behaviour, improved design of the built environment and increased pedestrian safety. by Anthony C. Davies, Jia Hong Yin and Sergio A. Velastin Department of Electronic and Electrical Engineering, King's College London, United Kingdom Although crowds are made up of independent individuals, each with their own objectives and behaviour patterns, the behaviour of crowds is widely understood to have collective characteristics which can be described in general terms. For example, descriptions such as 'an angry crowd', a 'peaceful crowd' etc. are well accepted. The use of terms such as "mob", "mob rule", have at least since Roman times, carried an implication that a crowd is something other than the sum of its individual parts and that it may possess behaviour patterns which differ from the behaviour expected individually from its participants. It might be assumed that if there is some scientific basis for the study of crowd behaviour, it must belong properly within the social sciences and psychology, and that the physical sciences and engineering have no business in getting involved with such studies. However, electronic engineers have an understanding of field theory and of flow dynamics which may provide insight into characteristics of crowd behaviour, and can also provide the expertise to suggest solutions to crowd monitoring and control based on technological developments in image processing and image understanding. In the search for a scientific understanding of crowd behaviour there is a partial analogy with achievements in the study of gases. A gas is made up of individual molecules, moving about more-or-less independently of one another, with differing velocities and directions. However, the ideal-gas theory provides an extremely accurate basis for predicting the properties and behaviour of gases over a wide range of conditions, without IEE Electronic and Communications Engineering Journal, Vol. 7, No. 1 (Feb), IEE Electronic and Communications Engineering Journal, Vol 7, No. 1, pp.37-37, 1995 2 taking any detailed consideration of the behaviour of individual molecules. This theory represents a major scientific achievement which could hardly have occurred if the notion had prevailed that the equations of motion for each individual molecule have to be solved in order to predict the overall behaviour of a gas in any particular situation. Of course, the individuals making up a crowd are much more variable and complex than the molecules making up a gas, but it is at least reasonable to hope that some mathematical rules of behaviour might be derived which could usefully approximate the behaviour of real crowds. The observation of commuters emerging from a busy station concourse during a morning rush-hour, and the manner in which such a crowd flows past fixed obstacles, must remind electrical engineers of the behaviour of charged particles moving under the influence of an electrical field. As the crowd density and velocity increase, changes can be observed from what may resemble laminar flow to something having some characteristics of turbulence, providing an analogy with fluid flow in aerodynamics. This reinforces the common-sense idea that major obstacles should be designed to have smoothly rounded corners and suggests that in the design of such concourses something could be learned from the flow of air around an aerofoil. If some individuals are moving in the opposite direction (like particles of the opposite charge) collisions may be observed occasionally if the crowds are moving too fast or if some individuals have too much inertia (for example, individuals pushing overloaded baggage trolleys). Particularly in recent years, television news reporting has presented images of huge crowds demonstrating peacefully for political change, while also bringing to our attention the dangers of large crowds unable to achieve their collective objectives (for example, resulting in disasters at football matches) and dangers arising from failures in controlling their movements in an orderly way when trying to evacuate areas subject to terrorist threats or fire. The management of crowds at football matches, pop concerts, carnivals, airports and even in the day-to-day movement of commuters in and out of large cities is a substantial problem with serious consequences for human life and safety and for public order if it is not managed successfully. There are two aspects to this problem to which a scientific understanding of crowd behaviour could contribute. One is in the design of the environments in which crowds are expected to arise and the other is in the real-time monitoring and control of crowds within existing, typically urban, structures. There is, of course, some need to manage crowds on open areas (e.g. refugees congregating in rural regions) but the lack of a built environment of constraining buildings make the problems rather different and more related to the distribution of food and medical assistance. Journal, Vol. 7, No. 1 (Feb), IEE Electronic and Communications Engineering Journal, Vol 7, No. 1, pp.37-37, 1995 3 IEE Electronic and Communications Engineering The development of models of crowd behaviour would provide a basis for informing architects and townplanners. Such models could also supplement existing methods of prediction through their use in computersimulation to investigate the characteristics of crowd flow through proposed designs. It is well known that crowd flow through airport and station concourses is related to the line-of-sight topology [1] of the structure. If there are many obstacles preventing individuals seeing their intended exit or other destination, they may not take to optimum route and may move more slowly. A larger crowd-storage capacity must then be provided to prevent dangerous levels of congestion, with clear implications for the evacuation rate that would be possible (e.g. in conditions of electrical power failure, darkness, and perhaps thick smoke). The free flow of crowds through such man-made facilities is likely to be enhanced by long line-of-sight paths, smoothly contoured obstacles and rounded rather than angular corners in corridors, and a segregation of high-density simultaneous flows in opposing directions. Poorly located exits and entrances from a large open area can give rise to rotational movements in crowds which hinder free-flow and can be dangerous in conditions near to maximum capacity. All this is common-sense and can be taken into account without any quantitative study. However, if aspects of such crowd behaviour could be expressed in the form of mathematical laws, this would usefully supplement heuristic and experience-based design of such facilities, and as mentioned above, would provide a foundation for better computer simulation and hence prediction of the suitability of planned facilities. When space is at a premium (as in most large cities), architects may wish to seek optimum solutions, for which the support of mathematically-based theory could be very helpful. For facilities already in existence, there is an established practice of using extensive closed-circuit television monitoring of crowds. This is used in a number of ways. For example, the television monitors are routinely observed to look for problem conditions. These may be problems arising from various forms of congestion or problems arising from individual incidents not related to a crowd, such as a theft, the outbreak of a fire, or the placing of a bomb by a terrorist. Of course, the cameras installed primarily in response to the latter class of problem (e.g. for what may be collectively be termed 'security') are then available for use for the former class of problems (e.g. for what may be collectively be termed 'crowd safety'). When congestion (crowd density) exceeds a certain level (this level being dependent upon the collective objective of the crowd and the environment), danger may occur for a variety of reasons. Physical pressure may result directly in injury to individuals or to the collapse of parts of the physical environment (for example the collapse of barriers). Alternatively the mood of the crowd may change for the worse if they sense frustration in achieving their objective, whether this objective is to get into a football stadium, to fight with the IEE Electronic and Communications Engineering Journal, Vol. 7, No. 1 (Feb), IEE Electronic and Communications Engineering Journal, Vol 7, No. 1, pp.37-37, 1995 4 supporters of the opposing team, or to get home at the end of the working day when some emergency has shut down the commuter transport system throughout a large metropolitan region. It is normal practice in the management of such crowds to control the arrival rate, stopping it completely when the crowd within a particular area has become too great for safety. This applies both to the entry to an auditorium or a football stadium when it is judged to be full to capacity, as well as to entry to a railway or subway station where there is a continual flow due to passengers leaving on departing trains, and where the entry does not need to be permanently stopped. In the latter case, input flow may either be periodically stopped while the internal crowd is reduced by departures or else may be slowed down by reducing temporarily the number or width of entry-doors. Of course, all such approaches lead to the probability of a new crowd congestion problem being created in a different section, formed by those individuals who have been denied access to their target area. These requirements have led to a continuing increase in the number of installed video cameras, so that more and more public areas are subject to surveillance. In some cases, the systems include continual video recordings, usually by some form of time-lapse technique to make the storage requirement economically viable. However, the successful use of these facilities presents some problems. Human observers are normally positioned to watch the TV monitors, and it is often not considered cost-effective to have one monitor per camera and it is even less likely to have one observer per monitor. Therefore a typical observer is confronted with a two dimensional array of monitors, a switch to select the one or ones from which a time-lapse video recording is made and, in some cases, means for a pan and zoom control of various individual cameras. In the case of short-term events (such a pop-concerts and football matches) it is reasonable to assume a high level of concentration by the observers as well as a high level of experience. However, in the routine monitoring at airports, shopping areas and station concourses, the observers are likely to lose concentration, and since significant events are very infrequent, these may either not be observed at all, or else only when it is too late to take effective action. This situation is aggravated in some cases by management noticing that the observers are not active for the majority of the time, and allocating to them other duties such as routine form-filling tasks, which reduces the rigour of their observations even more. The observers may also be tempted to read newspapers or play cards to relieve their boredom if they are not themselves adequately monitored. It is clear, therefore, that there would be a substantial advantage if some form of image processing could be used on the video images, to automatically spot crowd problems as they arise, and to alert the observersperhaps by triggering a flashing light or an audible alarm -for them to concentrate on dealing with the IEE Electronic and Communications Engineering Journal, Vol. 7, No. 1 (Feb), IEE Electronic and Communications Engineering Journal, Vol 7, No. 1, pp.37-37, 1995 5 incident. The trigger could also be used to automatically start up a time-lapse video recording or perhaps increase the frame rate of an ongoing recording. An overview of a possible system is shown in Fig. 1. The situation is rather different when collecting data to derive information of interest to architects and townplanners. The problem is not time-critical and the processing can be done off-line. The objective is to derive mathematical models of the behaviour of crowds which can form a basis for useful predictions about crowds. For high density crowds, a major difficulty is validation of the models. It is difficult to estimate the actual density or velocity of real crowds. Human observers can be, and are, used to try to determine this data by watching recorded video sequences, but this is a time-consuming and difficult process. There is thus a considerable benefit from being able to develop methods of automatically collecting this data by the use of image processing techniques applied to the video sequences. Provided that such methods can be adequately validated for accuracy by comparison with manual observations or other alternatives, they can be used as a basis for deriving the mathematical models. Our objective for the models is that they should not involve actual counting of individuals or tracking of the movements of individuals but should be based on a collective description of crowds (e.g. analogous to the ideal-gas theory which ignores individual molecules). Therefore, our work has deliberately not attempted to segregate and identify individuals in video images of crowds. If specially designed and located cameras and other electronic equipment could be installed in all the experimental sites, much better data could be collected. However, this would involve far greater cost than using the existing (and increasing) installed base of closed-circuit TV cameras provided primarily for security reasons. Although these cameras are often not located in the best positions for data collection, the authors believe that any solution must take this into consideration, if it going to be widely applicable. Human observers of crowds, particularly those experienced in the management of crowds in public places, can detect many crowd features, in some cases quite easily. Normally they can distinguish between a moving and a stationary crowd and estimate the majority direction and speed of movement of a large crowd, without needing to visually identify or count the separate individuals forming the crowd. They could also easily estimate in a qualitative way the crowd density. The 'mood' of a crowd would also be apparent to a human observer (especially if the vision was augmented by sound). To require image processing and computer vision techniques to match such capabilities of human observers is at present unrealistic. However, study of the methods used by human observers may help in the choice of image processing algorithms likely to be useful in automatic assessment of crowd behaviour. Journal, Vol. 7, No. 1 (Feb), IEE Electronic and Communications Engineering Journal, Vol 7, No. 1, pp.37-37, 1995 6 IEE Electronic and Communications Engineering The development environment consists of equipment for on-site recording (VCR and a camera) and an image processor hosted by a workstation for real-time implementation and algorithmic development (Fig. 2) . The image processor contains a network of transputers and a dedicated frame grabber/processor capable of handling up to four monochrome video signals. Initial development was carried out using peak-time recordings made at Liverpool St. railway station (a major commuter station in London, UK) using two camera positions typical of conventional surveillance CCTV equipment. The video images are digitised at a resolution of 512x512 pixels and 256 grey levels. Fig. 3 shows a typical digitised image with about 17 people within an "area of interest" (AOI), shown by a white rectangle. It is well-established that crowd congestion which is reaching a danger-level can be spotted by observers noting that the up-and-down oscillatory head movements of individuals walking in a freely-flowing crowd stop when the crowd is too dense for free movement. While working with the authors, Hentschel [2] investigated frequency-domain techniques to identify these up-and-down movements to discriminate between stationary and non-stationary flow. A possible alternative is to compute the 2-dimensional Discrete Fourier Transform (DFT) for each image in a time sequence followed by a measurement of temporal changes in the resulting magnitude and/or phase spectra (frequency domain). This approach has two main disadvantages. First, the DFT for a single image is related to local changes of intensity and not to temporal (interframe) properties. Secondly, it involves a high computational and memory cost. A more effective method is to isolate motion properties in the image sequence through a data-reducing coding mechanism, such as the Discrete Cosine Transform (DCT) whose form for a one-dimensional "image" f x t ( , ) of N elements is given by: where k is a constant derived from the maximum signal frequency (Nyquist criterion) and the maximum expected motion to be observed. This transform associates sinusoids with the time-varying parts of an image (faster moving objects are associated with high frequency sinusoids and non-moving objects are associated with constant levels), which can be detected by applying the DFT on g t ( ) . For one-dimensional "images" of N pixels, the DCT calculates a single value for each image in a time-sequence (a data reduction of N to 1), therefore reducing significantly the computational cost of the DFT. As we are only concerned here with detecting movement in the vertical direction, the DCT of a two-dimensional image sequence is given by: IEE Electronic and Communications Engineering Journal, Vol. 7, No. 1 (Feb) where NxN is the image size. In other words, the total amount of motion in each image, weighted by vertical position, is accumulated in g t ( ) . To improve peak detection, the d.c. component of g t ( ) is removed by computing its derivative (difference) to which the DFT is applied. The frequencies of head oscillations correspond to peaks in the resulting frequency spectrum. Fig. 4 shows a typical result for a sequence with 32 images, where the peak at about 2Hz agrees with manual observations. Human observers could be expected to estimate crowd density by counting individuals, making use of their ability to rapidly identify the separate individuals using higher-order knowledge about the shape and characteristics of humans. For example the heads of people in a crowd are easy for a human observer to locate, and so counting heads would be a natural approach. To expect an image processing and computer vision system to identify each individual in a crowd as a preliminary step to counting and tracking is unrealistic at present. Although many sophisticated systems have been devised for military applications (e.g. identifying and tracking multiple targets and decoys) and for some civil applications (vehicle tracking and identification on motorways, pig identification, etc.) there is not yet any realistic possibility of using such methods economically for crowd-behaviour studies. In any case, the main theme of our research is to find general models of crowd behaviour which do not rely upon detecting the behaviour of individuals.
For the last decades, automatic crowd monitoring in urban areas has mainly been performed by means of image processing REF .
Crowd Monitoring Using Image Processing
IIA/SOCO
Engineering