How Motion Detection Video Analytics Works: A Thorough Breakdown
Tags: Video Analytics, Security Camera, White Paper, Machine Learning

Whether manufacturers want to admit it or not, every video analytics package in the video security industry begins with motion detection. The video analytics included with the Ring video doorbell for example are based on motion detection.

What are motion detection video analytics used for?

Motion detection is cheap and easy to implement. Processors today are powerful enough to run motion detection algorithms entirely on the hardware without the need of the cloud. And when the security conditions and requirements are right, it still provides some form of security for your average consumer. This is especially if the person is away from their home, their motion-detecting video feed is running indoors, and the settings have been calibrated right (big ifs).

Background subtraction: The most widely adopted approach to motion detection video analytics

The methods behind motion detection video analytics have been studied for nearly 30 years and the process is well known. The most conventional and widely adopted approach to motion detection — and the one most suited to video analytics for security and surveillance cameras — is called “background subtraction”. The way to understand this method conceptually is to detect moving objects by looking at the difference between the existing frame and a “reference frame” — what the scene is supposed to look like with nothing moving within it.

This approach makes intuitive sense as it sort of reflects how a human actually sees things. Implementing this algorithm consists of three steps:

Background Initiation

You create a reference frame by building the background — what the scene looks like without anything moving in it. This can be done in a variety of ways. You can take a moving average or try other, more complicated methods with arcane names like “fuzzy” and “neuro-inspired”. For 80% of cases however, a simple moving average works fine.

Foreground Detection

This is the meat and potatoes of the background subtraction method. You compare the current (immediate) frame and then do a subtraction. You literally take out the pixels that are the same in between the two frames (see below for an example image).

The blob of pixels remaining — if it is big enough — is thus dubbed “motion”.

Background Maintenance

Scenes are never static. The sun goes up and the sun goes down, shifting the shadows. Someone might move a box and set it down on a table. There are many reasons. If you never change the background reference image then you are going to get false alerts up the wazoo. Thus it is up to the algorithm and the product’s designers to incorporate and digest the changes in the background scene. Such modifications are done here in this step.

The key to making this motion detection method work is in the initiation and maintenance of the background (Steps 1 and 3), as well as the settings for what to bring to the attention of the end user.

Quirks and drawbacks of motion detection video analytics

The conceptual simplicity and elegance of the background subtraction algorithm with regards to motion detection causes issues for it though in the real world. Here are a few of the major issues associated with motion detection video analytics:

Ghosts

In this context, “ghosts” have nothing to do with spirits and the afterlife. This has to do with a logical flaw in the background detection algorithm. A person moves from one place to another and that is correctly identified by the background subtraction algorithm as being “motion”. Great! But wait there’s more. Someone moving from one place to another means that the camera is suddenly now capable of seeing something that the moving object was originally blocking.

If the algorithm is not configured correctly then background subtraction is going to identify this newly revealed thing and flag it as motion too — creating a “ghost”. Much easier to show with an image than to explain with words:

The motion detection video analytics (shown as a green bounding box) flagged this as “motion”. It is clearly not. It is just that this locker had not been visible to the algorithm before the guy started moving.

Ghosts for the most part can be adjusted out with prudent settings management. For CCTV operators it is basically something they can ignore. After all, it only happens when there is actually motion on the screen. However it is good to know that this sort of thing exists.

False alerts

The foremost problem though with motion detection video analytics is false alerts. Because the video is being scanned and reviewed frame by frame on a pixel level, it is very easy to send a whole flurry of false positives solely on the basis of fake “motion” created by differences in pixel brightness from frame to frame. Sometimes these frame by frame differences are created from changes in lighting but other times it might not even be due to that — modern video codecs can cause such distortions as they try to compress the data size to improve transmission times and stay under bandwidth caps.

Settings also have a huge role to play here. Motion detection is dependent on counting and subtracting pixels, it is not actually paying attention to what it is actually seeing. A person, a cat, a car — it is all just a blob of similarly colored pixels to it. And because every scene all across the day is an entirely different set of colored pixel blobs, this leaves it to the system integrator to figure out what sensitivities and triggers work for this particular blob. Ignoring this will cause false alerts.

False positive + false negative in motion detection

In the above frame grab we show a motion detection algorithm that is not performing very well. Its settings are not calibrated for the video scene and as a result, there are alerts on things that we intuitively know are not moving and there are no alerts on things we know are DEFINITELY moving like the passing car. With calibration, we can get to a point where we are getting the right alerts on the right things — but until then the end user is going to be getting false alerts.

Of course, this is a technical explanation that does not very well explain that in the end, your end user or client is going to look at their iPhone and see something like the below:

AAAHHHHHHH

This can be very frustrating for everyone involved, and it is less effective. There is ample evidence that shows that the more false alerts people get, the less likely they are to read the next alert to come regardless of whether it’s true or not.

Learn more about the newest advances in security video analytics

Umbo Computer Vision’s Light A.I. is a step up from motion detection because unlike with the background detection algorithm, it is not blindly subtracting pixels from one frame to another. Light uses neural networks to actually recognize the shapes of humans and their behaviors from one frame to another over time. It’s better than motion detection to say the least.

Real monitoring centers and security teams have been using Light A.I. for their security systems for years now. Interested in learning how to do better than just motion? Say hello today.

Citations

Cucchiara, Rita, et al. “Detecting moving objects, ghosts, and shadows in video streams.” IEEE transactions on pattern analysis and machine intelligence 25.10 (2003): 1337–1342.

Cucchiara, Rita, et al. “Detecting objects, shadows and ghosts in video streams by exploiting color and motion information.” Proceedings 11th international conference on image analysis and processing. IEEE, 2001.

Tian, Ying-Li, and Arun Hampapur. “Robust salient motion detection with complex background for real-time video surveillance.” 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION’05)-Volume 1. Vol. 2. IEEE, 2005.

Rakibe, Rupali S., and Bharati D. Patil. “Background subtraction algorithm based human motion detection.” International Journal of scientific and research publications 3.5 (2013): 2250–3153