Home Surveillance with AI Facial Recognition
By: Ben Fraser
Tags: Facial Recognition, AWS, DIY, Surveillance

As the quantity of Internet of Things devices in our lives is steadily increasing, so is our potential exposure to unwanted data leakage. This is a major concern moving forward which is driving development in many new unexpected areas; for example blockchain, to enable secure verification of data transmissions. But for now at least, the IoT is a goldmine, or more accurately a datamine of opportunity to anyone who cares to take advantage of it.

I recently purchased two $30 wifi cameras, with two ideas in mind; 1) To monitor our property entrance for security and insurance, and 2) To implement a remote home viewing system responsive to smoke alarms, to increase the safety of our pets while we’re at work.

Out of the box, the cameras could accomplish both of those requirements, and I could have just set them up, one on the corner of the house facing the front gate, and one in the living room to monitor the pets/fires, although we would’ve been getting frequent notifications triggered by the various animals running around, or by my partner and I.

The notification and viewing functionality is provided by a back-end system consisting of a constant video stream from the cameras and a processing service hosted in Shenzhen. The cameras are programmed to lower the sampled framerate when minimal frame to frame change is detected, but they are always transmitting. This poses two concerns: what is happening to my data after it leaves my camera, and what security needs to be relaxed to allow the camera to make its connection back to base?

Obviously if one is concerned about privacy this is absolutely not usable. There is zero user control over the data once it leaves the camera, and no transparency to any of the process.

I identified a few possible solutions:

Stop using the proprietary app, change the online monitoring service to a reputable security monitoring company with secure access. A paid service, and expensive.

Use a local server i.e. my home PC to record and process the feeds, while also providing access from the internet.

Use a cloud service such as AWS to record and process the feed, while also providing access from the internet.

While each of these are workable, they all require the home network to be accessed from the internet, which is not acceptable in many situations, and in the case of setting up a home server, also quite a commitment. Another issue is the security of the device itself. I discovered that it can be accessed with any arbitrary credentials, and the ones I had been prompted to save during setup were completely irrelevant. ID and password harvesting perhaps..

With some development I devised a hybrid solution enabling the automatic processing and notification of the visual events in a controlled and secure manner, which combines the open-source camera software iSpy and number of Amazon Web Services. These include Amazon Rekognition, an enormous AI visual processing engine; Amazon Simple Storage Service (S3), an infinitely scaleable storage system; Amazon Simple Notification Service (SNS), a message delivery and triggering system; and Amazon Lambda, a powerful event driven platform for executing packages of code, called ‘functions’.

I decided to build the security monitoring system first, then adapt it for in-home monitoring. I needed it to record every person who came through the gate, ignoring everything else, check if the person is unknown, then send a notification if so. The concept is as follows:

Camera is restricted to the home network, monitored by iSpy software on a local PC, with recording triggers calibrated accordingly.

Recorded event is uploaded securely and automatically to S3

S3 upload triggers a Lambda function which sends the video to Rekognition for analysis

Rekognition analyses the video, comparing any faces found against an indexed collection of faces, then sends a completion message to SNS

SNS uses the completion message to trigger another Lambda function

This Lambda function retrieves the event data from Rekognition, analyses the response, decides if a notification needs to be sent and what to do with the event data.

Lambda then sends the appropriate notification to SNS which automatically distributes emails or texts to subscribed users

Users can access and view the event securely on the AWS platform from anywhere in the world and respond accordingly.

On my trusty whiteboard it looks like this:

The integration between all the separate systems is tightly controlled by individual permission sets, allowing each service to connect to others on a strict need-to-know basis. This requires careful planning and meticulous implementation. The Lambda functions are executing code I wrote in Python, an extremely versatile language that is supported by Lambda. Each interaction requires completion confirmation, so if any one step fails, it can be automatically fault-analysed and retried for success.

I’ve so far got it to the stage of being a fully operational viable system that serves exactly the purpose I required it to: Automatically detect and record events then notify someone if the event contains a person who isn’t on the whitelist.

The entire thing is very cheap, $30 per camera and around $2.37 so far on compute and storage services despite processing a monumental amount of data during development and debugging. As far as product development and distribution goes, there are a number of paths open from here, but for now I’m just happy to have successfully created an innovative solution that didn’t exist before, and learnt a lot in the process.