Patent AU-2020202221-A1 (AU)
Title: Privacy preserving visitor recognition and movement pattern analysis based on computer vision
Inventor: SAVJIYANI RIDDHI
IPC: G06T7/292
Filed: 2020-03-28
Abstract: Computer vision analysis techniques, including object detection and facial recognition, are 
applied to images obtained by one or more image capturing devices deployed within a 
physical space. The facial recognition analysis provides facial recognition data based on 
stored employee images. By identifying and separating employees among the people 
present within a physical space, the visitors (non-employees) can be accurately identified. In 
turn, using the computer vision, a variety of functionalities may be enabled based on the 
availability of visitor recognition data. For example, employee identification and elimination 
process may be performed to obtain a net visitor count in a retail space. Similarly, a privacy 
preserving student count can be obtained in school classrooms and libraries. In yet another 
embodiment, continuous human (object) tracking and metadata collection can be used to 
further analyse visitor movement patterns in a physical space. For example, visitor 
movement pattern heatmap may be generated by tracking visitors at an exhibition event 
space. Thereby greatly enhancing efficiency and effectiveness of operations within a 
physical space involving employer, employee and visitor relations.  
Field of the invention 
The instant disclosure relates generally to visitor recognition and movement pattern analysis 
and, in particular, to privacy preserving metadata analysis techniques for visitor recognition 
and movement pattern analysis based on computer vision technology.  
Background of the invention 
Counting number of visitors and tracking their movement through a real-world space 
offers benefits in a variety of areas including retail stores, event spaces and 
corporate locations.  
For example, retail stores have long been trying to gather data that would allow them to 
better understand their visitor count and movement pattern. Such useful information can be 
used to make important decisions about products, services, advertising, store design, and, in 
particular, how customers interact with store layout, make correlations with sales data, 
calculate visitor conversion ratio, product placement locations and much more.  
Unlike online sales, it is difficult to measure visitor traffic, conversion rates and visitor 
movement heatmap in physical stores. This is the reason most stores do not even keep 
accurate count of the number of visitors that walk through their doors. Door monitors, if they 
have them, tend to simply alert the proprietor as to the opening or passing through of a door, 
without any indication as to whether the visitor is entering or leaving the establishment and 
without saving information for later analysis.  
To partially address this issue, a variety of systems have been developed to measure the 
number and direction of people traversing a passage or entrance per unit of time. These 
systems are known variously as "foot traffic counters", "people counters" and the like, and 
use a variety of technologies including infrared (IR) beams, computer vision, thermal 
imaging and pressure-sensitive mats. However, these prior arts are foreign to the idea of 
facial recognition and do not offer any alternative method for employee recognition. As such,
the resulting visitor count includes movement of employees coming in and going out of a 
retail store. Whenever an employee leaves a store to clean outside display windows or to go 
on a short break, the employee re-entries are counted as a new visitor. This results in an 
inaccurate visitor count and provides unreliable statistic relating to an average visitor time in 
a store.  
Previous attempts at building more advanced systems using object detections were a step 
forward compared to those foot traffic counters. These systems used computer vision to 
identify humans and measure their dwell time by tracking the (human) object and matching 
the entry signature of the object to the exist signature of the object. Computer vision 
algorithms have shown to be an effective means for detecting and tracking people. These 
algorithms have also been shown to be effective in analysing the behaviour of people in the 
view of the means for capturing images. This allows the possibility of connecting the visual 
information in the behaviour analysis from the captured video images to the employee 
recognition. However, the prior arts lack the employee recognition techniques that are 
disclosed in the present invention. The above prior arts are foreign to the concept of 
recognizing the employees based on facial recognition on video images.  
Furthermore, various prior arts in facial recognition, object recognition and object tracking 
technologies are implemented using rigid designs that utilize resources inefficiently and 
provide limited functionality, privacy, and automation. For example, when tracking and 
analysing object movements using computer vision, some arts annotate the location of 
objects in a video using bounding boxes, polygons, or masks. However, analysing 
annotations frame by frame may not always be suitable because in some cases a video 
sequence can be several minutes or hours long. At other times, the videos are short, but 
there are thousands of them. This could result in use of resources inefficiently and limit the 
functionalities in some applications. This is where Heatmaps can be very useful. The current 
invention utilizes heatmaps to provide for a quick visual way to analyse the screen location 
of object exposure across an entire video. In one embodiment, the parts of a frame where an 
object appears more frequently (i.e. high exposure areas) are coloured red (hot) and 
conversely, the parts of the video where an object appears infrequently are coloured blue 
(cool).  
The present invention is based on advancements in modern computing, such as EDGE 
computing, computer vision technology and smart cameras. Therefore, it is an objective of 
the present invention to provide novel techniques that automate accurate recognition of 
plurality of visitors by identifying and eliminating employees present at a physical space 
using facial recognition technology. It would be further beneficial if such automated 
techniques could be used to provide more detailed and accurate analysis of visitors by 
analysing their behaviours such as movement patterns.
300 
301 
HUMAN DETECTION 302 
FACIAL RECOGNITION 303 
VISITOR RECOGNITION 304 
PATH TRACKING 
305 
SAVE METADATA 
306 
FIG.3