Patent Description:
Access control systems are often used at public and private premises, such as households, commercial buildings, businesses, retail establishments, schools, hospitals and government buildings, to list a few examples. Access control system nodes are then installed at access points of the premises (e.g. front and interior doors of a building) to control access to restricted areas, such as the building itself or to areas within the buildings. The access control systems authenticate identities of (or authorize) individuals and then permit those authenticated individuals to access the restricted areas through the access point.

Historically, the main components of the access control systems were access card readers and possibly door controllers. The access card readers were often installed at the access points and enabled presentation of credentials to obtain access to the restricted areas. Typically, individuals would interact with the access card readers by presenting access cards such as keycards or contactless smart cards to the readers. The access card readers would read the credential information of the keycards and compare the information against a database of authorized individuals to determine if the individuals were authorized to access the restricted areas. If the individuals were authorized, then the access card readers might signal the door controller to unlock doors or not generate alarms, for example.

<CIT> describes an apparatus for acquiring an image for iris recognition using a distance of a facial feature.

<CIT> describes a face authenticating apparatus and an entrance and exit management apparatus.

These traditional access control systems have limitations. In one example, the individuals must specifically present the access cards to the card reader at each access point to ingress / egress restricted areas. Individuals typically must place their access cards such that the access cards either make direct physical contact with the access readers or are within a few inches of the access readers. This formal interaction process can be an inconvenience to users of such systems. The access cards can also be stolen, which enables otherwise unauthorized individuals to obtain access.

On the other hand, operators of modern access control systems have increasingly incorporated additional security components into the access control systems. These additional components often include surveillance cameras that capture image data, video management systems (VMS) that store the image data, and possibly video analysis systems that analyze the image data.

These modern access control systems can even employ image analysis and facial recognition techniques upon the image data to authorize the users. This facial recognition based authorization is in place of or in addition to the key card based authorization. For this purpose, the cameras typically send the image data over the networks (local and/or remote) to the other components of the system for analysis.

In general, according to one aspect, the invention features an access control system as set out in claim <NUM>.

Preferably, the facial signature module determines a highest ranked acceptable facial patch for each of the individuals, and computes a single facial signature for each of the individuals from the highest ranked acceptable facial patch for each of the individuals.

Typically, the facial signature module compares each of the facial patches against one another to determine whether the facial patches are associated with same individuals or different individuals.

In some embodiments, the surveillance cameras include the facial cropper module and the facial signature module.

The access control system might include a local control unit at the access point that includes the facial cropper module and the facial signature module, in other cases.

In some cases, the local control unit includes a cache of the stored facial signatures, and the facial recognition module matches the computed facial signatures to the cache of the stored facial signatures.

The facial recognition module may also execute upon a microcontroller of the local control unit.

The access control system can include a connected services system that is remote to the access control point and that includes a server, wherein the server includes the facial recognition module. In these embodiments, the facial recognition module executes upon a microcontroller of the server.

In one embodiment, the surveillance cameras include the facial cropper module and the facial signature module and send the computed facial signatures to a connected services system that includes the facial recognition module.

In general, according to another aspect, the invention features a method for controlling an access control system as set out in claim <NUM>.

The above and other features of the invention including various novel details of construction and combinations of parts, and other advantages, will now be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular method and device embodying the invention are shown by way of illustration and not as a limitation of the invention. The principles and features of this invention may be employed in various and numerous embodiments without departing from the scope of the invention.

<FIG> shows a distributed access control system <NUM> to which the invention is applicable.

The access control system <NUM> monitors areas near an access point <NUM>. The term "access point" refers to a place through which an individual <NUM> enters or leaves a building or other structure. Examples of access points <NUM> include doors within or around buildings, elevators, and guard stations at a multi-building facility.

In the specific illustrated case, the access point includes a door <NUM> and a door frame of the door <NUM>, with a door lock system <NUM> located between the door <NUM> and the door frame <NUM>. The access point <NUM> is installed in a room <NUM> of a building. Hinges <NUM> rotatably connect the door to the door frame. An individual <NUM> is standing near the door <NUM>.

The access control system has various components. These components include surveillance cameras <NUM>, a local control unit <NUM>, a video management system (VMS) <NUM>, and a connected services system <NUM>. In the illustrated example, two cameras <NUM>-<NUM> and <NUM>-<NUM> are mounted to a wall <NUM> and are located on opposite sides of the door <NUM>. The placement of the cameras <NUM> on opposite sides of the door <NUM> provides both entry and exit access control.

Preferably, multiple cameras <NUM> are placed to capture images of the individuals <NUM> at the access point <NUM>. In the illustrated example, two cameras <NUM>-<NUM> and <NUM>-<NUM> are placed on opposite sides of the access point <NUM> to improve the chance that facial images of the individuals <NUM> are captured by at least one of the cameras <NUM>. The cameras <NUM>-<NUM> and <NUM>-<NUM> have fields of view <NUM>-<NUM> and <NUM>-<NUM> that are pointed to capture at least faces of the individuals <NUM> in image data <NUM>-<NUM> and <NUM>-<NUM>. The cameras also have lenses which enable the cameras <NUM> to capture the facial images of the individuals when the individuals <NUM> are up to <NUM> meters away from the access point <NUM>. This allows time for the access control system to perform facial recognition of the individual before the person physically reaches the access point <NUM>. In one implementation, as many as four cameras <NUM> are placed at the access point <NUM>.

The cameras <NUM> might have additional features. In one example, the cameras <NUM> capture the image data <NUM> in uncompressed, raw video format. The cameras <NUM> include an indicator <NUM> and possibly an intercom <NUM>. The cameras <NUM> are typically <NUM> to <NUM> mega pixel cameras. If required, higher mega pixel cameras can be used at lower frame rates.

The local control unit <NUM> is a computer system that receives and distributes the image data <NUM> from the cameras <NUM> and also controls the door lock system <NUM>. The local control unit <NUM> receives image data <NUM>-<NUM> and <NUM>-<NUM> from cameras <NUM>-<NUM> and <NUM>-<NUM> via cables <NUM>-<NUM> and <NUM>-<NUM>. The local control unit can then distribute the image data <NUM> to other components over local or remote networks. The local control unit <NUM> sends door unlock signals to the door lock system <NUM>.

The cables <NUM> are of different types and enable the exchange of different information between the cameras <NUM> and the local control unit <NUM>. The cables <NUM> are typically coaxial cables or shielded twisted pair cables, in examples. Power, image data <NUM> and control signals are passed through these cables <NUM>.

The local control unit <NUM> is preferably securely located in proximity to the access point <NUM>. In the illustrated example, the local control unit <NUM> is included within a plenum <NUM> above the access point. The plenum <NUM> is located between a ceiling <NUM> of the room <NUM> and a top of the door frame <NUM>.

The local control unit <NUM> sends the image data over a local network <NUM>. The local network <NUM> might be a local area network or wide area network that supports standard and/or proprietary communications protocols. Example communications protocols include TCP/IP and Ethernet. The local network might be a wired network, or a wireless network using WiMax or WiFi, in examples.

The VMS <NUM> is located on the local network <NUM>. The VMS <NUM> receives the image data <NUM> sent from the local control unit <NUM> and stores the image data <NUM>. In one example, the cameras <NUM> stream the image data over the local network <NUM> to the VMS <NUM>. Display devices on the network <NUM> such as monitors can provide multiple live views of the image data <NUM> via suitable network client connections.

The connected services system <NUM> includes a server <NUM>, an authorized user table <NUM> and a facial recognition database <NUM>. The connected services system <NUM> is typically cloud based and the server <NUM> is a low latency cloud server.

The connected services system <NUM> is a remote system that communicates over a leased data connection or private/public network <NUM> with the local control unit <NUM>. The connected services system <NUM> is sometimes administered by separate business entities than the owners and/or occupants of the buildings, which contain the local control unit <NUM>. In one example, the connected services system <NUM> can be administered by a third party and/or an entity providing services to the local control unit. In one example, the network <NUM> is the internet.

A facial signature is a file computed for each individual <NUM> that represents the face of the individual. Each facial signature includes a unique value or unique dataset of values that represent the face of the individual <NUM>. To compute the facial signature for each individual <NUM>, a predetermined facial recognition algorithm or transform scans one or more facial images of the individual <NUM>, and computes the facial signature from the facial images.

The facial signatures support different file formats. The formats can be binary or text based. In one specific example, the facial signatures are stored in Common Biometric Exchange File Format (CBEFF). The CBEFF is a standard format for biometric information of individuals such as facial information, and describes a set of data elements necessary to support biometric technologies in a common way. In another specific example, the facial signatures are stored in ISO/IEC <NUM>-<NUM> format. ISO/IEC <NUM>-<NUM> is an international format that defines a standard scheme for codifying data describing human faces, within a CBEFF-compliant data structure. In other specific examples, the facial signatures might be stored using any of the following United States American National Standards Institute (ANSI)/ National Institute of Standards and Technology (NIST) formats: ANSI/NIST- ITL <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>, ANSI/NIST- ITL <NUM>-<NUM>, and National Information Exchange Model (NIEM). Use of these formats promotes standardized biometric data interchange and interoperability of biometric-based application programs such as the facial recognition algorithms, in examples. Additionally, proprietary formats for storing the facial signatures might be used.

The facial recognition database <NUM> includes entries <NUM>-<NUM>. <NUM>-N typically for each individual (e.g. employee, worker) registered with the access control system. In some cases, the database <NUM> further includes entries for other individuals such as non-employees, visitors and otherwise. Individuals registered with the access control system may or may not be authorized users, however.

The entries <NUM> include stored facial signatures <NUM> and user identity information <NUM> for each user, if known. In more detail, the user identity information <NUM> might include a user name, employee number, visitor badge number, or credential that is unique to each user. In this way, the user identity information <NUM> in each entry <NUM> can be used to identify the individual corresponding to the stored facial signature <NUM>.

The authorized user table <NUM> includes user records <NUM> for each authorized user. The user records <NUM> include information for authorizing the users in the facial recognition database <NUM>. In examples, the user records <NUM> include information such as the user identity information <NUM> and an authorization level (e.g. allow or deny access).

In general, the individuals <NUM> are originally registered as users and then authorized when the individuals are hired as employees of a business or other entity at which the access control system <NUM> is installed. A guard or security operator at the connected services system <NUM> creates at least one entry <NUM> in the database <NUM> for each individual/user. Then, the security operator creates an associated user record <NUM> for each user/individual in the authorized user table <NUM>.

The stored facial signatures <NUM> are preferably computed using predetermined facial recognition algorithms or transforms. The predetermined algorithms scan facial images of the individuals <NUM> and convert the facial images to the facial signatures <NUM> in response. The algorithms also preferably create the facial signatures <NUM> in a predetermined or pre-agreed upon file format (e.g. CBEFF or NIEM).

The access control system <NUM> generally operates as follows. The cameras <NUM> send the image data <NUM> over the cables <NUM> to the local control unit <NUM>. The local control unit forwards the image data <NUM> over the local network <NUM> for storage at the VMS <NUM>. The cameras <NUM> or the local control unit <NUM> then perform image analysis on the image data. The image analysis determines facial information of individuals <NUM> at the access point <NUM>. Then, either the local control unit or the connected services system <NUM> perform facial recognition operations upon the facial information.

The facial recognition operations determine whether the individuals <NUM> are identified and authorized users of the access control system. The facial recognition operations first match the facial information determined by the cameras <NUM> or the local control unit <NUM> against the facial recognition database <NUM> to obtain an associated identity for the individuals. The identity information of the individuals are then compared to the authorized user table <NUM> to determine whether the identified individuals are authorized users of the access control system <NUM>.

<FIG> shows a side view of the access control system <NUM> in <FIG>. Only camera <NUM>-<NUM> is shown.

By way of a specific example, two individuals <NUM>-<NUM> and <NUM>-<NUM> are at the door <NUM>. Individual <NUM>-<NUM> is standing, and individual <NUM>-<NUM> is in a wheelchair <NUM>. The field of view <NUM>-<NUM> of the camera <NUM>-<NUM> is pointed to enable the camera <NUM>-<NUM> to capture facial views of both individuals <NUM>-<NUM> and <NUM>-<NUM>.

<FIG> illustrates the first embodiment of the access control system <NUM>-<NUM>.

This embodiment distributes image analysis and facial recognition between the local control unit <NUM> and the connected services system <NUM>. The cameras <NUM> typically send the image data <NUM> directly to the local control unit <NUM> for analysis. For this reason, relatively inexpensive cameras <NUM> can be used.

In the illustrated example, the local control unit <NUM> includes various components. These components include various interfaces, modules <NUM>, a microcontroller <NUM>, an operating system <NUM>, and a memory <NUM>.

The interfaces include a bidirectional audio interface <NUM>, a camera interface <NUM>, a door lock interface <NUM>, and a network interface <NUM>.

A number of modules <NUM> are shown. These modules <NUM> include a facial detector module <NUM>, a facial cropper module <NUM>, a facial signature module <NUM>, and a machine learning and classifier module <NUM>. The modules might be software or firmware modules.

The modules <NUM>, the operating system <NUM>, and the microcontroller <NUM> are shown in a stack-like arrangement. The modules <NUM> are on top of the operating system <NUM>, and the operating system <NUM> is on top of the microcontroller <NUM>. This arrangement is due to the fact that the operating system <NUM> operates as an intermediary between the modules <NUM> and the microcontroller <NUM>.

The operating system <NUM> facilitates operation of the modules <NUM> and schedules their execution on the microcontroller <NUM>. Specifically, the operating system <NUM> passes information between the microcontroller <NUM> and the modules <NUM>, loads instructions of the modules <NUM> into the memory <NUM>, and schedules the modules <NUM> for execution upon the microcontroller <NUM>.

The microcontroller <NUM> communicates with each of the interfaces to enable the local control unit <NUM> to communicate and exchange information with other components of the access control system <NUM>-<NUM>. Via the cable <NUM> for each camera <NUM>, the camera interface <NUM> receives the image data <NUM> from the cameras <NUM>, sends control signals to the cameras <NUM>, and controls remote power <NUM> for powering the cameras <NUM>. The remote power <NUM> might be based upon alternating current (AC), direct current (DC), power over cable (PoC), or power over Ethernet (PoE/PoE++), in examples.

The door lock interface <NUM> enables the microcontroller <NUM> to control the door lock system <NUM>. For this purpose, the microcontroller <NUM> sends door unlock signals <NUM> via the door lock interface <NUM> to the door lock system <NUM>. The network interface <NUM> enables communication between the local control unit <NUM> and the components that connect to local network <NUM> and the public network <NUM>. The bidirectional audio interface <NUM> sends and receives speech signals <NUM> to/from the intercom <NUM> of camera <NUM>-<NUM>.

In the illustrated example, some of the communications associated with the modules <NUM> are shown. The modules are shown communicating with other modules <NUM>, the network interface <NUM>, and the microcontroller <NUM> via the operating system <NUM>. A virtual communications channel <NUM> is shown between the facial signature module <NUM> and the network interface <NUM>. This channel <NUM> abstracts a communications path that includes the facial signature module <NUM>, the operating system <NUM>, the microcontroller <NUM>, and the network interface <NUM>.

The facial signature module <NUM> and the network interface <NUM> exchange messages over the virtual communications channel <NUM>. These messages include facial recognition request messages <NUM> and facial recognition response messages <NUM>, and user authorization request messages <NUM> and user authorization response messages <NUM>. The facial signature module <NUM> sends the request messages <NUM>,<NUM> via the network interface <NUM> and public network <NUM> to the connected services system <NUM>. In response, the connected services system <NUM> sends the response messages <NUM>,<NUM>. Each facial recognition request message <NUM> has a corresponding response message <NUM>, and each user authorization request message <NUM> has a corresponding response message <NUM>.

The memory <NUM> also includes image quality factors <NUM>. During operation of the access control system <NUM>-<NUM>, one or more modules <NUM> access and use the image quality factors <NUM>.

The access control system <NUM>-<NUM> generally operates as follows. The camera interface <NUM> receives the image data <NUM> from the cameras <NUM>, and forwards the image data <NUM> to the microcontroller <NUM>. The microcontroller <NUM> and the operating system <NUM> provide the image data <NUM> to the modules <NUM>. The modules <NUM> detect faces of the individuals <NUM> in the image data <NUM> and compute facial signatures from the detected faces.

The modules <NUM> first prepare and send the facial recognition request messages <NUM> to the connected services system <NUM>. Specifically, the facial signature module <NUM> includes the computed facial signatures in the request messages <NUM>, and sends the request messages <NUM> via the network interface <NUM> to the connected services system <NUM>.

The connected services system <NUM> receives the request messages <NUM>, determines whether the information in the request messages <NUM> (i.e. the computed facial signatures) closely match any stored facial signatures <NUM>, and sends facial recognition response messages <NUM> back to the local control unit <NUM>.

At the local control unit <NUM>, the facial signature module <NUM> creates and sends the user authorization request messages <NUM> if the content of the facial recognition response messages <NUM> identifies an individual. Specifically, if the facial recognition response messages <NUM> include non-NULL user identity information <NUM> for users, the facial signature module <NUM> creates the user authorization request messages <NUM>, includes the user identity information <NUM> in the request messages <NUM>, and sends the request messages <NUM> to the connected services system <NUM>. If the response messages <NUM> have NULL references for the user identity information <NUM>, no request messages <NUM> are created.

The connected services system <NUM> receives the request messages <NUM>, and sends user authorization response messages <NUM> back to the local control unit <NUM> in response. The response messages <NUM> indicate whether the individuals represented by the user identity information <NUM> in the request messages <NUM> are authorized users.

At the local control unit <NUM>, the facial signature module <NUM> enables access to the door <NUM> when the user authorization response messages <NUM> indicate that the individual <NUM> is an authorized user and authorized to pass through the door. The local control unit <NUM> enables access to the door <NUM> by sending a door unlock signal <NUM> via the door lock interface <NUM> to the door lock system <NUM>. In another implementation, the connected services system <NUM>, via its server <NUM>, might send the door lock signal <NUM> to the local control unit <NUM>.

<FIG> shows detailed operation of and interactions between various modules <NUM> in the access control system. In one example, the modules <NUM> are located in the local control unit <NUM> within the first embodiment of the access control system <NUM>-<NUM> in <FIG>. These modules <NUM> include the facial detector module <NUM>, the facial cropper module <NUM>, the machine learning and classifier module <NUM>, and the facial signature module <NUM>.

Operation of these modules <NUM> is generally as follows.

In step <NUM>, the facial detector module <NUM> receives frames of image data <NUM>. In one example, when the facial detector module <NUM> is located in the local control unit <NUM> within the first embodiment of the access control system <NUM>-<NUM> in <FIG>, the facial detector module <NUM> receives the image data <NUM> from the microcontroller <NUM> of the local control unit <NUM>.

In step <NUM>, the facial detector module <NUM> scans the frames of image data <NUM> and detects faces of individuals <NUM> in each frame of the image data <NUM>. Frames that do not include faces of the individuals <NUM> are dropped. The facial detector module <NUM> then sends the frames that include the detected faces to the facial cropper module <NUM> in step <NUM>. Because the facial detector module <NUM> drops frames that do not include faces of the individuals <NUM>, this limits the amount of information exchanged between the modules <NUM>/<NUM> and improves processing time.

According to step <NUM>, the facial cropper module <NUM> identifies and extracts facial patches of the individuals <NUM> from each received frame. The facial cropper module <NUM> sends the facial patches to the facial signature module <NUM> for more analysis.

In step <NUM>, the facial signature module <NUM> extracts and compares the facial patches across the image data to determine whether the facial patches are associated with same or different individual(s). This operation is performed for the following reason.

An individual <NUM> standing at the access point <NUM> will be captured on multiple, if not many, frames of image data <NUM> from possibly multiple cameras <NUM>. For the purpose of facial recognition, typically only one or several facial patches of an individual <NUM> are required. Thus, the facial signature module <NUM> associates detected face patches in one frame with detected face patches in subsequent frames. In this way, though each individual <NUM> might correspond to a collection of facial patches extracted from multiple frame images, the facial signature module <NUM> will select only one or several facial patches for each individual <NUM>. Thus, the facial signature module <NUM> compares each of the facial patches against one another to determine whether the facial patches are associated with same individuals or different individuals <NUM>.

The facial signature module <NUM> compares the facial patches against the image quality factors <NUM> to determine whether the facial patches are suitable for subsequent facial recognition. To improve the likelihood that the facial patches are suitable for subsequent facial recognition, the machine learning and classifier module <NUM> in step <NUM> trains a classifier on the image quality factors <NUM>. The machine learning and classifier module <NUM> uses machine learning for this purpose. In response to the training, the machine learning and classifier module <NUM> stores any updates to the image quality factors <NUM> to the memory <NUM>.

More detail for the image quality factors <NUM> is as follows. The image quality factors include image blur, motion blur, lighting level, eye detection, and facial pose factors. The image blur factor measures the extent to which the face of an individual <NUM> in an image, such as the facial patch, is in or out of focus. The motion blur factor measures the extent to which the individual is stationary or moving. The eye detection factor measures the extent to which the individual's eyes are open and discernible. The lighting level factor measures the extent to which light is directly shone upon an individual's face, and accounts for shadows and background lighting. The facial pose factor measures the extent to which the face is directly facing a lens of the camera <NUM>, or tilted or angled relative to the lens.

The machine learning and classifier module <NUM> also trains the classifier on the image quality factors <NUM> for the following reasons. In one example, a direct correlation between the combination of the image quality factors <NUM> and facial recognition success is not known. In another example, the facial recognition might be performed at a remote location having possibly different or unknown facial recognition algorithms. These algorithms may be affected differently by the image quality factors <NUM>.

In step <NUM>, the facial signature module <NUM> ranks each facial patch. In one implementation, the facial signature module <NUM> ranks the facial patches by detecting features within each facial patch, and comparing the features against the (now trained) image quality factors <NUM> in the memory <NUM>. The features might include eyes, nose, nostrils, eyebrows, ears, lips, and mouth of the individual <NUM>, scars or other distinguishing features, and possibly facial expressions, in examples. To obtain the overall rank for each facial patch, the facial signature module <NUM> then might add the values returned from the feature comparisons, in one example.

In step <NUM>, for each facial patch, the facial signature module <NUM> determines whether the ranked value for each of the facial patches at least meets a threshold acceptable rank value. Facial patches having a ranked value below this threshold are dropped from the analysis, and facial patches having a ranked value that meet the threshold are designated as acceptable facial patches.

In step <NUM>, for each acceptable facial patch for each individual <NUM>, the facial signature module <NUM> selects one or more acceptable facial patches having the highest ranking(s).

In step <NUM>, for each individual <NUM>, the facial signature module <NUM> computes facial signatures from the acceptable facial patches. When only one highest ranked facial patch was selected in step <NUM>, the facial signature module <NUM> computes a single facial signature for that individual <NUM> from the one highest ranked facial patch.

The access control system generally computes the facial signatures of individuals <NUM> arriving at the access point <NUM> as follows. The facial signature module <NUM> computes the facial signatures from the facial patches. For this purpose, the facial signature module <NUM> preferably uses the same predetermined facial recognition algorithms or transforms that were employed at the connected services system <NUM> when registering the individuals in the facial recognition database <NUM>. The facial signature module <NUM> also computes the facial signatures using the predetermined or pre-agreed upon file format that was employed when creating the facial signatures <NUM> for the users in the facial recognition database <NUM>.

In one implementation, the facial signature module <NUM> ranks the facial patches using the image quality factors <NUM> to determine acceptable facial patches for individuals, and computes the facial signatures from the acceptable facial patches. Moreover, the facial signature module <NUM> might also determine a highest ranked acceptable facial patch for each of the individuals <NUM>, and compute a single facial signature for each of the individuals from the highest ranked acceptable facial patch for each of the individuals <NUM>.

According to step <NUM>, the facial signature module <NUM> creates a facial recognition request message <NUM> for each computed facial signature. The facial signature module <NUM> sends the request message <NUM> to a facial recognition module, and the facial recognition module sends a facial recognition response message <NUM> in response.

<FIG> are diagrams that show exemplary frames of the image data <NUM>-<NUM> and <NUM>-<NUM> from each of the cameras <NUM>-<NUM> and <NUM>-<NUM>, respectfully. The image data <NUM> are used to illustrate operation of various modules <NUM> in the first embodiment of the access control system <NUM>-<NUM> in <FIG>. The image data <NUM>-<NUM> and <NUM>-<NUM> includes facial images of the same individual <NUM> at the access point <NUM>.

In <FIG>, the image data <NUM>-<NUM> includes frames <NUM>-<NUM>-<NUM> and <NUM>-<NUM>-<NUM>. The frames <NUM>-<NUM>-<NUM> and <NUM>-<NUM>-<NUM> include different side views of the same individual <NUM>. In <FIG>, the image data <NUM>-<NUM> include frames <NUM>-<NUM>-<NUM>, <NUM>-<NUM>-<NUM>, and <NUM>-<NUM>-<NUM>. The frames <NUM>-<NUM>-<NUM> and <NUM>-<NUM>-<NUM> include different front views of the individual <NUM>, and frame <NUM>-<NUM>-<NUM> is empty.

During operation of the access control system <NUM>-<NUM>, the various modules <NUM> analyze the image data <NUM> as follows. The facial detector module <NUM> detects facial images of the individual <NUM> in frames <NUM>-<NUM>-<NUM> and <NUM>-<NUM>-<NUM> in the image data <NUM>-<NUM> of <FIG>, and detects facial images of the individual <NUM> in frames <NUM>-<NUM>-<NUM> and <NUM>-<NUM>-<NUM> in the image data <NUM>-<NUM> of <FIG>. The facial detector module <NUM> sends the frames <NUM>-<NUM>-<NUM>, <NUM>-<NUM>-<NUM>, <NUM>-<NUM>-<NUM>, and <NUM>-<NUM>-<NUM> in which facial images have been detected to the facial cropper module <NUM>.

The facial cropper module <NUM> identifies and extracts facial patch <NUM>-<NUM> from frame <NUM>-<NUM>-<NUM>, facial patch <NUM>-<NUM> from frame <NUM>-<NUM>-<NUM>, facial patch <NUM>-<NUM> from frame <NUM>-<NUM>-<NUM>, and facial patch <NUM>-<NUM> from frame <NUM>-<NUM>-<NUM>.

More detail for facial patches <NUM>-<NUM> through <NUM>-<NUM> is as follows. Facial patch <NUM>-<NUM> is a complete side view of the individual's face and includes both eyes, nose and mouth. Facial patch <NUM>-<NUM> is a partial or "cut off" side view of the individual's face. One of the eyes is missing, and only portions of the nose and mouth exist. Facial patch <NUM>-<NUM> is a distinct front view of the individual's face and includes both eyes, nose and mouth. Facial patch <NUM>-<NUM> is a blurred or unevenly lit front view of the individual's face. The eyes of the individual <NUM> are not discernible, due to the individual likely blinking when the image was captured.

The facial detector module <NUM> then sends the facial patches <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM> to the facial signature module <NUM> for further analysis.

The facial signature module <NUM> then ranks the facial patches <NUM> against the image quality factors <NUM> to determine acceptable facial patches for the individuals <NUM>. Here, the facial signature module <NUM> would likely determine that only facial patch <NUM>-<NUM> and <NUM>-<NUM> are acceptable facial patches.

The facial signature module <NUM> would likely determine that only facial patch <NUM>-<NUM> and <NUM>-<NUM> are acceptable facial patches for the following reasons. Facial patches <NUM>-<NUM> and <NUM>-<NUM> include the entire face of the individual, include facial features such as both eyes, the nose and the mouth. The facial patches <NUM>-<NUM> and <NUM>-<NUM> are also in focus without motion blur, and were captured under better than average lighting conditions. In contrast, facial patch <NUM>-<NUM> does not include the entire face of the individual, and the face is missing an important facial feature such as one of the eyes. Facial patch <NUM>-<NUM> is also out of focus, and the eyes of the individual in the facial patch <NUM>-<NUM> are closed. As a result, facial patches <NUM>-<NUM> and <NUM>-<NUM> would likely be acceptable, and patches <NUM>-<NUM> and <NUM>-<NUM> would be unacceptable.

If the facial signature module <NUM> were to select only one highest ranked acceptable facial patch for the individual <NUM>, the facial signature module <NUM> would likely select facial patch <NUM>-<NUM>. This is because the facial signature module <NUM> ranks facial patches having front views of an individual's face, such as facial patch <NUM>-<NUM>, higher than side views such as facial patch <NUM>-<NUM>.

<FIG> shows more detail for the connected services system <NUM> in the first embodiment of the access control system <NUM>-<NUM> in <FIG>.

The server <NUM> of the connected services system <NUM> includes various components. These components include interfaces, the memory <NUM>, a server microcontroller <NUM>, an operating system <NUM>, and various modules <NUM>.

The interfaces include a public network interface <NUM> and a database interface <NUM>. The public network interface <NUM> enables the connected services system to communicate over the public network <NUM>, and the database interface <NUM> enables the server to communicate with the facial recognition database <NUM> and the authorized user table <NUM>.

The modules <NUM> include a facial analytics module <NUM> and a facial recognition module <NUM>. The modules might be software or firmware modules.

The modules <NUM>, the operating system <NUM>, and the server microcontroller <NUM> are shown in a stack-like arrangement. The modules <NUM> are on top of the operating system <NUM>, and the operating system <NUM> is on top of the server microcontroller <NUM>. This arrangement is due to the fact that the operating system <NUM> operates as an intermediary between the modules <NUM> and the server microcontroller <NUM>.

The operating system <NUM> facilitates operation of the modules <NUM> and schedules their execution on the microcontroller <NUM>. Specifically, the operating system <NUM> passes information between the microcontroller <NUM> and the modules <NUM>, loads instructions of the modules <NUM> into the memory <NUM>, and schedules the modules <NUM> for execution upon the server microcontroller <NUM>.

The server microcontroller <NUM> communicates with each of the interfaces to enable the connected services system <NUM> to communicate and exchange information with other components. The public network interface <NUM> forwards the facial recognition request messages <NUM> and the user authorization request messages <NUM> received over the public network <NUM> to the server microcontroller <NUM>. The server microcontroller <NUM> sends the facial recognition response messages <NUM> and the user authorization response messages <NUM> via the public network interface <NUM> over the public network <NUM>. The database interface <NUM> provides information associated with the facial recognition database <NUM> and the authorized user table <NUM> to and from the server microcontroller <NUM>.

In the illustrated example, some of the communications associated with the modules <NUM> are shown. The modules are shown communicating with other modules <NUM>, with the database interface <NUM>, and with the server microcontroller <NUM> via the operating system <NUM>. A database virtual communications channel <NUM> is shown between the facial recognition module <NUM> and the database interface <NUM>. This channel <NUM> abstracts a communications path that includes the facial recognition module <NUM>, the operating system <NUM>, the server microcontroller <NUM>, and the database interface <NUM>.

The connected services system <NUM> processes the facial recognition request messages <NUM> generally as follows. The public network interface <NUM> receives the facial recognition request messages <NUM> sent from the local control unit <NUM>, and forwards the request messages <NUM> to the public network interface <NUM>. The request messages <NUM> include the facial signatures <NUM> of the individuals <NUM> computed at the access point <NUM>. The public network interface <NUM> forwards the facial recognition request messages <NUM> to the server microcontroller <NUM>. The server microcontroller <NUM> provides the request messages <NUM> to the modules <NUM> via the operating system <NUM>. The modules <NUM> at the server <NUM> then determine whether the computed facial signatures in the request messages <NUM> correspond to users registered in the facial recognition database <NUM>.

In more detail, the facial recognition module <NUM> extracts the computed facial signatures from the request messages <NUM>, and executes a lookup <NUM> of the computed facial signatures against the facial recognition database <NUM>. The facial recognition module <NUM> sends the lookup messages <NUM> over the database virtual channel <NUM> to the facial recognition database <NUM>. The facial recognition database <NUM> then compares the facial signatures in the lookup messages <NUM> against the stored facial signatures <NUM> in each of the entries <NUM>-<NUM>.

The facial recognition database <NUM> processes each lookup message <NUM> as follows. The facial recognition database <NUM> matches the facial signature in the lookup message <NUM> against the stored facial signatures <NUM> in the entries <NUM>, and computes a match confidence score.

The match confidence score is calculated for the following reasons. One reason is that the match is usually not an exact one. This is because although the facial images/patches from which the computed facial signatures <NUM> and the stored facial signatures <NUM> might be of the same individual <NUM>, the facial images are different images, captured at different times and under possibly different conditions. Another reason is that there may be multiple stored facial signatures <NUM> for the same individual <NUM>. As a result, the facial recognition database <NUM> computes the match confidence score to determine which of the stored facial signatures <NUM> best match the facial signature in the lookup message <NUM>.

If the confidence score for the "best" matching facial signature <NUM> is below a certain threshold, or no matching signature <NUM> is found, the facial recognition database <NUM> returns a NULL entry <NUM> in response to the lookup <NUM>. Otherwise, the facial recognition database <NUM> returns the entry <NUM> with the closest matching stored facial signature <NUM>.

The facial recognition module <NUM> then creates the facial recognition response messages <NUM> from the entries <NUM> returned in response to the lookup messages <NUM>. Specifically, if a non-NULL entry <NUM> is returned for each lookup <NUM>, the facial recognition module <NUM> extracts the user identity information <NUM> from the entry <NUM> and includes this information in the associated response message <NUM>. Otherwise, the module <NUM> includes a NULL reference in the response message <NUM>.

The facial recognition module <NUM> then sends the response messages <NUM> over the public network to the local control unit <NUM>.

The connected services system <NUM> then processes each user authorization request message <NUM> sent from the local control unit <NUM> as follows. The facial recognition module <NUM> extracts user identity information <NUM> from the request message <NUM>, and includes the user identity information <NUM> in a lookup message <NUM>. The facial recognition module <NUM> then sends the lookup message <NUM> over the virtual database channel <NUM> to the authorized user table <NUM>. The facial recognition module <NUM> then determines whether the user identity information <NUM> (e.g. identifier (ID) ) in the lookup message <NUM> matches a user record <NUM> in the authorized user table <NUM>. If such a user record <NUM> is found, the facial recognition database <NUM> returns the user record <NUM> in response to the lookup <NUM>; otherwise, the facial recognition database returns a NULL record.

The facial recognition module <NUM> receives the response from the lookup message <NUM>, and creates an empty user authorization response message <NUM>. If the lookup <NUM> returned a non-NULL user record <NUM>, the facial recognition module <NUM> sets a Boolean TRUE value within the response message <NUM>. Otherwise, the facial recognition module <NUM> sets a Boolean FALSE value. The facial recognition module <NUM> then sends the response message <NUM> over the public network <NUM> to the local control unit <NUM>.

As a result, the access control system includes: one or more surveillance cameras <NUM> at the access point <NUM> that capture the image data <NUM>, the facial cropper module <NUM>, the facial signature module <NUM>, and the facial recognition module <NUM>. The facial cropper module <NUM> extracts facial patches from the image data <NUM>, and the facial signature module <NUM> computes facial signatures from the facial patches. The facial recognition module <NUM> receives the computed facial signatures from the facial signature module <NUM>, matches the computed facial signatures to stored facial signatures <NUM>, and sends user identity information <NUM> of individuals corresponding to the stored facial signatures to the facial signature module <NUM> when the computed facial signatures match the stored facial signatures.

<FIG> shows more detail for the facial recognition request messages <NUM> and the facial recognition response messages <NUM>. Each response message <NUM> is paired with a corresponding request message <NUM>.

The facial recognition request message <NUM> has a header <NUM> and a payload <NUM> portion. The header <NUM> includes fields such as a message ID <NUM>-<NUM>, a source ID <NUM>, and a destination ID <NUM>. In one example, when the access control system uses IP/Ethernet protocols, the IDs <NUM>/<NUM> are media access control (MAC) addresses. The payload <NUM> portion includes the computed facial signature <NUM> of an individual <NUM> computed by the facial signature module <NUM>.

The facial recognition response message <NUM> also has a header <NUM> and a payload <NUM> portion. The header <NUM> includes a message ID <NUM>-<NUM>, a request message ID <NUM>, a source ID <NUM> and a destination ID <NUM>. The value of the request message ID <NUM> is the same value of the message ID <NUM>-<NUM> in the corresponding request message <NUM>. This is indicated by arrow with reference 59A. In this way, the request and response messages <NUM>,<NUM> are paired to one another.

In the illustrated example, the payload <NUM> portion of the response message <NUM> includes the user identity information <NUM> of an identified user in the database <NUM>, such as "john doe, employee #<NUM>".

The user identity information <NUM> corresponds to a stored facial signature <NUM> in the database <NUM>. This stored facial signature <NUM> closely matched the computed facial signature <NUM> extracted from the corresponding request message <NUM>.

<FIG> shows more detail for the user authorization request messages <NUM> and the user authorization response messages <NUM>. Each response message <NUM> is paired with a corresponding request message <NUM>.

The user authorization request message <NUM> has a header <NUM> and a payload <NUM> portion. The header <NUM> includes fields such as a message ID <NUM>-<NUM>, a source ID <NUM>, and a destination ID <NUM>. In the illustrated example, the payload <NUM> portion includes the same user identity information <NUM> contents as in the payload <NUM> of the facial recognition response message <NUM> in <FIG>.

The user authorization response message <NUM> also has a header <NUM> and a payload <NUM> portion. The header <NUM> includes a message ID <NUM>-<NUM>, a request message ID <NUM>, a source ID <NUM> and a destination ID <NUM>. The value of the request message ID <NUM> is the same value of the message ID <NUM>-<NUM> in the corresponding request message <NUM>. This is indicated by arrow with reference 59B. In this way, the request and response messages <NUM>,<NUM> are paired to one another.

The payload <NUM> portion of the response message <NUM> includes an indication as to whether the user identity information <NUM> in the request message <NUM> was determined to be associated with an authorized user. In one implementation, as shown, the indication is a Boolean value. This value is shown in the payload <NUM> as "matchFound" <NUM>.

<FIG> is a block diagram that shows more detail for the authorized user table <NUM>. The table <NUM> includes user records <NUM> of authorized individuals/users. Exemplary user records <NUM>-<NUM> through <NUM>-N are shown.

Each user record <NUM> typically includes at least the following fields: the user identity information <NUM>, an authorization level <NUM>, an employee number <NUM>, and a supervisor name <NUM>.

<FIG> illustrates a second embodiment of the access control system. This second embodiment is indicated using reference <NUM>-<NUM>.

This embodiment performs the image analysis and facial recognition operations at the local control unit <NUM>. As in the first embodiment of the access control system <NUM>-<NUM> in <FIG>, relatively inexpensive cameras can be used. The local control unit <NUM> includes a copy of the facial recognition database <NUM>, in the form of a facial recognition database cache <NUM>'. The connected services system <NUM> is generally not used in this embodiment.

The local control unit <NUM> has similar components as the local control unit <NUM> in the first embodiment of the access control system <NUM>-<NUM> in <FIG>, and includes additional components. The local control unit <NUM> includes the same interfaces, memory <NUM>, modules <NUM>, microcontroller <NUM>, and operating system <NUM>. In addition, the local control unit includes a facial recognition database cache <NUM>', an authorized user table cache <NUM>', and additional modules <NUM>. Specifically, the local control unit <NUM> additionally includes the modules <NUM> that are in the connected services system <NUM> in the first embodiment of the access control system <NUM>-<NUM> in <FIG>. These modules are the facial recognition module <NUM> and the facial analytics module <NUM>.

The facial recognition database cache <NUM>' and the authorized user table cache <NUM>' are typically copies of the facial recognition database <NUM> and the authorized user table <NUM> at the connected services system <NUM>, respectfully. The facial recognition database <NUM> and the authorized user table <NUM> can update their respective caches <NUM>'/<NUM>' periodically, or an operator can program the contents of the caches <NUM>'/<NUM>' based on security objectives, in examples.

The access control system <NUM>-<NUM> generally operates as follows. The cameras <NUM> send the image data <NUM> via the cables <NUM> to the camera interface <NUM>. The microcontroller <NUM> obtains the image data <NUM> from the camera interface <NUM> and provides the image data for processing by the modules <NUM>. The facial detector module <NUM>, the facial cropper module <NUM>, the facial signature module <NUM>, and the machine learning and classifier module <NUM> operate in a substantially similar fashion as in the first embodiment of the access control system <NUM>-<NUM> in <FIG>.

The facial signature module <NUM> creates and sends each facial recognition request message <NUM> to the facial recognition module <NUM>, and the facial recognition module <NUM> creates and sends each facial recognition response message <NUM> in response. In more detail, the facial recognition module <NUM> receives each request message <NUM>, extracts the computed facial signature <NUM> from the request message <NUM>, and sends the lookup message <NUM> including the computed facial signature <NUM> to the facial recognition database cache <NUM>'. The cache <NUM>' responds to the lookup <NUM> by sending the entries <NUM> having stored facial recognition information <NUM> that closely matches the computed facial signature <NUM>, or by sending a NULL entry <NUM> when no entries match. The facial recognition module <NUM> then extracts the user identity information <NUM> from the matching entry <NUM>, includes the user identity information <NUM> in a response message <NUM>, and sends the response message <NUM> back to the facial signature module <NUM>.

The facial signature module <NUM> then creates the user authorization request messages <NUM> if the content of the facial recognition response messages <NUM> identify individuals (i.e. the payload <NUM> has non-NULL user identity information <NUM>). The facial signature module <NUM> extracts the user identity information <NUM> from the facial recognition response messages <NUM>, and includes this user identity information <NUM> in the user authorization request messages <NUM>.

The facial signature module <NUM> then sends the user authorization request messages <NUM> to the facial recognition module <NUM>.

The facial recognition module <NUM> then processes each user authorization request message <NUM> sent from the facial signature module <NUM>, and creates and sends the user authorization response messages <NUM> in response. The facial recognition module <NUM> extracts the user identity information <NUM> from the request message <NUM>, and includes the user identity information <NUM> in a lookup message <NUM>. The facial recognition module <NUM> then sends the lookup message <NUM> to the authorized user table cache <NUM>'. The cache <NUM>' then determines whether the user identity information <NUM> in the lookup message <NUM> matches a user record <NUM> in the cache <NUM>'. If a matching user record <NUM> is found, the cache <NUM>' returns the user record <NUM> in response to the lookup <NUM>; otherwise, the cache <NUM>' returns a NULL record.

The facial recognition module <NUM> receives the response from the lookup message <NUM>, and creates a user authorization response message <NUM>. If the lookup <NUM> returned a non-NULL user record <NUM>, the facial recognition module <NUM> sets a Boolean TRUE value within the payload <NUM> of the response message <NUM>. Otherwise, the facial recognition module <NUM> sets a Boolean FALSE value. The facial recognition module <NUM> then sends the response message <NUM> back to the facial signature module <NUM>, in response to the associated user authentication request message <NUM>.

If the response messages <NUM> indicate that the individuals <NUM> are authorized, the facial signature module <NUM> sends the door unlock signal <NUM> via the door lock interface <NUM> to the door lock system <NUM>.

<FIG> illustrates the third embodiment of the access control system. This third embodiment is indicated using reference <NUM>-<NUM>. An exemplary camera <NUM>-<NUM> within the access control system <NUM>-<NUM> is shown.

This embodiment distributes image analysis and facial recognition between the cameras <NUM> and the local control unit <NUM>. The cameras are "smart" cameras that can analyze the image data <NUM>. The cameras <NUM>-<NUM> analyze the image data <NUM>, and the local control unit <NUM> performs the facial recognition. As in the second embodiment of the access control system <NUM>-<NUM> in <FIG>, the local control unit <NUM> includes the facial recognition database cache <NUM>'. The connected services system <NUM> is generally not used in this embodiment.

The camera <NUM>-<NUM> has various components. These components include an imager <NUM>, memory <NUM>, a network interface <NUM>, a microcontroller <NUM>, an operating system <NUM>, and modules <NUM>.

The modules <NUM> include a frame capture module <NUM>, the facial detector module <NUM>, the facial cropper module <NUM>, the facial signature module <NUM>, and the machine learning and classifier module <NUM>. The modules might be software or firmware modules.

The microcontroller <NUM> communicates with the network interface <NUM> to enable the camera <NUM>-<NUM> to communicate and exchange information with the local control unit <NUM> and the VMS <NUM>, in examples.

The access control system <NUM>-<NUM> generally operates as follows. At the camera <NUM>-<NUM>, the imager <NUM> creates an image representation, in pixels, of a scene within the field of view <NUM>-<NUM> of the camera <NUM>-<NUM>. The imager <NUM> sends the images via the microcontroller <NUM> to the frame capture module <NUM>, which places the images in frames of the image data <NUM>-<NUM>. The facial detector module <NUM>, the facial cropper module <NUM>, and the facial signature module <NUM> then analyze the image data <NUM>-<NUM> in a substantially similar fashion as in the local control units <NUM> of the first and second embodiments of the access control systems <NUM>-<NUM>/<NUM>-<NUM> shown in <FIG> and <FIG>, respectively.

The facial signature module <NUM> creates the request messages <NUM>,<NUM> and sends the request messages <NUM>,<NUM> via the network interface <NUM> to the local control unit <NUM>. The local control unit then <NUM> creates and sends the response messages <NUM>,<NUM> back to the facial signature module <NUM>.

At the facial signature module <NUM>, if the user authorization response messages <NUM> indicate that the individuals <NUM> are authorized, the facial signature module <NUM> sends the door unlock signal <NUM> via the door lock interface <NUM> to the door lock system <NUM>.

<FIG> shows more detail for the local control unit <NUM> in the third embodiment of the access control system <NUM>-<NUM>.

The local control unit <NUM> has substantially the same components as the local control units <NUM> of the first and second embodiments of the access control systems <NUM>-<NUM>/<NUM>-<NUM> shown in <FIG> and <FIG>. However, the local control unit <NUM> has a different arrangement of modules <NUM>. The modules <NUM> include the facial recognition module <NUM> and the facial analytics module <NUM>.

The local control unit <NUM> generally operates as follows. The camera interface <NUM> receives the request messages <NUM>,<NUM> sent over the cable <NUM>-<NUM> by the camera <NUM>-<NUM>. The microcontroller <NUM> forwards the request messages <NUM>,<NUM> to the modules <NUM>. The modules <NUM> perform the facial recognition in a substantially similar fashion as the connected services system <NUM> in the first embodiment of the access control system <NUM>-<NUM> in <FIG>. The facial recognition module <NUM> creates the response messages <NUM>,<NUM> and the microcontroller <NUM> sends the response messages <NUM>,<NUM> via the camera interface <NUM> and cable <NUM>-<NUM> back to the camera <NUM>-<NUM>.

The facial signature module <NUM> at the camera <NUM>-<NUM> then processes the response messages <NUM>,<NUM> to determine whether the individual is both identified and authorized. If the individual is authorized, the camera <NUM>-<NUM> sends the door unlock signals <NUM> over the cable <NUM>-<NUM> to the camera interface <NUM> of the local controller <NUM>.

The microcontroller <NUM> of the local controller <NUM> receives the door lock signal from the camera interface <NUM>, and forwards the door unlock signal <NUM> via the door lock interface <NUM> to the door lock system <NUM>. In another implementation, the facial recognition module <NUM> at the local control unit <NUM> can send the door unlock signal <NUM> upon determining that that the individual <NUM> is an authorized user.

<FIG> illustrates the fourth embodiment of the access control system. This fourth embodiment is indicated using reference <NUM>-<NUM>. Exemplary camera <NUM>-<NUM> within the access control system <NUM>-<NUM> is shown.

This embodiment distributes image analysis and facial recognition between the cameras <NUM> and the connected services system <NUM>. As in the third embodiment of the access control system <NUM>-<NUM> in <FIG>, the cameras are "smart" cameras that can analyze the image data <NUM>. As in the first embodiment of the access control system <NUM>-<NUM> in <FIG>, the connected services system <NUM> performs the facial recognition. There is no local control unit <NUM> in this embodiment.

The access control system <NUM>-<NUM> generally operates as follows. The camera <NUM>-<NUM> has substantially similar components and operates in a substantially similar manner as the camera <NUM>-<NUM> in the third embodiment of the access control system <NUM>-<NUM> in <FIG>. The camera <NUM>-<NUM> analyzes the image data <NUM>-<NUM>, and creates the request messages <NUM>,<NUM>. The facial recognition module <NUM> then sends the request messages <NUM>,<NUM> to the connected services system <NUM>, which performs the facial recognition to identify the individuals, and then determines whether the identified individuals are also authorized users The connected services system <NUM> creates and sends the response messages <NUM>,<NUM> back to the camera <NUM> in response to the request messages <NUM>,<NUM>.

The facial signature module <NUM> receives the response messages <NUM>,<NUM> over the network interface <NUM> from the connected services system <NUM> and processes the response messages <NUM>,<NUM>. If the response messages <NUM>,<NUM> indicate that the individuals <NUM> are identified and authorized, the facial signature module <NUM> sends the door unlock signal <NUM> via the door lock interface <NUM> to the door lock system <NUM>.

<FIG> shows more detail for the connected services system <NUM> in the fourth embodiment of the access control system <NUM>-<NUM>.

The connected services system <NUM> has substantially similar components and operates in a substantially similar manner as the connected services system <NUM> in the first embodiment of the access control system <NUM>-<NUM> in <FIG>. As there is no local control unit in this embodiment, the server <NUM> of the connected services system <NUM> exchanges the request messages <NUM>,<NUM> and the response messages <NUM>,<NUM> with the camera <NUM>-<NUM>.

If the facial recognition module <NUM> at the connected services system <NUM> determines that the individual <NUM> is authorized, the facial recognition module <NUM> might also send the door unlock signal <NUM>. The camera <NUM>-<NUM> would receive the door lock signal <NUM> over its network interface <NUM>, and the microcontroller <NUM> would forward the door unlock signal <NUM> via the door lock interface <NUM> to the door lock system <NUM>.

<FIG> is a simplified block diagram showing another way that the access control system could be organized.

Here, the camera <NUM> includes the facial recognition module <NUM> and the facial recognition database cache <NUM>'. The local access controller <NUM> includes the authorized user table <NUM>. The camera <NUM> and the local access controller <NUM> connect to and communicate over the local network <NUM>.

In operation, the camera <NUM> executes facial recognition operations to identify users of the access control system. Once the camera <NUM> has identified the users, the camera <NUM> sends the information identifying the users over the local network <NUM> to the local access controller <NUM>. The local access controller <NUM> then determines whether the identified users are also authorized users.

<FIG> is a simplified block diagram showing another way the present access control system could be organized.

Here, the local control unit <NUM> includes the facial recognition module <NUM> and the facial recognition database cache <NUM>'. The local access controller <NUM> includes the authorized user table <NUM>. The local control unit <NUM> and the local access controller <NUM> connect to and communicate over the local network <NUM>. The camera <NUM> connects to the local control unit via cable link <NUM>.

In operation, the local control unit <NUM> executes facial recognition operations to identify users of the access control system. Once the local control unit <NUM> has identified the users, the local control unit <NUM> sends the information identifying the users over the local network <NUM> to the local access controller <NUM>. The local access controller <NUM> then determines whether the identified users are also authorized users.

<FIG> is a simplified block diagram showing yet another way that the present access control system could be organized. Here, the connected services system <NUM> includes the facial recognition module <NUM> and the facial recognition database cache <NUM>'. The local access controller <NUM> includes the authorized user table <NUM>.

The local access controller <NUM> and the camera <NUM> are on the local network <NUM>. The connected services system <NUM> communicates with the camera <NUM> and the local access controller <NUM> via the public network <NUM>.

In operation, the connected services system <NUM> executes facial recognition operations to identify users of the access control system. Once the connected services system <NUM> has identified the users, the local control unit <NUM> sends the information identifying the users over the local network <NUM> to the local access controller <NUM>. The local access controller <NUM> then determines whether the identified users are also authorized users.

Claim 1:
An access control system, comprising:
one or more surveillance cameras at an access point that capture image data;
a facial cropper module that extracts facial patches from the image data;
a facial signature module that ranks the facial patches using image quality factors to determine acceptable facial patches for individuals and computes facial signatures from the acceptable facial patches, wherein each facial signature includes a unique value or unique dataset of values that represent the face of an individual, wherein the image quality factors include an image blur factor, a lighting level factor, a facial pose factor, and an eye detection factor, wherein the image blur factor measures the extent to which a face of an individual in an image is in or out of focus;
a facial recognition module that receives the computed facial signatures from the facial signature module, matches the computed facial signatures to stored facial signatures to obtain identity information of individuals corresponding to stored facial signatures matching the computed facial signatures, and determines whether the individuals are authorized to pass through the access point based on the identity information; and
a lock system for the access point which is unlocked when the facial recognition module determines that the individuals are authorized users and sends an unlock signal to the lock system.