Privacy Protection Camera

A video camera may create an anonymized video stream by detecting people's faces, then anonymizing the faces by pixelating the faces. The camera may be a single housing where the outbound transmissions may be restricted to anonymized content. Some devices may include a secure portal or access mechanism by which authorized users may access raw video prior to the anonymization process, or may be able to receive information that may assist in identifying individual people in the video feed. The authorized users may provide credentials or have some other mechanism to gain access to the sensitive raw video feed. The devices may embed the anonymization routines into hardware or software such that a raw video feed may be unavailable when initially installed.

BACKGROUND

Video and still cameras used in public places may capture people's faces. In many cases, such cameras may be connected to the Internet and may be used for judging the traffic congestion on a street or in a train station, for example. By capturing people's faces, a person's image may be made public. In some jurisdictions, such a transmission may violate the person's privacy.

Personal privacy in today's ubiquitous internet and social media is a large concern to many individuals. People often do not want their images made public, especially when their location or association may infer other information. For example, a video camera may capture images of persons entering or leaving a religious or political organization. Such images may be used to target such people for nefarious means or merely to violate the people's freedom to participate in those organizations.

In the past, people's images on a video or still image may not raise many concerns, but with today's technology, a facial recognition software product may be populated with publicly available databases of people's faces from social media. Such technology is becoming widely available and therefore any public image may be analyzed to identify people in those images.

SUMMARY

A video camera may create an anonymized video stream by detecting people's faces, then anonymizing the faces by pixelating the faces. The camera may be a single housing where the outbound transmissions may be restricted to anonymized content. Some devices may include a secure portal or access mechanism by which authorized users may access raw video prior to the anonymization process, or may be able to receive information that may assist in identifying individual people in the video feed. The authorized users may provide credentials or have some other mechanism to gain access to the sensitive raw video feed. The devices may embed the anonymization routines into hardware or software such that a raw video feed may be unavailable when initially installed.

DETAILED DESCRIPTION

Privacy Protection Camera

Overall Device

A video camera may produce anonymized video feeds by detecting human faces and removing those faces from the video feed. The resulting anonymized video feed may show human faces as pixelized images, blank areas, or some other anonymized representation. The human face detection mechanism and anonymization mechanism may be embedded in the device such that during initial installation, raw or non-anonymized images may not be accessible.

The video camera may be a single device with an image sensor and lens systems, an output mechanism, and a processor that may detect faces and obscure or anonymize the faces prior to transmitting a video feed on the output mechanism. Such an embodiment may be a privacy-enabled camera that operates in such a mode by default, thereby only generating anonymized video streams when installed.

The video camera may be controllable remotely, and may, for example, have a computer-controlled zoom or focus features. Some cameras may also include pan and tilt control. A user may access the remote control features to change the camera angle and field of view.

The remote control may also include remotely configuring the camera device. In some cases, a remote connection may be established to configure the camera to transmit a non-anonymized video stream. Typically, such a configuration may include providing credentials or authenticating a connection prior to being permitted to access the non-anonymized video stream.

It should be noted that throughout this specification and claims, the term “video feed” is used as a general descriptor of the images produced by a camera. In many cases, the video feed may be a motion picture video feed, with images being transmitted continually. In other cases, however, the camera may produce individual images which may be anonymized. The example of a video feed shall be considered to apply to embodiments where individual images may be created.

Camera Interfaces

A privacy enabled camera may have a minimal external interface. Since the camera may be designed for providing anonymized video, the mechanical and software interfaces for the camera may not be overly complex.

The camera's mechanical or mounting interfaces may include one or more mechanisms for securing the camera to a wall, ceiling, vehicle, or other location. The mounting interfaces may include a pan and tilt mechanism by which the camera may be pivoted to change the view.

The camera may include one or more network interfaces. The network interfaces may be wired or wireless. In some cases, one connection may provide anonymized streams, while another connection may provide raw video streams or otherwise provide personally identifiable information regarding the people who may be in an image or video.

A camera's software interfaces may include an administrative interface through which various setup and configuration operations may be performed. Such operations may include configuring a device's network connections, adjusting any parameters concerning the video feed that may be produced, as well as setting up access to any personally identifiable information about people captured in a video feed.

Any access to personally identifiable information, such as people's faces, may be granted access after providing credentials. The credentials may restrict access to personally identifiable information only to those people who are authorized to access such data. In many devices, an anonymized video feed may be accessible without having to provide authenticated credentials, but such credentials may be required prior to accessing personally identifiable information. Some devices may require one set of credentials for the anonymized feed, while requiring a second set of credentials for a non-anonymized feed.

Anonymized and Non-Anonymized Video Feeds

A camera may produce anonymized video feeds, and in some instances, non-anonymized video feeds. Anonymized images may be produced by detecting human faces in an image, then obscuring the faces beyond detection. The resulting image or video feeds of such images, may be obscured such that recreating the original image may be impossible.

Several different types of non-anonymized video feeds may be produced. One non-anonymized video feed may be the raw video feed with no facial recognition or other processing. Another may be a feed that may include images of people recognized in the video feed. Such a feed may be used in parallel with the anonymized video feed for security and other uses. Still another feed may be a metadata feed that may include metadata describing faces recognized in the raw video feed.

The facial detection technology may analyze images within a video stream to detect human faces. Such analyses may be performed in real time or near-real time on a video stream. In many cases, a facial detection algorithm may return a set of coordinates representing the location of the human face within the image. An obfuscation technique may be applied within the coordinates to obscure the face, thereby generating an anonymized face.

A face may be obscured by several mechanisms. For example, a face may be obscured by pixelating or mosaicking the area of the face. Another method may be to replace the face with a solid color, such as grey or flesh tone. The resulting obscured image may not be able to be analyzed to reverse the obscured image to determine the original image.

Some cameras may allow authorized users to access a non-anonymized feed. The non-anonymized feed may be the original video feed prior to anonymization, or may be a separate set of data that may be used to identify people in the video. Some systems may include audit related information. Such information may be in an access log, where actions of an administrator or other authorized user may be tracked and stored. In many such systems, the data accessed by the authorized user may be tracked such that any data breach may be traced back to the person who may have originally accessed the data.

Some systems may perform some analysis of a detected face and may generate information from which a person may be identified. For example, some systems may identify a human face and may capture a representative image of the face. The representative image may be tagged to a frame or time stamp of a video stream, and may include a set of coordinates to determine where in the image that particular face was found.

In another example, a facial detection algorithm may perform some steps of facial recognition by generating a set of measurements that may be used by a facial detection comparison algorithm. In such an example, a camera device may analyze a face from an image and generate a set of features from the face, and those features may be used by a facial recognition system to compare the captured face with a database of known faces. In such an example, the camera may perform a step of characterizing a captured image of a face, then a second step of comparing the characterized parameters against a database may be performed by a second system.

Use Cases

A privacy enabled camera may have several use cases. For example, such cameras may be used for monitoring foot traffic in public areas, maintenance and janitorial inspection, monitoring hospital or healthcare facilities.

Monitoring public areas may be performed using privacy enabled cameras. For many uses, anonymized video feeds may be adequate to perform a function. For example, a camera may be used to monitor the number of people in a specific area and where those people might be going. The density, speed, and direction of foot traffic may be determined without need to identify any of the people, and therefore a privacy enabled camera may fulfil the goal of such analysis. By removing the personally identifiable information from the video feed, analyses such as traffic density may be performed without having to expose people's identities. Such analyses may then be performed on less secure systems and such video feeds may be more widely distributed to others who may perform various analyses on the video feeds.

Cameras in public spaces may, however, also capture information that may be useful to police. For example, if a crime were committed on camera or where a suspect may be fleeing a scene of a crime, the police may have a need to access the personally identifiable information in a video feed to aid in their investigation.

In some jurisdictions, the police or other similar agencies may have legal authority to access live video feeds that may include personally identifiable information. In some situations, a video feed may be stored along with the personally identifiable information, and the personally identifiable information may be accessed after receiving a judicial warrant.

Another use case may be for surveillance systems that may be used in part for maintenance and janitorial reasons. In one such use case, a video camera may be used to inspect the cleanliness of a public area, such as a train station. In such a use case, any people in the video may be unwanted, as the people may block the areas being inspected. Such a use case may block the human faces such that the maintenance inspection may be performed without exposing the maintenance inspector to any personally identifiable information.

A third use case may be in a hospital or healthcare environment. In such an environment, patient's rights to medical privacy may place restrictions on who can view raw images that may contain a person's face. By generating an anonymized video stream, the video stream may serve many purposes without sacrificing patient's privacy.

For example, a hospital may install privacy enabled cameras in hallways for general security monitoring. If a patient were to fall, for example, a security guard may be able to alert the medical staff that a patient needs assistance. However, the security guard may be unable to tell who the patient may be, thereby ensuring that the patient's privacy may not be violated.

However, a hospital camera system may have certain circumstances where identifying people in a security camera may be valuable. For example, a diagnosis of a highly infectious disease in a hospital may trigger a need to identify those people, including staff and patients, who may have had contact with the disease. In such a case, the video may be analyzed to identify each person in the vicinity of the disease and alert those people or contain them to prevent further spread of the infectious disease.

In the examples of the use cases, a privacy enabled camera may generate anonymized video feeds that may be used for a wide variety of uses without encumbering those uses. Because the video feeds may be anonymized, any privacy restrictions that may have limited dissemination and use of the video feeds may be lifted. The anonymized video feeds may be more widely available to different agencies, companies, or individuals for their consumption than the non-anonymized video feeds, and thereby more uses of the video may be found while maintaining a higher level of privacy for the general public.

Some use cases may include various audit mechanisms, which may log any authenticated users and their accesses of non-anonymized video. Some systems may use various authentication mechanisms, such as public key infrastructure (PKI) authentication, where the authenticated user may be identified and their operations tracked. Such systems may allow auditors to trace accesses and content of non-anonymized content.

In the specification and claims, references to “a processor” include multiple processors. In some cases, a process that may be performed by “a processor” may be actually performed by multiple processors on the same device or on different devices. For the purposes of this specification and claims, any reference to “a processor” shall include multiple processors, which may be on the same device or different devices, unless expressly specified otherwise.

Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by an instruction execution system. Note that the computer-usable or computer-readable medium could be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, of otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

FIG. 1is a diagram illustration of an embodiment100showing a privacy enabled camera. Embodiment100may be a camera that may be mounted in public areas where the faces of people in a video stream may be anonymized to protect the public's privacy.

A camera102may represent a conventional video camera but with an internal processing capability to detect human faces then obscure the faces. The camera102may operate in an anonymizing mode as a default or standard configuration. In some cases, the camera102may be able to generate and transmit non-anonymized video feeds or other information that may allow for a person to be identified from the video.

The camera102may be configured to send anonymized video feeds by default. In such a configuration, the camera may be installed, turned on, and may insure people's privacy without any further configuration. In some cases, a camera may have an administrative user interface by which a user may be able to configure non-anonymized feeds which may be accessed after proper authentication.

The camera102may have a power connection104and network connection106. In many cases, the camera102may have a very limited set of interfaces such that unauthorized access to a non-anonymized video stream may be difficult or impossible to access without credentials. Some cameras may be equipped with a reset switch114or other limited user interface. Such cameras may be accessible through the network connection106where a user may receive anonymized video feeds, and when properly authenticated, may access an administrative user interface where the camera102may be configured.

The network connection106may be illustrated as a hard-wired network connection, although some embodiments may use a wireless connection. In some cases, two or more network connections may be present. Some such embodiments may have one network connection used for non-anonymized video while another network connection may be used for anonymized video.

A zoom mechanism108, as well as a tilt mechanism110and pan mechanism112may be remotely controllable adjustments. A zoom mechanism108may allow a user to remotely zoom in and out, and the tilt and pan mechanisms may allow a user to direct the camera's image in a desired location. Some versions of the camera102may not include some or all of the remotely controllable adjustments.

The operation of the camera102may capture a video feed from an image sensor116, perform facial detection118, anonymize the faces using a face anonymizer120, and generate an anonymized video feed122which may be encoded by a video encoder126. In some cases, a raw video feed124may be generated, which may be accessible to authorized users after encoding by a video encoder128.

Some such designs may include a single printed circuit board where a camera sensor may be connected to a processor that may perform facial detection and obscuration prior to a video feed leaving the printed circuit board. Such a design may ensure that no mechanical interface may be available to access a raw video feed. Such designs may still permit an authorized user to configure access to a raw video feed or other personally identifiable information about people captured in the video feed.

FIG. 2is a diagram of an embodiment200showing components in a camera system, as well as various other components that may be accessed over a network connection. Embodiment200may be an example architecture that may perform the functions of a privacy enabled camera, although other embodiments may have different architectures and different configurations.

The diagram ofFIG. 2illustrates functional components of a system. In some cases, the component may be a hardware component, a software component, or a combination of hardware and software. Some of the components may be application level software, while other components may be execution environment level components. In some cases, the connection of one component to another may be a close connection where two or more components are operating on a single hardware platform. In other cases, the connections may be made over network connections spanning long distances. Each embodiment may use different hardware, software, and interconnection architectures to achieve the functions described.

Embodiment200illustrates a device202that may have a hardware platform204and various software components. The device202as illustrated represents a conventional computing device, although other embodiments may have different configurations, architectures, or components.

In many embodiments, the device202may be a server computer. In some embodiments, the device202may still also be a desktop computer, laptop computer, netbook computer, tablet or slate computer, wireless handset, cellular telephone, game console or any other type of computing device. In some embodiments, the device202may be implemented on a cluster of computing devices, which may be a group of physical or virtual machines.

The hardware platform204may include a processor208, random access memory210, and nonvolatile storage212. The hardware platform204may also include a user interface214and network interface216.

The random access memory210may be storage that contains data objects and executable code that can be quickly accessed by the processors208. In many embodiments, the random access memory210may have a high-speed bus connecting the memory210to the processors208.

The nonvolatile storage212may be storage that persists after the device202is shut down. The nonvolatile storage212may be any type of storage device, including hard disk, solid state memory devices, magnetic tape, optical storage, or other type of storage. The nonvolatile storage212may be read only or read/write capable. In some embodiments, the nonvolatile storage212may be cloud based, network storage, or other storage that may be accessed over a network connection.

The user interface214may be any type of hardware capable of displaying output and receiving input from a user. In many cases, the output display may be a graphical display monitor, although output devices may include lights and other visual output, audio output, kinetic actuator output, as well as other output devices. Conventional input devices may include keyboards and pointing devices such as a mouse, stylus, trackball, or other pointing device. Other input devices may include various sensors, including biometric input devices, audio and video input devices, and other sensors.

The network interface216may be any type of connection to another computer. In many embodiments, the network interface216may be a wired Ethernet connection. Other embodiments may include wired or wireless connections over various communication protocols.

The software components206may include an operating system218on which various software components and services may operate.

An image sensor220may capture video images. The images may be processed by a facial detection component222, and any faces may be anonymized using a facial anonymization component224, resulting in an anonymized video feed226. An anonymized video feed interface228may be processed by an encoder227, then made available on an anonymized video feed interface228, which may be accessed through a network connection230.

The anonymized video feed interface228may be a user interface accessed through the network interface228through which the anonymized video feed226may be accessed. The interface228may be a software interface where a user may be able to select different parameters about the anonymized video feed226, such as the frame rate frequency, resolution of the images, and other aspects. In some cases, the user may access the interface228to control the camera's zoom, pan, and tilt controls, when available.

The interface228may be accessible with or without authentication. Credentials may be authenticated through an authentication engine238. Some embodiments may permit access to the anonymized video feed without providing authentication credentials, while other embodiments may require authentication prior to access. Some may permit access to an anonymized video feed without credentials, but may require credentials to access various configuration and control capabilities, such as changing camera direction or zoom controls.

Some embodiments may generate a raw video feed232, which may be made available, after encoding with the encoder241, through a secure feed interface234over a network connection236. The raw video feed232may be a video feed where human faces have not been anonymized. In some embodiments, a second, separate network connection236may permit access to a raw video feed232, while in other embodiments, both the anonymized video feed and non-anonymized video feed may be accessed through the same network connection.

Some embodiments may perform facial analysis240to generate a facial identifier feed242, which may be encoded by an encoder241and made available on a secure feed interface234. A facial identifier feed242may include information that may assist in identifying people in a video feed. Embodiments that may include a facial identifier feed242may not generate a raw video feed232.

A facial identifier feed242may include any information that may assist in the identification of a person in a video feed. For example, some facial identifier feeds may include a representative image of the person who may have been anonymized in the anonymized video feed. Such an image may include coordinates and a frame identifier so that the representative image or other identifying information may be tied back to a location in the video feed.

Another facial identifier feed may be a set of parameters that may be performed as part of a facial recognition algorithm. For example, a facial recognition algorithm may generate a set of description parameters that may describe a person's face mathematically. These parameters may be compared against a database of known persons to determine a match. In such a system, the mathematical parameters may be generated by the camera, then made available in a facial identifier feed242.

Many feeds may be encrypted prior to transmitting. The anonymized video feed226may be encrypted prior to transmitting in some cases, while in other cases, the anonymized video feed226may be made available without encryption.

The facial identifier feed242or the raw video feed232may be encrypted prior to transmitting. In some cases, the facial identifier feed242or the raw video feed232may be encrypted and embedded into the anonymized video feed226. The embedding of encrypted information into the anonymized video feed226may allow the anonymized video feed to be widely disseminated, yet while retaining the information to still identify people within the video.

A camera device202may have an administrative interface244, where a user may be able to access various controls, such as zoom, pan, and tilt controls248, amongst other controls. From the administrative interface244, a configuration interface246may be accessed. The configuration interface246may be a mechanism to establish a raw video feed or facial identifier feed, as well as to establish credentials for accessing such feeds.

The device202may be connected to a network250where other devices may also connect. For example, one set of devices252may have applications254that may consume anonymized video streams. These devices252may operate on various hardware platforms256. Similarly, another set of devices258may have applications260that may consume the non-anonymized or secure video streams, and may operate on various hardware platforms262.

The applications254that may consume the anonymized video feeds may include any type of application or use case where the identity of people in the video are immaterial or of secondary nature. For example, a video of a train station may be used to judge the traffic patterns, wait times, congestion, or factors such as the cleanliness of the station. Such use cases may not require the knowledge of a person's identity.

The applications260that may consume the secure video feeds may be those application where a person's identity may be important to the specific task. For example, applications where criminal suspects may be identified may require an ability to positively identify specific people in a video.

An authentication server264may operate on a hardware platform266, and may authenticate credentials for access to the anonymized video feeds or the various secure video feeds.

FIG. 3is a flowchart illustration of an embodiment300showing a power on sequence for a privacy enabled camera. Embodiment300is a simplified method showing the steps performed by a camera during startup.

Embodiment300is an example of the steps that may be performed during the installation of a privacy enabled camera. The power on sequence may be the first operations that occur when a customer installs the camera and turns it on. In the sequence, the camera may be configured to produce anonymized video from the beginning.

A camera may be mounted in block302and turned on in block304. A connection may be established in block306with another device, which may be authenticated in block308. If the authentication is improper in block310, the connection may be disallowed in block312. If the authentication is proper in block310, the camera may encode the video stream in block314and begin transmitting an anonymized video stream in block316.

Some embodiments may permit access to an anonymized video stream without providing credentials and authentication. Such cameras may produce anonymized video streams that may be publicly accessible without authentication, and such a configuration may be acceptable in various jurisdictions where surreptitiously capturing people's faces on video may be illegal or otherwise improper or undesirable.

The sequence of embodiment300may illustrate that a privacy enabled camera may produce anonymized video initially. Such a configuration may be different from conventional cameras where raw video may be initially produced, and if an anonymized video stream may be desired, an anonymization process may be added to the video stream.

The privacy enabled camera may be configured to produce anonymized video by default. Such a camera may require extra configuration to access non-anonymized video, or such non-anonymized video may not be available at all from the camera. Such a camera may ensure the public's privacy by allowing only authorized personnel to access the non-anonymized video, and by only generating anonymized video from the initial setup and power on actions. This may mean that the personnel who install and configure the cameras may not have any access to the non-anonymized video, thereby removing a security risk.

Because the cameras may produce anonymized video by default, the cameras may ensure that people's privacy may be inherently protected. The only video produced with such a camera, at least without proper authenticated access, may be anonymized video. Such cameras may address the public's concern about their personal privacy, as well as any legal or regulatory restrictions on capturing people's images without consent.

FIG. 4is a flowchart illustration of an embodiment400showing a sequence for processing images captured from a camera sensor. Embodiment400is a simplified method showing the steps to anonymize a video stream.

Embodiment400may illustrate a method where people's faces are detected then anonymized. Additionally, some embodiments may create information that may permit authorized users to identify the people whose faces were obscured. The identifying information may include creating a representative image of the obscured person or may include mathematical parameters that may define characteristics used by facial matching algorithms.

The example of embodiment400may illustrate two different methods by which information may be generated that may assist in the identification of people captured in the video. Such information may be embedded into the anonymized video stream or may be transmitted separately from the anonymized video stream.

A sensor may begin collecting video in block402. A first image frame may be received in block404and any faces in the frame may be identified in block406.

For each of the identified faces in block408, the area in the frame representing the face may be identified in block410. The area in the frame representing the face may be anonymized in block412.

Various methods may be used to anonymize the face in a video image. For example, some embodiments may pixelate or mosaic the area of the face. Other embodiments may replace the face with a solid color, and still others may replace the face with an avatar or some other image. In many such methods, the face may be obscured to the point that the face may not be reconstructed from the resulting anonymized image.

In some embodiments that produce only anonymized video streams, the process may return to block408.

In other embodiments where identifiable information may be available to authenticated users with the proper permissions, several different types of identifiable information may be available. Some embodiments may produce a raw video feed, which is not illustrated in this embodiment. Other embodiments may produce a representative image in block414or determine facial characteristic parameters in block420. This identifying information may be encrypted in block416, and may be either embedded in the video stream in block418or a separate stream of identifiable information may be generated in block422.

Once all of the faces are processed in block408, the frame may be transmitted as part of a video feed in block424and the process may return to block404to process another frame.

FIG. 5is a flowchart illustration of an embodiment500showing a method for configuring a camera for non-anonymized access. Embodiment500is a simplified method showing the steps to establish a non-anonymized information feed and define credentials for access to such a feed.

Embodiment500may illustrate a process for configuring access to information from which person's identities may be determined from a video feed. In some cases, a non-anonymized information feed may be a raw video feed where people's faces are not obscured.

A connection may be received in block502as well as credentials in block504. The credentials may be authenticated in block506, but if the authentication fails in block508, the connection may be disallowed in block510.

If the credentials are properly authenticated in block508, a configuration interface may be entered in block512. In many embodiments, the configuration interface may be a software interface that may be transmitted to and displayed on a user's device. One such manner for defining such a user interface may be through HTML, for example.

A user may configure the non-anonymized information feed in block514. Depending on the features available in the device, the user may select the type of non-anonymized information feed, the frequency of transmission, any network connection configurations or protocols, or other variables with the information feed.

A user may establish credentials for accessing the non-anonymized feed in block516. In some cases, the user may determine the type and level of encryption, establish public/private keys for encryption, or perform other types of credential and encryption configuration.

The information feed may be made available in block518. If additional configuration settings may be desired in block520, the process may return to block514. If no additional configuration settings may be desired in block520, the configuration may end in block522.

FIG. 6is a flowchart illustration of an embodiment600showing a method for accessing non-anonymized information from a privacy enabled camera.

Embodiment600is a simplified method for gaining access to non-anonymized information.

Embodiment600may illustrate how a credentialed user may access non-anonymized information from a privacy enabled camera. The camera may be configured such that only authorized and authenticated connections may receive information that may identify or assist in identifying persons captured in a video stream.

A connection may be received in block602along with credentials in block604. The credentials may be authenticated in block606, and if the authentication failed in block608, the connection may be disallowed in block610. If the credentials may be properly authenticated in block608, the non-anonymized feed may be transmitted in block612.