Image anonymization using analytics tool

A mechanism is provided in a data processing system for image anonymization. The mechanism identifies a plurality of items in an image and extracts properties of the plurality of items into one or more data structures. The mechanism queries an analytics tool about whether a combination of properties of the plurality of items identifies a person in the image. Responsive to determining a given combination of properties of the plurality of items identifies a person in the image, the mechanism modifies the plurality of items in the image to form an anonymized image and outputs the anonymized image.

BACKGROUND

The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for image anonymization using an analytics tool.

Data anonymization is a type of information sanitization whose intent is privacy protection. It is the process of either encrypting or removing personally identifiable information from data sets, so that the people whom the data describe remain anonymous. Data anonymization enables the transfer of information across a boundary, such as, for example, between two departments within an agency or between two agencies, while reducing the risk of unintended disclosure, and in certain environments in a manner that enables evaluation and analytics post-anonymization. In the context of medical data, anonymized data refers to data from which the patient cannot be identified by the recipient of the information. The name, address, and full post code must be removed together with any other information which, in conjunction with other data held by or disclosed to the recipient, could identify the patient. Generalization and perturbation are two popular anonymization approaches for relational data.

Increasingly, users of Information Technology (IT) systems are required to handle sensitive information, such as Personal Identifiable Information (PII), other Sensitive Personal Information (SPI), and information in images that is otherwise of a secret, sensitive, or proprietary nature. At times, images containing such information must be saved or presented to third parties with the sensitive information anonymized because of laws, regulatory compliance requirements, or company policy. For example, an insurance company may send an image of an automobile accident to an outsourcing investigator in order to determine which car caused the accident. According to company policy, no identifiable element should be found in the image. Such an image may include a few people with exposed faces, a few cars with exposed license numbers and identifiable models, and a few houses in the background. Even if faces are changed or obscured, there may be elements in the image that can be combined to identify individuals in the image.

SUMMARY

In one illustrative embodiment, a method, in a data processing system, is provided for image anonymization. The method comprises identifying a plurality of items in an image and extracting properties of the plurality of items into one or more data structures. The method further comprises querying an analytics tool about whether a combination of properties of the plurality of items identifies a person in the image. The method further comprises modifying the plurality of items in the image to form an anonymized image responsive to determining a given combination of properties of the plurality of items identifies a person in the image, and outputting the anonymized image.

DETAILED DESCRIPTION

The illustrative embodiments provide mechanisms for anonymizing an image by identifying parts of the image that, in combination, can possibly identify an individual in the image using analytics. A user of the system selects an image to anonymize, the image containing a plurality of items or items that could identify an individual. These items may include people, vehicles, buildings, and the like. The mechanism uses image processing techniques to identify properties of the items. For example, for a person, the mechanism may determine height, hair color, eye color, clothing style, or the like. For a vehicle, the mechanism may determine make, model, vehicle style, condition, color, state of license, and so forth. For a building, the mechanism may determine a residence, a business, a parking structure, a restaurant, or the like. Then, for each combination of item properties, the mechanism determines a probability that the given combination can identify one or more individuals in the image. The mechanism then anonymizes the image by replacing or obscuring the items or changing particular properties of the items in the image. The mechanism then outputs the anonymized image to the user.

As shown inFIG. 1, one or more of the computing devices, e.g., server104, may be specifically configured to implement a mechanism for image anonymization using analytics. The configuring of the computing device may comprise the providing of application specific hardware, firmware, or the like to facilitate the performance of the operations and generation of the outputs described herein with regard to the illustrative embodiments. The configuring of the computing device may also, or alternatively, comprise the providing of software applications stored in one or more storage devices and loaded into memory of a computing device, such as server104, for causing one or more hardware processors of the computing device to execute the software applications that configure the processors to perform the operations and generate the outputs described herein with regard to the illustrative embodiments. Moreover, any combination of application specific hardware, firmware, software applications executed on hardware, or the like, may be used without departing from the spirit and scope of the illustrative embodiments.

It should be appreciated that once the computing device is configured in one of these ways, the computing device becomes a specialized computing device specifically configured to implement the mechanisms of the illustrative embodiments and is not a general purpose computing device. Moreover, as described hereafter, the implementation of the mechanisms of the illustrative embodiments improves the functionality of the computing device and provides a useful and concrete result that facilitates image anonymization using analytics.

As a server, data processing system200may be, for example, an IBM eServer™ System p® computer system, Power™ processor based computer system, or the like, running the Advanced Interactive Executive (AIX®) operating system or the LINUX® operating system. Data processing system200may be a symmetric multiprocessor (SMP) system including a plurality of processors in processing unit206. Alternatively, a single processor system may be employed.

As mentioned above, in some illustrative embodiments the mechanisms of the illustrative embodiments may be implemented as application specific hardware, firmware, or the like, application software stored in a storage device, such as HDD226and loaded into memory, such as main memory208, for executed by one or more hardware processors, such as processing unit206, or the like. As such, the computing device shown inFIG. 2becomes specifically configured to implement the mechanisms of the illustrative embodiments and specifically configured to perform the operations and generate the outputs described hereafter with regard to image anonymization using analytics.

FIG. 3is a block diagram illustrating a system for image anonymization using analytics in accordance with an illustrative embodiment. An extract component302receives a selected image301, which includes a plurality of items, such as a person and a car, for example. Extract component302performs image processing to identify the items and extract properties to form person properties303and car properties304. The image processing techniques may include image sharpening, image smoothing, edge detection, pattern recognition, and other known image processing techniques. Person properties303may include approximate height, body type, hair color, hair style, eye color, clothing items, etc. Car properties304may include make, model, type, color, etc. Other types of items may include other properties. For example, building properties may include type, color, signage, etc.

Analytics tool305receives person properties303and car properties304. Analytics is the discovery and communication of meaningful patterns in data. Especially valuable in areas rich with recorded information, analytics relies on the simultaneous application of statistics, computer programming, and operations research to quantify performance. Analytics often favors data visualization to communicate insight. Specifically, areas within analytics include predictive analytics, enterprise decision management, retail analytics, store assortment and stock-keeping unit optimization, marketing optimization and marketing mix modeling, web analytics, sales force sizing and optimization, price and promotion modeling, predictive science, credit risk analysis, and fraud analytics. Since analytics can require extensive computation (see big data), the algorithms and software used for analytics harness the most current methods in computer science, statistics, and mathematics.

Analytics tool305is a software application, computer system, or service that performs analytics on data based on queries from the image anonymization system. In one embodiment, analytics tool305may be a software application executing on the same computing device as the image anonymization system. In an alternative embodiment, analytics tool305may execute on a different computing device, such as a server, connected to the image anonymization system via a local area network, a wide area network such as the Internet, or other communication means. In another embodiment, analytics tool305is a service executing on the same or a different computing device that performs analytics on data based on queries. For example, analytics tool305may be a service executing in the cloud.

Examples, of an analytics tool is the IBM Watson™ Analytics system available from International Business Machines (IBM®) Corporation of Armonk, N.Y. IBM Watson™ Analytics offers the benefits of advanced analytics without the complexity. A smart data discovery service available on the cloud guides data exploration, automates predictive analytics, and enables effortless dashboard and infographic creation.

Some items, such as a person's face or a vehicle's license plate, are in themselves identifying and prior art solutions recognize these items and mask, obscure, or obfuscate them. Other items, such as type and color of a vehicle, type and color of clothing, type and size of dwelling, items stored on a piece of property, and non-identifying signage may be combined in context to be identifying. The system of the illustrative embodiment identifies these combinations of items and masks, obscures, or obfuscates them.

The context includes at least the image metadata, which may include location, time of day, season, weather, the roles or rights of the viewer, and the combination of image elements that may or may not contribute to identification of one or more people. The context may also include any number of data sources containing demographic and other information about the locality as well as the potentially identifying image elements.

By way of example, map services and satellite imaging may provide an image of a person getting out of a car in front of a house. In one illustrative embodiment, the present system identifies and analyzes elements of the image other than, and in addition to, the car's license plate and the person's face. In another illustrative embodiment, the present system identifies and analyzes elements of the image other than the car's license plate and the person's face, wherein a different system may identify and blur the car's license plate and the person's face. These other elements may include the type and color of the car, items carried in, on, or towed by the car, the person's clothing, the dwelling and any items stored or displayed on the property. The system identifies the location and time of day the image was captured. Using analytics tool305, the system queries a number of relevant data sources for statistical information, including but not limited to:

sales figures by region for the vehicle make and model;

popularity of the vehicle color;

popularity of clothing style and color, and,

demographic information for the location, including income, dwelling types, group membership, etc.

In the depicted example, the data sources may include, for example, weather data311, demographic information312, sales information313, statistics314, and user roles315. Various combinations of data sources, including more or fewer data sources may be used within the scope and spirit of the illustrative embodiment. The goal of the queries is to ascertain whether each given combination of image elements (properties) constitute an “average,” middle-of-the-bell-cure situation or an outlier.

In the case of an outlier, one may conclude that the combination of properties is a property set that could potentially identify an individual. The system of the illustrative embodiment outputs the identified property sets306. Anonymize component307receives the identified property sets306and replaces, blurs, obscures, or obfuscates the image elements in property sets306to form anonymized image308.

Examples of identifying property sets306may include:

make and model of a vehicle that is atypical for the locality (e.g., an expensive sports car in a rural area or a large sports utility vehicle in a European city);

clothing style and color (e.g., trendy, bright colored clothing in a typically conservative area);

signs of group membership (e.g., printed t-shirt or hat) that is atypical for the area;

dress that is inconsistent with the season and weather (e.g., light clothing in cold weather might indicate cold-climate upbringing); or

physical attributes atypical for the area.

In one embodiment, an administrator configures a model of elements to be replaced. The administrator may define relationships between elements, e.g., that car number plates are part of a car. For each element, the administrator assigns an algorithm or service that can identify that element in an image and extract its properties into a well-defined data structure, which contains, for example, the area of each element, color, model, real dimensions, etc. The administrator configures analytics tool305to decide based on textual description of the elements in the image what is the probability of identifying a person based on the elements. The administrator defines which combinations of elements should be checked in order to decide if they could identify the person who uses, owns, or otherwise relates to them in a way that could lead to identification of the person.

A user, such as a third party with a specified user role, selects an image for presentation, storage, or transmission. The system of the illustrative embodiments attempts to identify every item in the image and extract the properties of the items into one or more textual data structures using defined algorithms or services. The system queries all possible combinations, or alternatively predefined combinations, of properties of elements that were extracted from image to build queries. The analytics tool returns for each set of properties whether the set could potentially identify a person. For example, the analytics tool may return a probability that the property set identifies a person, which can then be compared to a threshold. Ifa given set of properties could identify a person, then the system replaces the items with other items or obscures the items.

In one particular example, an image may contain a man with a green polo shirt getting out of a small red sports car near the Eiffel tower. None of the elements alone would identify that person; however, the combination of those elements might identify the man. For instance, the green polo shirt may be have a distinctive style that is easily identifiable as being made by a particular designer and may be sold in only a few cities, and the sports car may be manufactured and sold primarily in one country. In this case, the query for the analytics tool may be, “what are the chances of identifying a man wearing a green polo shirt owning or using a small red sports car near the Eiffel tower?” If the result from the analytics tool is that there is a high chance of identifying the man, then the system replaces those items in the image (or some of them); however, if the result is that there is a low chance of identifying the man, then the system does not replace the items.

In one embodiment, the system starts with single property sets and proceeds with gradually larger sets. If a set is found to be identifying, then all subsequent sets that contain this set could be ignored. The system may replace only the items in an identifiable set that got the highest rank for identifying when tested as a standalone set. After reaching some limit of sets or traversing all of the preconfigured combination size, the system returns the anonymized image.

In an example embodiment, the system anonymizes images in the context of the user's rights and roles. For example, a publicly available satellite imaging system serving up images to unknown users may require a very high degree of protection of the people in the images, whereas an insurance company's or local law enforcement agency's internal, private system serving up images (e.g., auto accident images) may take into account the rights and roles of the particular user as defined by local/relevant legislation.

FIG. 4is a flowchart illustrating operation of a system for image anonymization using analytics in accordance with an illustrative embodiment. Operation begins (block400), and user selects an image for presentation, storage, or transmission (block401). The system uses image processing techniques to identify items in the image (block402) and to extract properties of the items into one or more textual data structures (block403).

The system then determines all possible combinations of item properties (block404). The system may start with single property sets and proceed with gradually larger sets. If a given set is found to be identifying, then all sets that contain the given set may be ignored. The system considers a first combination of item properties (block405) and builds queries for an analytics tool (block406). The analytics tool determines whether the combination identifies a person based on analytics performed using a plurality of data sources (block407). The system determines whether the combination of properties is identifying (block408). If the combination of properties is identifying, then the system obscures the corresponding items in the image (block409).

Thereafter, or if the combination of properties is not identifying in block408, the system determines whether the combination of item properties the last combination (block410). If the system determines that the combination of item properties is not the last combination, then the system considers the next combination (block411), and operation returns to block406to build queries for the analytics tool. If the system determines that the combination of item properties is the last combination in block410, then operation ends (block412).

Thus, the illustrative embodiments provide mechanisms for image anonymization using analytics. The mechanisms leverage weather data, demographic data, sales statistics, other statistics, and user rights and roles, for example, to determine whether combinations of item properties are likely to identify a person in an image. The mechanism then replace, obscure, or obfuscate items in the image corresponding to combinations of item properties that identify one or more people in the image, thus anonymizing the image.